Revitalens的隐形眼镜药水好用不# PennySaver - 省钱一族
x*y
1 楼
【 以下文字转载自 Database 讨论区 】
发信人: xnxky (change, change, change), 信区: Database
标 题: About the latency of reading from mysql binlog
发信站: BBS 未名空间站 (Wed Nov 18 14:05:32 2015, 美东)
Hi, friends, I have a very urgent problem about mysql binlog latency, and
any help is greatly appreciated!
Here is my scenario:
We use ec2 instance running https://github.com/shyiko/mysql-binlog-connector
-java to read binlogs from an aws rds mysql master (which is row based
replication and bandwidth and throughput are not an issue).The number of
binlogs is around 1k per second. The P99 network IO is about 50ms and the
P99 time between 2 IOs is about 90 ~ 100ms.
However, when we extract the timestamp in the binlog event and calculate the
difference between the current time and the timestamp, the P99 is about 4
seconds. We are hoping to make it only several hundred ms. Further
investigatiosn indicate that the difference between the timestamp and the
corresponding start time of network IO reading from mysql master is
responsible for most of the latency. And we are quite sure that we are
processing much faster than the pace of mysql generating binlogs, so each
time we read all the binlogs available there when it's stable.
So, the question is that is this latency caused by some bad settings in
mysql? Or by the mysql connector? Thanks a lot.
发信人: xnxky (change, change, change), 信区: Database
标 题: About the latency of reading from mysql binlog
发信站: BBS 未名空间站 (Wed Nov 18 14:05:32 2015, 美东)
Hi, friends, I have a very urgent problem about mysql binlog latency, and
any help is greatly appreciated!
Here is my scenario:
We use ec2 instance running https://github.com/shyiko/mysql-binlog-connector
-java to read binlogs from an aws rds mysql master (which is row based
replication and bandwidth and throughput are not an issue).The number of
binlogs is around 1k per second. The P99 network IO is about 50ms and the
P99 time between 2 IOs is about 90 ~ 100ms.
However, when we extract the timestamp in the binlog event and calculate the
difference between the current time and the timestamp, the P99 is about 4
seconds. We are hoping to make it only several hundred ms. Further
investigatiosn indicate that the difference between the timestamp and the
corresponding start time of network IO reading from mysql master is
responsible for most of the latency. And we are quite sure that we are
processing much faster than the pace of mysql generating binlogs, so each
time we read all the binlogs available there when it's stable.
So, the question is that is this latency caused by some bad settings in
mysql? Or by the mysql connector? Thanks a lot.
