b*r
2 楼
我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
there are 2 services, service_a and service_b.
service_a majorly does calculation for the given entity. after that if the
calculated result is different to what's persisted in Oracle DB, then update
the corresponding rows for the given entity in Oracle DB. entities are
received continuously from some queuing services.
service_b majorly does read from Oracle DB periodically, let us say every 4
hours. it basically reads all updated entities done by service_a
in the past 4 hours, then outputs them in a certain text file format to
Amazon's S3 for storage.
a little bit more background, in the past (before 2012) service_b
read all entities, not just updated ones by service_a in the past 4 hours to
create a complete snapshot of all entities for various clients. then it
turned out Oracle could not handle that much concurrent write and reads at
the same time, or degraded too much, so right before I joined this small
factory someone changed the logic to have service_b only read delta (updated
entities) in the past 4 hours. they created another process to merge the
delta into base to create the final snapshot somewhere else.
now, manager wants to move even further to get rid of Oracle completely,
maybe because Oracle is too costly to maintain. we are looking if there are
some open-sourced Sql/no-sql DB such as mysql, cassandra, dynamodb, etc.,
which can replace Oracle, such that both service_a and service_b can work
without interruption.
furthermore, it's even better if, after Oracle is replaced, service_b can
read directly from the new DB for ALL entities, not just UPDATED entities,
to create complete snapshot without going through the current steps of
creating delta, merging with baseline, etc. basically service_b goes back to
the old time (before 2012) when it used to work.
any good suggestions for the DB candidate? if it has very good / quick way
to create current DB replica, that would be very nice. the new DB should
also support high data consistency and availability. even if the consistency
might not be matchable to Oracle we hope it can get as close as possible.
currently we are considering the following candidates: cassandra, dynamodb,
mysql. are there other candidates?
现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
there are 2 services, service_a and service_b.
service_a majorly does calculation for the given entity. after that if the
calculated result is different to what's persisted in Oracle DB, then update
the corresponding rows for the given entity in Oracle DB. entities are
received continuously from some queuing services.
service_b majorly does read from Oracle DB periodically, let us say every 4
hours. it basically reads all updated entities done by service_a
in the past 4 hours, then outputs them in a certain text file format to
Amazon's S3 for storage.
a little bit more background, in the past (before 2012) service_b
read all entities, not just updated ones by service_a in the past 4 hours to
create a complete snapshot of all entities for various clients. then it
turned out Oracle could not handle that much concurrent write and reads at
the same time, or degraded too much, so right before I joined this small
factory someone changed the logic to have service_b only read delta (updated
entities) in the past 4 hours. they created another process to merge the
delta into base to create the final snapshot somewhere else.
now, manager wants to move even further to get rid of Oracle completely,
maybe because Oracle is too costly to maintain. we are looking if there are
some open-sourced Sql/no-sql DB such as mysql, cassandra, dynamodb, etc.,
which can replace Oracle, such that both service_a and service_b can work
without interruption.
furthermore, it's even better if, after Oracle is replaced, service_b can
read directly from the new DB for ALL entities, not just UPDATED entities,
to create complete snapshot without going through the current steps of
creating delta, merging with baseline, etc. basically service_b goes back to
the old time (before 2012) when it used to work.
any good suggestions for the DB candidate? if it has very good / quick way
to create current DB replica, that would be very nice. the new DB should
also support high data consistency and availability. even if the consistency
might not be matchable to Oracle we hope it can get as close as possible.
currently we are considering the following candidates: cassandra, dynamodb,
mysql. are there other candidates?
l*a
3 楼
2条女狗,怎么训都不行,一在家就喜欢到处尿尿。
地毯上都是一块块的尿渍,想换地板,但是据说地毯泡了尿会膨胀,
只有竹地板不怕水。
有用竹地板的同学吗?进来说说是不是不怕水?
还有什么好建议的?
地毯上都是一块块的尿渍,想换地板,但是据说地毯泡了尿会膨胀,
只有竹地板不怕水。
有用竹地板的同学吗?进来说说是不是不怕水?
还有什么好建议的?
b*r
5 楼
自己顶一下。给大牛们跪了。
update
4
【在 b********r 的大作中提到】![](/moin_static193/solenoid/img/up.png)
: 我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
: 现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
: there are 2 services, service_a and service_b.
: service_a majorly does calculation for the given entity. after that if the
: calculated result is different to what's persisted in Oracle DB, then update
: the corresponding rows for the given entity in Oracle DB. entities are
: received continuously from some queuing services.
: service_b majorly does read from Oracle DB periodically, let us say every 4
: hours. it basically reads all updated entities done by service_a
: in the past 4 hours, then outputs them in a certain text file format to
update
4
【在 b********r 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
: 现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
: there are 2 services, service_a and service_b.
: service_a majorly does calculation for the given entity. after that if the
: calculated result is different to what's persisted in Oracle DB, then update
: the corresponding rows for the given entity in Oracle DB. entities are
: received continuously from some queuing services.
: service_b majorly does read from Oracle DB periodically, let us say every 4
: hours. it basically reads all updated entities done by service_a
: in the past 4 hours, then outputs them in a certain text file format to
t*3
6 楼
狗前列腺不好了吧
p*2
8 楼
update
4
size多大?
throughput,latency什么要求?
【在 b********r 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
: 现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
: there are 2 services, service_a and service_b.
: service_a majorly does calculation for the given entity. after that if the
: calculated result is different to what's persisted in Oracle DB, then update
: the corresponding rows for the given entity in Oracle DB. entities are
: received continuously from some queuing services.
: service_b majorly does read from Oracle DB periodically, let us say every 4
: hours. it basically reads all updated entities done by service_a
: in the past 4 hours, then outputs them in a certain text file format to
c*a
10 楼
co抗议,lz给个包子
b*r
11 楼
size不小。整个的database complete snapshot用BerkeleyDB格式可能是1T左右,当然
每次的updated entities没那么多,在1~100M左右。2012以前,是每次都从Oracle读整
个的当前snapshot,对系统的影响比较大。后来就不从Oracle直接读,建立了service_
b来work around。当然这时数据stale有可能出现,但目前这个问题还不用太关心。
latency希望不要差Oracle太多,1,2倍问题不大。差个数量级是不行的。
【在 p*****2 的大作中提到】![](/moin_static193/solenoid/img/up.png)
:
: update
: 4
: size多大?
: throughput,latency什么要求?
每次的updated entities没那么多,在1~100M左右。2012以前,是每次都从Oracle读整
个的当前snapshot,对系统的影响比较大。后来就不从Oracle直接读,建立了service_
b来work around。当然这时数据stale有可能出现,但目前这个问题还不用太关心。
latency希望不要差Oracle太多,1,2倍问题不大。差个数量级是不行的。
【在 p*****2 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
:
: update
: 4
: size多大?
: throughput,latency什么要求?
s*r
13 楼
热~这么多天了 只有一次早餐包~
g*g
14 楼
MySQL with a readonly replica. Front your DB with Memcached.
update
4
【在 b********r 的大作中提到】![](/moin_static193/solenoid/img/up.png)
: 我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
: 现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
: there are 2 services, service_a and service_b.
: service_a majorly does calculation for the given entity. after that if the
: calculated result is different to what's persisted in Oracle DB, then update
: the corresponding rows for the given entity in Oracle DB. entities are
: received continuously from some queuing services.
: service_b majorly does read from Oracle DB periodically, let us say every 4
: hours. it basically reads all updated entities done by service_a
: in the past 4 hours, then outputs them in a certain text file format to
update
4
【在 b********r 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 我现在准备上的一个项目,没有什么头绪,或者说是头绪太多。恳请大牛斧正和指教。
: 现在开发环境是C/C++,Oracle.下面用英语,描述起来方便些。
: there are 2 services, service_a and service_b.
: service_a majorly does calculation for the given entity. after that if the
: calculated result is different to what's persisted in Oracle DB, then update
: the corresponding rows for the given entity in Oracle DB. entities are
: received continuously from some queuing services.
: service_b majorly does read from Oracle DB periodically, let us say every 4
: hours. it basically reads all updated entities done by service_a
: in the past 4 hours, then outputs them in a certain text file format to
i*4
19 楼
这个要顶!
s*x
20 楼
are data for different client totally independent ?
If so, simply shard the database based on client id.
You can build your own sharding map. Oracle to MySQl should be easy
conversion.
Sharding or partitioning is the key for many distributed systems.
If so, simply shard the database based on client id.
You can build your own sharding map. Oracle to MySQl should be easy
conversion.
Sharding or partitioning is the key for many distributed systems.
g*g
22 楼
ding
j*p
24 楼
到处乱尿,会不会很臭啊。。。。
看房子的时候看过一家养狗的,
一进门,那强烈的狗尿味让我连看的兴趣都没了。
看房子的时候看过一家养狗的,
一进门,那强烈的狗尿味让我连看的兴趣都没了。
p*r
25 楼
spt.
m*k
26 楼
很好奇你们公司start service b 的时候为何没考虑DB master slave replica, if
oracle is too expensive for that on cost, switch to mysql immediately,
instead of re-invent the wheel by creating the delta + merge
(很可能service A 写数据库的code是oracle depended, nobody wanted to change. )
oracle is too expensive for that on cost, switch to mysql immediately,
instead of re-invent the wheel by creating the delta + merge
(很可能service A 写数据库的code是oracle depended, nobody wanted to change. )
j*g
28 楼
spt
j*3
29 楼
mark
h*x
31 楼
Ding. Never get anything during two months.
p*2
32 楼
他说的不是replication 是 backup
)
【在 m*****k 的大作中提到】![](/moin_static193/solenoid/img/up.png)
: 很好奇你们公司start service b 的时候为何没考虑DB master slave replica, if
: oracle is too expensive for that on cost, switch to mysql immediately,
: instead of re-invent the wheel by creating the delta + merge
: (很可能service A 写数据库的code是oracle depended, nobody wanted to change. )
)
【在 m*****k 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 很好奇你们公司start service b 的时候为何没考虑DB master slave replica, if
: oracle is too expensive for that on cost, switch to mysql immediately,
: instead of re-invent the wheel by creating the delta + merge
: (很可能service A 写数据库的code是oracle depended, nobody wanted to change. )
b*y
34 楼
包子就这么香吗?以后改为中国北京时间
j*i
39 楼
杀吃了。再养只不乱尿的
w*r
43 楼
同抗议
m*k
44 楼
看到s3了,
这么说来他们的servcie b + merge process 就是要把oracle 存个snapshot到s3,的
确像是个backup,但这之前因该有不同的clients 用这个数据,他提到了
“a little bit more background, in the past (before 2012) service_b
read all entities, not just updated ones by service_a in the past 4 hours to
create a complete snapshot of all entities for various clients.”
http://serverfault.com/questions/33760/oracle-real-time-databas
这么说来他们的servcie b + merge process 就是要把oracle 存个snapshot到s3,的
确像是个backup,但这之前因该有不同的clients 用这个数据,他提到了
“a little bit more background, in the past (before 2012) service_b
read all entities, not just updated ones by service_a in the past 4 hours to
create a complete snapshot of all entities for various clients.”
http://serverfault.com/questions/33760/oracle-real-time-databas
b*r
47 楼
太感激大牛们!!各位的意见很有启发!
不好意思,有的地方我说的不是很清楚。这里的snapshot主要是for replica,not for
backup.我不知道具体的原因为什么当时没有用oracle replica,可能是由于钱的原因
,也可能是应为不想create replica的时候影响到oracle的读写。人已经离开,没法去
问了。
现在mysql也是oracle所有,我们有点担心oracle会做些手脚,让Mysql在性能上落后
oracle越来越远,否则谁还会去用oracle?
1)在什么情况下mysql的performance会相比oracle有明显恶化?比如,table has
more than 1 m rows.
2) how often we can ask mysql to create readonly replica without impacting
its regular write/read performance? hourly, every 10 minutes, every 1 minute?
3) 假设我们相信oracle的性能在任何情况下(用同样的cache,index, partition等)
都比mysql要好,我们如何说服人换到mysql是个准确的选择(出去费用上的考虑)
4)大牛们又没有做过oracle vs mysql side by side comparison,但相同的Load下?
或者我们只能用数量取胜,比如oracle我们只有2个instances,但Mysql我们可以用4/8
,反正是很便宜?
to
【在 m*****k 的大作中提到】![](/moin_static193/solenoid/img/up.png)
: 看到s3了,
: 这么说来他们的servcie b + merge process 就是要把oracle 存个snapshot到s3,的
: 确像是个backup,但这之前因该有不同的clients 用这个数据,他提到了
: “a little bit more background, in the past (before 2012) service_b
: read all entities, not just updated ones by service_a in the past 4 hours to
: create a complete snapshot of all entities for various clients.”
: http://serverfault.com/questions/33760/oracle-real-time-databas
不好意思,有的地方我说的不是很清楚。这里的snapshot主要是for replica,not for
backup.我不知道具体的原因为什么当时没有用oracle replica,可能是由于钱的原因
,也可能是应为不想create replica的时候影响到oracle的读写。人已经离开,没法去
问了。
现在mysql也是oracle所有,我们有点担心oracle会做些手脚,让Mysql在性能上落后
oracle越来越远,否则谁还会去用oracle?
1)在什么情况下mysql的performance会相比oracle有明显恶化?比如,table has
more than 1 m rows.
2) how often we can ask mysql to create readonly replica without impacting
its regular write/read performance? hourly, every 10 minutes, every 1 minute?
3) 假设我们相信oracle的性能在任何情况下(用同样的cache,index, partition等)
都比mysql要好,我们如何说服人换到mysql是个准确的选择(出去费用上的考虑)
4)大牛们又没有做过oracle vs mysql side by side comparison,但相同的Load下?
或者我们只能用数量取胜,比如oracle我们只有2个instances,但Mysql我们可以用4/8
,反正是很便宜?
to
【在 m*****k 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 看到s3了,
: 这么说来他们的servcie b + merge process 就是要把oracle 存个snapshot到s3,的
: 确像是个backup,但这之前因该有不同的clients 用这个数据,他提到了
: “a little bit more background, in the past (before 2012) service_b
: read all entities, not just updated ones by service_a in the past 4 hours to
: create a complete snapshot of all entities for various clients.”
: http://serverfault.com/questions/33760/oracle-real-time-databas
d*t
49 楼
Ding!
co 排包子
co 排包子
g*g
50 楼
read replica的做法都是基于commit log asynchronously replay, 可能会有几秒延迟
,对性能不会有影响。
几个M记录的表对MySQL不算什么。性能通常都是由架构和设计决定的,Oracle和MySQL
的性能差异极其有限,如果不是MySQL更快的话。
for
【在 b********r 的大作中提到】![](/moin_static193/solenoid/img/up.png)
: 太感激大牛们!!各位的意见很有启发!
: 不好意思,有的地方我说的不是很清楚。这里的snapshot主要是for replica,not for
: backup.我不知道具体的原因为什么当时没有用oracle replica,可能是由于钱的原因
: ,也可能是应为不想create replica的时候影响到oracle的读写。人已经离开,没法去
: 问了。
: 现在mysql也是oracle所有,我们有点担心oracle会做些手脚,让Mysql在性能上落后
: oracle越来越远,否则谁还会去用oracle?
: 1)在什么情况下mysql的performance会相比oracle有明显恶化?比如,table has
: more than 1 m rows.
: 2) how often we can ask mysql to create readonly replica without impacting
,对性能不会有影响。
几个M记录的表对MySQL不算什么。性能通常都是由架构和设计决定的,Oracle和MySQL
的性能差异极其有限,如果不是MySQL更快的话。
for
【在 b********r 的大作中提到】
![](/moin_static193/solenoid/img/up.png)
: 太感激大牛们!!各位的意见很有启发!
: 不好意思,有的地方我说的不是很清楚。这里的snapshot主要是for replica,not for
: backup.我不知道具体的原因为什么当时没有用oracle replica,可能是由于钱的原因
: ,也可能是应为不想create replica的时候影响到oracle的读写。人已经离开,没法去
: 问了。
: 现在mysql也是oracle所有,我们有点担心oracle会做些手脚,让Mysql在性能上落后
: oracle越来越远,否则谁还会去用oracle?
: 1)在什么情况下mysql的performance会相比oracle有明显恶化?比如,table has
: more than 1 m rows.
: 2) how often we can ask mysql to create readonly replica without impacting
H*a
51 楼
你这个是因为屋里地毯有狗狗尿留下来的味道,无论你觉得清洗的怎样彻底,她们都会
闻到的,就会重复在家里尿尿。狗狗有多大了,根本还是要训练到外面尿尿。我当年训
练我家狗的方法是在屋里放那种狗尿尿的pad(宠物店都有卖的,方块形状的),刚开
始多放几块,然后在狗要尿尿之前把他赶到上面或者把狗尿抹在上面,同时屋里其他地
方狗尿过的要马上清理,反复清理,目的就是这些地方残留的狗尿味道要比pad上弱很
多,这样狗狗下次就找pad去解决。等狗养成在pad上解决的习惯后,减少pad的数量,
最后到一块pad.那以后屋里狗尿的味道越来越越淡/少,他就改到外面去解决了。如果
狗不能hold尿时间太久,那就得多溜几次,而且每次去同样的地方,这样他会找原来撒
过尿的地方。这个就是一个routine和反复练习的过程。另外刚开始训练在pad上和出去
撒尿,只要狗做到了,就给奖励treats,要是在屋里非pad的地方撒尿,就要马上训斥
,总之让狗有个信号反映。
什么地板也经不起狗尿长期,除非是tile类的会好一些。
闻到的,就会重复在家里尿尿。狗狗有多大了,根本还是要训练到外面尿尿。我当年训
练我家狗的方法是在屋里放那种狗尿尿的pad(宠物店都有卖的,方块形状的),刚开
始多放几块,然后在狗要尿尿之前把他赶到上面或者把狗尿抹在上面,同时屋里其他地
方狗尿过的要马上清理,反复清理,目的就是这些地方残留的狗尿味道要比pad上弱很
多,这样狗狗下次就找pad去解决。等狗养成在pad上解决的习惯后,减少pad的数量,
最后到一块pad.那以后屋里狗尿的味道越来越越淡/少,他就改到外面去解决了。如果
狗不能hold尿时间太久,那就得多溜几次,而且每次去同样的地方,这样他会找原来撒
过尿的地方。这个就是一个routine和反复练习的过程。另外刚开始训练在pad上和出去
撒尿,只要狗做到了,就给奖励treats,要是在屋里非pad的地方撒尿,就要马上训斥
,总之让狗有个信号反映。
什么地板也经不起狗尿长期,除非是tile类的会好一些。
c*3
52 楼
S3和oracle差了4小时,不象replica,更象是给第3方用的?
写个service-c吧。service-a把数据写入oracle之后立即(async)call service-c把同
样的数据写入S3。如果service-c fail,service-c就把同样数据从oracle中再读出来
写入S3。
service-b可以退休了。
换成MySQL也一样,如果MySQL to MySQL replica 太慢。service-c写入MySQL。
写个service-c吧。service-a把数据写入oracle之后立即(async)call service-c把同
样的数据写入S3。如果service-c fail,service-c就把同样数据从oracle中再读出来
写入S3。
service-b可以退休了。
换成MySQL也一样,如果MySQL to MySQL replica 太慢。service-c写入MySQL。
s*a
53 楼
狗在自己窝里不会尿,搞个CAGE把狗关起来,每2-3个小时出去尿一次,
晚上睡觉就把狗关在CAGE里,训练两周就好了
晚上睡觉就把狗关在CAGE里,训练两周就好了
w*u
60 楼
女狼如何。。。。。
n*g
61 楼
还有个OPTION,装质量高的VYNIL地板贴片,比较现实,不怕尿不怕水,防刮性能良好,
真是家有狗的人士必备的地板呀。。。
真是家有狗的人士必备的地板呀。。。
相关阅读
湾区房子的一个问题请问有人申请过 STAR PROGRAM 吗?家里到底有没有必要备灭火器?Milgard 和 Simonton 窗户哪个好?怎么查出哪个online company卖了我的电话信息?大伙慢慢灌,我也要不常来了大家看看这个contractor的估价合理吗?home inspection可否指点下选哪几个?谢谢!新装的锅炉就有很多锈,正常吗?家里有没有必要常备半个月的干粮和水这个好象gas表是不是坏了?美国到底是买房还是租房划算想在楼梯下面的储藏间里加一个电源插座,电话cable插座这把枪怎样?Dallas 寻 6月底-7月初的一个傍晚 遇见的买房人大家看到了这个 北京最牛违章建筑了吗?大家用过countertop soap dispenser吗? (转载)买的condo附带趴车位,请问如何转title?阁楼保温材料是否石棉Cost to Insulate an Attic! (转载)