Facebook Announces Apollo, a New NoSQL Database for On-line Low Latency Storage - 未名空间MITBBS历史存档

国际科技财经博客移民网络热点娱乐民生时事公众号

Redian新闻

>未名空间

>MobileDevelopment - 移动开发

Facebook Announces Apollo, a New NoSQL Database for On-line Low Latency Storage

Facebook Announces Apollo, a New NoSQL Database for On-line Low Latency Storage# MobileDevelopment - 移动开发

l*a2014-06-15 07:06

1 楼

【以下文字转载自 Military 讨论区】
发信人: tukala (tukala), 信区: Military
标题: 转帖：11.09华府华人团体组织抗议ABC
发信站: BBS 未名空间站 (Wed Nov 6 11:51:13 2013, 美东)
表达华裔的愤怒！发出民族的声音！大华府及全美二十多个城市11/9将举行万人大游行！
大华府各华人社团及广大华侨华人朋友：
大家好！为强烈抗议美国广播电视公司(ABC)夜间节目“吉米.克摩尔脱口秀”的辱华
言论和吉米.克摩尔本人“道歉不认错”的行为，以捍卫华人的民族尊严，向全世界发
出我们正义的声音，由华盛顿地区多个华人社团共同发起组织，并协同洛杉矶、纽约/
新泽西、旧金山、波士顿、西雅图、圣荷西、圣地牙哥、休斯顿、达拉斯、奥斯汀、芝
加哥、迈阿密、奥兰多、新奥尔良、丹佛、北卡罗来纳教堂山、夏洛特，费城、底特律
等二十多个城市的华侨华人将联合举行“11/9全美万人大游行”，要求ABC高层向受到
伤害的全球华人公开道歉，并对吉米.克摩尔处以停职、解雇等严厉制裁。预计参加游
行的人数会超过万人，或将成为2008年抗议CNN辱华言论之后规模最大的一次抗议活动。
作为“11/9全美万人大游行”的一部分，大华府游行活动组织者均表示，ABC辱华事件
的出现绝非偶然，“杀掉所有中国人。”看似童言无忌的一句话，却折射出弥漫在美国
学校和家长中的种族歧视观念对孩子们内心所产生的巨大影响，而且通过ABC的广泛传
播，客观上向美国社会传达了一种种族歧视和种族仇恨的信号。中华民族并不缺少幽默
感，但我们不能容忍以看似幽默的方式，对一个伟大民族的公然歧视和恐吓。如果我们
听之任之，今后谁都敢蹬着鼻子上脸，不仅语言冒犯而且仇恨杀人案将会越来越多。黑
人、印度人已经为我们做出了榜样，并且赢得了社会的尊重。华人也应该如此，积极地
站出来，表达自己的意愿，为整个民族发声。
华盛顿示威活动安排如下：
时间：2013年11月9日（星期六）上午10：00至12：00
地点：华盛顿市中心的华盛顿纪念碑前（地铁在蓝线和橘线的Smithsonian站下）
联系：何如意 301-768-0900、叶玉彬 703-582-3513、夏祥波 571-208-4441、粱康之
240-506-8750、王耀辉 301-919-3836、张斌 410-294-8637、肖水根 240-432-2912
报名：http://tinyurl.com/dc-chinese-protest-abc (方便及时联络、carpool、结伴而行等)
华盛顿华人社区联盟呼吁，大华府各个华人社团，以及广大华侨华人积极参加本次示威
游行活动，并用我们的实际行动，捍卫中华民族的尊严，让美国社会看到华人的力量。
我们也鼓励参加游行的华人朋友们自制标语牌，并以合适的文字（最好是英文）表达自
己的心声。样品可参阅
http://www.gwcac.org/。
另外，旅美华人于10月19日在白宫网站上发起请愿活动，要求ABC诚恳道歉，调查主持
人吉米，并关闭该夜间脱口秀节目。截止11月3日晚五时，白宫请愿网上请愿信联署人
数已超过9.6万人，距离最低门槛10万个签名还剩下几千个，即便超过这个门坎，也请
签名支持，签名越多，
声势越大。为此，请尚未签名的华人朋友抓紧登陆白宫请愿网址签名支持，为华人的
正义行动贡献自己的一份力量：https://petitions.whitehouse.gov/petition/
investigate-jimmy-kimmel-kids-table-government-shutdown-show-abc-network/
tLxzbBjg
华盛顿华人社区联盟热诚欢迎华人朋友对本次游行活动提供赞助，有意者请联系：
周刚 301-288-1338, g********[email protected] 何晓慧 301-512-3316,
c*******[email protected]，
或直接通过Paypal转帐的，请用：https://www.paypal.com/cgi-bin/webscr?cmd=_s-
xclick&hosted_button_id=KNH2B7ZX5F7TA
11月9日上午10点，华盛顿纪念碑下，我们不见不散！

z*n2014-06-15 07:06

2 楼

Facebook Announces Apollo, a New NoSQL Database for On-line Low Latency
Storage
by Charles Humble on Jun 13, 2014 | Discuss

Speaking at QCon New York on Wednesday Jeff Johnson, from the core data
group at Facebook, announced Apollo, Facebook’s Paxos-like NoSQL database.
Written in C++11 on top of the Apache Thrift 2 RPC framework, Apollo is a
hierarchical storage system where all the data is split into shards, very
much analogous to region servers in HBase. The sweet-spot for it, Johnson
explained, is on-line low latency storage - in particular Flash and in-
memory.
As distinct from a document oriented, or key value store, Apollo is about
modifications to data structures, allowing you to represent maps, queues,
trees and so on, as well as key values. Within the system individual pieces
of data are quite small - a range of between 1 byte and 1MB, with a total
size anywhere from 1MB to 10+PB. It supports anything from a minimum of
three servers to thousands.
Each Shard has four components. The first is a quorum consensus protocol
which is based on Raft, a strong leader consensus protocol from Stanford.
Johnson explained that one of the things that his team really like about
Raft is that the leader failure recovery is really well defined, as is the
quorum view change. That said, he suggested, it isn’t really simpler to
work with than multi-paxos:
We’ve had to do tons and tons and stuff - everything from allowing you
to asynchronous write and read from disk to trying to deal with situations
when followers are getting behind because there’s other stuff going on on
the server or the disk is slow, corruption detection, and so on
The second component is storage. At the time of writing the primary storage
is based RocksDB, a Key/Value store that builds on top of Google’s LevelDB.
Whilst it is a Key/Value store Facebook are using it to emulate other data
structures. Apollo is designed to be storage agnostic and the team are also
working on adding support for MySQL as an alternative storage engine.
The third component is a Client API with read() and write() methods. Every
operation that Apollo performs at a Shard level is atomic, so you express
pre-conditions and if these are satisfied it returns the reads or writes.
For example this code:
read(conditions : {map(m1).contains(x)},
reads : {deque(d2).back()})
Says “If the map m1 contains the value x then return the value that is on
the back of the d2 deque.”
You can combine together any number of conditions and any number of reads.
Writes are very similar and again allow you to express conditions:
write(conditions : {ver(k1) == v}, reads : {},
writes : {val(k1) := x})
The final of the four shard components are Fault Tolerant State Machines (
FTSMs). These are primarily used by the system code but can also be used for
user code. Each FTSM is owned by a shard so that, for example, in a shard
of three machines all of them will be executing the same code at the same
time. They are able to access the persistent storage that is local to each
machine. Most importantly if one node dies the code continues to execute in
a proper order that all the nodes agree on.
Amongst other things the state machines are used for load balancing, data
migration, shard creation and destruction, and co-ordinating cross-shard
transactions. State machines can have external side effects, for example
they can send RPC requests to remote machines, but whenever they make a
change to persistent state they have to submit it to Raft to get all the
servers to agree.
Apollo isn't currently being used in production at Facebook, but the firm is
looking at using it to replace some memcahced use cases, and Johnson made
clear that Facebook makes very significant use of memcahced. "More
generally," Johnson told InfoQ "we’re looking into various in-memory
storage use cases at Facebook, either new ones or replacing some existing
ones, by comparing side-by-side with existing systems."
The company is also looking at using Apollo as a reliable queuing system for
outgoing Facebook messages to iOS, Android and carriers via SMS, and also
potentially for faster analytics.
Apollo is still in development and hasn’t been open-sourced though Johnson
did state that doing so was something Facebook were looking into and would
like to do. Johnson's presentation is currently available to QCon New York
attendees and will be published to everyone via InfoQ in due course.