Facebook数据业务盘点

随着Facebook上市,其大数据模式越来越受到人们的重视,刚刚在俄勒冈州Prineville新建的Facebook数据中心,更是号称全球能效最高的数据中心。Facebook的数据业务具体有哪些特色?下面略做盘点。

数据采集

Timeline

2011年12月发布的Timeline主要针对“Profile”进行调整。Facebook Profile相当于个人的档案和资料,通俗来说就是个人主页。新的个人Profile比过去版本更有视觉冲击力。Facebook 推出新的 Timeline 界面,它组织个人在 Facebook 上发布的信息,比如状态、图片、视频等等,以更加结构化的方式展现出来,就像一本放在 Facebook 上的自传。

Like按钮

该功能可以让用户标注自己喜欢的页面,并将这些页面收录至Facebook的搜索结果中,这一功能与谷歌利用页面相互之间的链接关系来决定搜索排名类似。Facebook表示:“只要用户点击了‘Like’按钮,所有的支持Open Graph协议的网站都将显示在搜索引擎中。”facebook将借助Open Graph协议将搜索引擎的索引范围进一步扩大,从而对谷歌产生威胁。

数据存储

Memcached

是一个分布式的内存缓存系统,Facebook用它作为Web服务器和MySQL服务器之间的缓存层(因为数据库访问相对比较慢)。多年来,Facebook已经对Memcached和它的周边软件进行了很多优化,比如对network stack的优化。Facebook每时每刻都有数10TB的数据缓存在数千台Memcached服务器上。它可能是世界上最大的Memcached服务器集群了。

Haystack

Haystack是Facebook的高性能图片存取系统,但严格来说,它并不仅限于存储照片。它要管理超过200亿张上传的照片,并且每一片照片被保存为四种不同的分辨率,因此有超过800亿张照片。它不仅要能处理上亿的照片,而且性能也是至关重要的。Facebook每秒钟要处理约120万张照片,而且不包括CDN上的,这是一个惊人的数字。

Cassandra

Cassandra是一个可以避免单点故障的分布式存储系统。它是NoSQL运动的一个典范,并已开放源代码。它甚至成为一个Apache项目。Facebook在收件箱搜索中使用它,除了Facebook,其他网站也在使用它。

数据分析

Hadoop架构

Hadoop 是当今在分布式/并行计算中最流行的开源工具,他不仅仅可以用于存储的分布式文件系统,可以用来构建大量集群计算机以实现大规模的数据集分布式存储和存档处理。Facebook是Hadoop的忠实用户和贡献源码的参与者,并且Facebook还贡献了Hive和Thrift两个重要Hadoop组件,目前已经收录在Apache的Hadoop的子项目中。

Hive

Hive起源于Facebook,它使得针对Hadoop进行SQL查询成为可能,从而非程序员也可以方便地使用。Hive是基于Hadoop的一个数据仓库工具,可以将结构化的数据文件映射为一张数据库表,并提供完整的SQL查询功能,可以将SQL语句转换为MapReduce任务运行。

Zookeeper、Thrift

Hadoop中的子项目还包括Zookeeper分布式锁,提供类似Google Chubby的功能。Thrift是Hadoop的跨语言接口,可以支持多种语言,例如php和ruby。

BigPipe

BigPipe是Facebook开发的的动态网页处理系统。为了获得最佳性能,Facebook用它来按分块处理每个网页(称为“pagelets”)。例如,聊天窗口,新闻Feed等是通过分块分开进行传输的。这些pagelets可以并行工作,不仅可以提高性能,而且即使其中一部分失效或中断,也不影响用户的正常访问。

转载:199it

It was being said that the Super Bowl 2012 was going to be dependent on the way things uncovered at the Super Bowl 2011. The smooth organization of the fake oakleys Super Bowl 2011 was going to be decided by the adoption of a new collective bargaining agreement between the league and the National Football Players Association. All these reviews that are made are strictly legal and ethical and this fake oakleys information is not disclosed by the credit card company to any other person or organization. One must also note cheap oakleys that the government Cheap NFL Authentic Jerseys always governs these reviews and ratings with the help of strict laws, and the person who is being reviewed, is empowered to sue the credit rating company or the credit card company, in case of any false information, or even misrepresentation.. The models join a long list of beautiful women (Christie Brinkley and Tyra Banks, to name a few) who have had the honor of covering the magazine, and it’s completely well deserved. We’ll have to wait until Feb. “I had an interception, I had two more tackles. I just started enjoying it. The number of times an article has been cited by individual mainstream news sources, blog post, or member of Google+ along with a link to the original article or post. News articles, blog posts and Google+ posts do not always link Wholesale NFL Jerseys to articles in a way that can be picked up by aggregators used by Altmetric, so the listed links are not necessarily a reflection of the entire scope of media, oakley outlet blog or Google+ interest. Netflix’s superior distribution channel has allowed it to maintain a foothold in this new omni platform media market where audiences using smartphones, tablets, and laptops are growing faster than audiences in front of the TV. So what companies would be better than the big four, who already have a firm grip on their share of the internet, to wholesale nfl jerseys get their hands dirty and dive into the world of content? Barron’s featured an article saying that Apple should step in and hijack the purchase of Time Warner before any confirmation from the DOJ. In a study done at State University of New York, it was found that college women who were doing the four legged polka without benefit of condoms showed fewer signs of depressive behavior than those who used condoms regularly or abstained from sex. Why? Because semen is only about three percent sperm. The actual duel was a pussyfooted thing that got resolved with a handshake. Naturally, this being the Wild West, the lack of Eastwoodian antics disappointed the audience, which promptly got rowdy and started breaking stuff. Joeckel, also a left tackle, was the far safer pick and ended up going No. 2 to Jacksonville.

发表回复