How Facebook Ships Code Twice a Day

Facebook

If you think of open source and don’t think of Facebook, you probably should. The social network has quietly been enhancing open source technologies, including PHP and Mercurial source control.

Facebook“Facebook’s main source repository is enormous — many times larger than even the Linux kernel, which checked in at 17 million lines of code and 44,000 files in 2013,” wrote Durham Goode and Siddharth Agarwal in a Facebook blog.

“Given our size and complexity, and Facebook’s practice of shipping code twice a day, improving our source control is one way we help our engineers move fast.”

InfoWorld reports that after speeding up PHP by creating the HipHop virtual machine, Facebook turned its attention to source control. After evaluating its existing Git technology, Facebook abandoned it for Mercurial, which it considered more extensible.

“Instead of building a new system from scratch, we decided to take an existing one and make it scale,” the Facebook bloggers wrote.

Our engineers were comfortable with Git and we preferred to stay with a familiar tool, so we took a long, hard look at improving it to work at scale. After much deliberation, we concluded that Git’s internals would be difficult to work with for an ambitious scaling project.

Instead, Facebook chose to improve Mercurial. Mercurial is a distributed source control system similar to Git, with many equivalent features. “Importantly, it’s written mostly in clean, modular Python (with some native code for hot paths), making it deeply extensible,” the Facebook engineers wrote.

According to InfoWorld, Facebook has also released many of its other innovations as open source products, including RocksDB, HipHop PHP VM, the Presto SQL query engine, the Corona scheduler for Hadoop, and Flashcache, a caching system designed specifically to work with flash disks to lengthen their lifetimes.

Facebook doesn’t describe its server farm in detail, but has confirmed it makes use of the open source configuration manager Chef, to which InfoWorld reports the social network has also made additions.

Post a Comment

Your email address will not be published.