Blu-Ray Cold Storage...? Neat idea I just hope it can scale to Facebook's 1.44 Billion global users...
BY RICH MILLER - JUNE 30, 2015
FOREST CITY, N.C. – The temperature remains constant as you walk through Facebook’s custom data storage facility. But as you approach the back of the room, you transition from cold storage to even colder storage.
In a row of 14 racks housing square enclosures, Facebook is test-driving the future of its long-term data storage. The racks are packed with thousands upon thousands of Blu-Ray disks.
That’s right: the optical media that plays your movies can now back up all your status updates and Facebook photos.
The Blu-Ray storage system is part of the company’s evolving effort to manage a flood of incoming data, with users uploading more than 900 million photos every day. To keep pace with all those uploads, Facebook must constantly seek new ways to add capacity.
“It’s amazing how much storage you can do with Blu-Ray,” said Keven McCammon, Datacenter Manager for Facebook’s facility in western North Carolina. “There are times when you can look back to look forward.”
The racks of Blu-Ray storage are still in the testing phase. But Facebook has high hopes for Blu-Ray as a tool to optimize its infrastructure. McCammon and his team are putting the system through its paces at the company’s massive East Coast data center campus.
“We’re doing some pretty extensive testing right now,” said McCammon. “We want to really make sure it can function in a production environment and can scale.”
With 1.44 billion users, Facebook’s storage needs may seem otherworldly to most users. But many companies will soon face similar challenges managing the explosive growth of data storage. The hyperscale data center players pioneer the design strategies and best practices that will soon filter into many other data centers. Facebook’s cold storage journey offers insights for managing the coming data tsunami.
Retooling Tiered Storage for the Hyperscale Age
Facebook’s cold storage system is a web-scale implementation of tiered storage, a strategy that organizes stored data into categories based on their priority and then assigns the data to different types of storage media to reduce costs. The goal is to create a top tier consisting of high-performance enterprise hardware and networking, while lower tiers can use commodity hardware or, for rarely-used assets, backup tape libraries.
The storage world has changed since tiered storage made its debut in 1990, but Facebook is applying many of the principles in its infrastructure, albeit with different technologies.
Facebook was an early adopter of solid state drives (SSDs), storage devices that use integrated circuits as memory. SSDs have no moving parts, unlike traditional hard disk drives (HDDs), which contain spinning disks and moveable read/write heads. Most importantly, SSDs are faster than hard disks, and can accelerate key parts of Facebook’s infrastructure.
While focusing on SSD and Flash in the high-performance portions of its storage infrastructure, Facebook continues to use plenty of hard disk drives to store photos. In 2012 it created a custom storage tray through the Open Compute Project, known as Knox.
By 2013, Facebook was storing more than an exabyte of images that were rarely accessed, with 82 percent of traffic focused on just 8 percent of photos. So the company created a “cold storage” design to shift these rarely-viewed photos from its expensive high-performance server farms to simpler data centers with no generators or UPS (uninterruptible power supply). Emergency backup power isn’t needed because the facility does not serve live production data.
“Reducing operating power was a goal from the beginning,” wrote Facebook’s Krish Bandaru and Kestutis Patiejunas in a blog post outlining the cold storage system. “So, we built a new facility that used a relatively low amount of power but had lots of floor space. The data centers are equipped with less than one-sixth of the power available to our traditional data centers, and, when fully loaded, can support up to one exabyte (1,000 PB) per data hall.”