Innovation promises to cut massive power use at big data companies in a flash

Jul 19, 2012
Vivek Pai (above), an associate professor of computer science at Princeton, worked with graduate student Anirudh Badam to develop a software technique that could radically cut power use. Credit: Photo by Frank Wojciechowski

Big data needs big power. The server farms that undergird the Internet run on a vast tide of electricity. Even companies that have invested in upgrades to minimize their eco-footprint use tremendous amounts: The New York Times estimates that Google, for example, uses enough electricity in its data centers to power about 200,000 homes.

Now, a team of Princeton University engineers has a solution that could radically cut that power use. Through a new software technique, researchers from the School of Engineering and have opened the door for companies to use a new type of memory in their servers that demands far less energy than the current systems.

The software, called SSDAlloc, allows the companies to substitute solid state memory, commonly called flash memory, for the more expensive and energy-intensive type of memory that is now used for most computer operations.

"The biggest potential users are the big data centers," said Vivek Pai, an associate professor of computer science who developed the program with his graduate student, Anirudh Badam. "They are going to see the greatest improvements."

A version of SSDAlloc is already being used with high-end flash memory manufactured by Fusion-io, of . Princeton has signed a non-exclusive with the company. Brent Compton, the Fusion-io's senior director of product management, said the software "simplifies performance for developers in ways that were out of reach just a couple of years ago."

The massive server centers that support operations ranging from online shopping to social media are built around a type of called , or RAM. While very fast and flexible, RAM needs a constant stream of electricity to operate.

The power not only costs money, it also generates heat that forces the companies to spend more funds on cooling.

The Princeton engineers' program allows the data companies to substitute flash memory — similar to chips used in "thumb drives" — for much of their RAM. Unlike RAM, flash only uses small amounts of electricity, so switching memory types can drastically cut a company's power bill. In extreme cases, depending on the type of programs run by the servers, that reduction can be as much as 90 percent (compared to a computer using RAM alone). And because those machines are not generating as much heat, the data centers can also cut their cooling bills.

Flash memory is also about 10 times cheaper than RAM, so companies can also save money on hardware upfront, Pai said.

So how does it work?

Badam, a graduate student in computer science, said that SSDAlloc basically changes the way that programs look for data in a computer.

Traditionally, a computer program will run its operations in RAM, which is fast and efficient, but unable to store information without power. When the program needs to store information longer, or when it needs to use data that is not in the RAM, it looks to storage memory — either flash memory or mechanical hard drives.

That is where a bottleneck occurs. The step at which the program switches to storage memory is glacially slow in computer terms. That is often the nature of the storage medium itself — mechanical hard drives are vastly slower than RAM. But it is also the result of underlying operating systems, such as Linux or Windows, that govern how the computer searches for information.

Flash memory is much faster than a hard drive, and flash is getting faster all the time. Currently, high-end flash memory has retrieval speeds of a million requests per second. (A top mechanical hard drive's retrieval speed is about 300 requests per second.)

That discrepancy created a dilemma for flash. The physical flash memory itself was fast enough that it could operate as an extension of RAM, but the underlying retrieval system's slow speed throttled its potential.

Based on earlier research, funded in part by the National Science Foundation, Pai and Badam felt they had a technique that could universally allow flash memory to serve as an extension of RAM. Other researchers had developed more narrow techniques, but they were difficult to program and only worked for certain applications. The Princeton researchers' idea would work with any program with minimal, and relatively straightforward, adjustments.

It was an ambitious idea. Two other research teams outside of Princeton had tried unsuccessfully to create similar results, and many experts were convinced that the technique could not be done through changes in software alone.

"It did seem like a long shot," Badam said.

What Badam did was write software that allows programmers to bypass this traditional system of searching for information in storage memory. His system allows for requests for information that take advantage of flash memory's extremely fast retrieval times. Essentially, SSDAlloc moves the flash memory up in the internal hierarchy of computer data — instead of thinking of flash as a version of a storage drive, SSDAlloc tells the computer to consider it a larger, somewhat slower, version of RAM.

"I wanted to make look like it was traditional memory," he said.

The first version of the software required programmers to write a very small percentage of their software — Pai estimates about 1 percent — to work with SSDAlloc. But while completing a scientific internship at Fusion-io last summer, Badam was able to refine SSDAlloc so that programmers no longer have to alter any of their code to work with the system.

"A good thing about SSDAlloc is that it does not alter the program," Badam said. "If you were using RAM and you want to use RAM, you can do that. If you want to use solid state you can use that."

Pai predicts that the need for faster memory access will continue to grow as more computing relies on the virtual cloud rather than on individual machines. The cloud, of course, must be supported by servers running those programs.

"Our system monitors what the host system is doing and moves it into and out of automatically," he said. "There is a whole class of applications in which this would be used."

Explore further: Facebook's Internet.org expands in Zambia

Related Stories

Faster, more durable flash memory sought out for project

Oct 29, 2010

USB flash drives have become nearly ubiquitous and are now dressed out as everything from Winnie the Pooh to hand grenades. But to further expand flash memory’s use in such applications as the main memory of a computer, ...

Elpida Memory develops resistance RAM prototype

Jan 24, 2012

Elpida Memory, the world's third largest Dynamic Random Access Memory ("DRAM") manufacturer, today announced the development of its first-ever high-speed non-volatile resistance memory (ReRAM) prototype. As the ReRAM prototype ...

Elpida and Spansion develop 4-Gigabit NAND flash memory

Sep 02, 2010

Elpida Memory and Spansion today announced they have created the industry's first charge-trapping 1.8 V, 4-gigabit SLC (Single Level Cell) NAND Flash memory. This NAND memory, based on Spansion's MirrorBit charge-trapping ...

Recommended for you

Researchers jailbreak iOS 7.1.2

29 minutes ago

Security researchers at the Georgia Tech Information Security Center (GTISC) have discovered a way to jailbreak current generation Apple iOS devices (e.g., iPhones and iPads) running the latest iOS software.

Smartphones as a health tool for older adults

1 hour ago

A team of researchers from the Universitat Politècnica de Catalunya · BarcelonaTech (UPC) and the Universitat Autònoma de Barcelona (UAB) is creating a smartphone app that will help older adults to understand ...

Can you trust that app?

1 hour ago

You're on your smartphone, browsing through Facebook. In a fit of productivity, you search for, say, a project management app to help you use your non-Instagram and cat video time more effectively. You download ...

Facebook's Internet.org expands in Zambia

Jul 31, 2014

(AP)—Facebook's Internet.org project is taking another step toward its goal of bringing the Internet to people who are not yet online with an app launching Thursday in Zambia.

Body by smartphone

Jul 30, 2014

We love our smartphones. Since they marched out of the corporate world and into the hands of consumers about 10 years ago, we've relied more and more on our iPhone and Android devices to organize our schedules, ...

User comments : 23

Adjust slider to filter visible comments by rank

Display comments: newest first

SteveL
5 / 5 (6) Jul 19, 2012
As SSDs become more popular we should see this technique pervade the industry. Faster data with less energy and SSDs actually able to store and retrieve data at the speeds they are capable of rather than at the speed limitations of the old bus will do wonders for data intensive applications. Kind of hard to miss with this.
dschlink
4.3 / 5 (6) Jul 19, 2012
Other advantages: by using SSD instead of RAM there will less data loss when power goes out. Also, since programs and data will not have to be reloaded, restart times will be faster. The combination of these two factors would allow very fast ramping of processing, both up and down. Processors can be shutdown when not needed and brought back online almost instantly with the required programs and data still resident.
dtxx
3.7 / 5 (3) Jul 19, 2012
I'd like to see the p/e cycle limit on these chips increase substantially before I would want to deploy them in this fashion. At this point I'd call this a virtual memory replacement, not a ram replacement.
Yogaman
5 / 5 (6) Jul 19, 2012
The description in the article, while true, fails to distinguish the technical innovation described in their paper. http://static.use...adam.pdf

Key to the innovation is that SSDAlloc caches multiple objects in SSD pages and in a large RAM page pool. The application gets a small pool of RAM pages for accessing objects. Each application object is allocated on page boundaries, even though most objects are much smaller. When a page faults, SSDAlloc fetches the needed object very quickly from its RAM cache or with minimum delay from the SSD into one of the application's object-per-page pages, and the application resumes. The objects in the RAM cache are compressed to save space.

So, in a way, this is a drop-in virtual memory that supports an object-based compressed RAM drive and SSD virtual memory system. All in all, pretty darn awesome.

Its charts say it works really well, too.
sirchick
2 / 5 (8) Jul 19, 2012
Why are SSDs so expensive for their capacity for consumer market still ! Its taking ages to rival the hard drive....does any one know if it every will rival the hard drive in terms of price for capacity?

I know its already won on the speed front.
that_guy
3 / 5 (4) Jul 19, 2012
sirchick, SSDs are important even when HDDs cost less for different reasons. But more importantly, SSDs are far less expensive than RAM, of which servers need very large amounts of.

The impetus that helps this be effective was the introduction of a 64 bit architecture (Hardware and Software) That can efficiently address exponentially larger amounts of memory.

Then you add the Flash as "Ram" and you have a system that can pull information extremely efficiently, and in fewer steps than pulling it from a HDD.
sirchick
3 / 5 (4) Jul 19, 2012
i didn't say they were not important, they just cost to much for their capacity.....its all well and good that its efficient and takes less steps but i have terabytes of computer data... and a small wallet.

As for a consumer market like the PC im on now, unless they end up as cheap as hard drives i just don't think they are worth that extra speed. The capacity is too small, at least for a home PC we don't have such budgets to be forking out on.

So again i ask why are they still so expensive when hard drives are so cheap...would of thought mechanical hard drives cost more to make than SSD =/
Estevan57
2.1 / 5 (21) Jul 20, 2012
In 2006 the largest hard drive available was 750 gig.
http://www.pcworl...ves.html
Patience, competition and demand will lower the cost of SSD. It already has, if you look at the history of the consumer pricing for it.
The interesting part of the article (as stated by others above) is the way that is is used to its full potential.
Oysteroid
1 / 5 (2) Jul 20, 2012
So is it just a new and advanced caching technique?

I won't even bother with HDD competition issues - pretty clear SSD wins hands down if all you want is to replace hard drives (and can afford the costs).

But using it as replacement RAM is quite another kettle of fish. Are they talking (as it appears) about advanced caching and VM? If that's the case it doesn't sound too revolutionary to me. A good engineering achievement - sure, but hardly more than that.

Or is there more to it than just a better caching algorithm?

(p.s. I'm not trying to critisize or troll the issue, just wondering what the story is).
Msafwan
not rated yet Jul 20, 2012
This remind me of Readyboost in windows. However in Readyboost it assumes Flash drive as a pagefile rather than RAM.
Oysteroid
1 / 5 (1) Jul 20, 2012
In 2006 the largest hard drive available was 750 gig.

In what seems like an ancient history today but was actually 2000-2001 time frame the company I was with decided to upgrade its servers. Hoping to future-proof them a bit a tried to push to have them equipped with the then-new 50GB drives. The idea was nixed not on the grounds of costs even but becausr..."no one will ever need that much".

I was sorely tempted to remind them of the infamous "No one will ever need more than 640KB RAM" but, remembering where my paycheck comes from, refrained.

Today, my home comp got 2TB hard drive and I won't be surprised if in a few years it would seem modest.
SteveL
5 / 5 (1) Jul 20, 2012
Why are SSDs so expensive for their capacity for consumer market still
In large part I'd expect this is due to economics of scale. If SSDs were produced in similar quantities as HDDs they would likely be significantly cheaper than HDDs. When there is some impedus in the market, possibly allowed by the technique developed in this article, you will see the demand increase and the costs reduced for SSDs.
Shootist
3 / 5 (2) Jul 20, 2012
"their eco-footprint"

bah, and hogwash.
sirchick
not rated yet Jul 21, 2012
Why are SSDs so expensive for their capacity for consumer market still
In large part I'd expect this is due to economics of scale. If SSDs were produced in similar quantities as HDDs they would likely be significantly cheaper than HDDs. When there is some impedus in the market, possibly allowed by the technique developed in this article, you will see the demand increase and the costs reduced for SSDs.


But surely the only reason the demand is not there is their price? If there were rivalling hard drives in price over capacity i would buy them now.
PPihkala
not rated yet Jul 21, 2012
SSDs use computer chips to store information. HDs use magnetically plated discs to store information. It would need really big innovations to make chips cheaper than magnetic storage. Even cheaper per bit are magnetic tapes, like DAT that are used to backup hard drives. Of course they are the slowest media to access, but cheap.
Estevan57
1.8 / 5 (19) Jul 22, 2012
http://www.tomsha...276.html

There, Sirchick, be happy.
If you research this a little you will find that demand is very high, and prices are dropping relatively rapidly.
30% from the begginning of 2012 for some brands.
If you squint your eyes and pretend it's 2008 you can get a good buy for the money.

The technology is young compared to HD. The price may never rival, but the consumer has to decide his pricepoint vs speed.

I don't have a need for SSD drive technology, my HD is fast enough, but I would like to see the day when a home PC doesn't need a power supply as big as a dictionary.
JRi
not rated yet Jul 22, 2012
As the program was said to replace RAM with SSD, it would have been interesting to see some comparison about the speed, energy consumption against conventional "all-RAM" system. There were benchmarks where other SSD-systems were compared to this one, tho.
SteveL
not rated yet Jul 22, 2012
I don't have an HDD on the computer I am using now, just a Crucial 128GB SATA 6GBS/s SDD. Takes up less room, uses less energy, it's quieter and it's faster. But yes, I had to pay a bit more. This system is used mainly for crunching (distributed computing) for einstein@home. A larger memory capacity was not needed as I use Ubuntu and don't have unneeded applications in my system.

For now, systems that require massive storage HDDs are still the way to go. Eventually though, I expect SDD to catch up and one day HDDs will be like BetaMax. Some day you'll remember them from the past, eventually we will forget all about them except when we read through some Wiki article about memory storage devices. Still we can expect HDD makers to hang on as long as they can.
PPihkala
not rated yet Jul 22, 2012
It is mindboggling to think how much HDDs have changed in less than 30 years. I still should have somewhere that 5.25 inch full height (=4 inch) HDD with capacity of 10 Megabytes. Yes, 10 MB, not GB or TB as they are today. And then one would also need the ISA bus MFM HDD controller card that doubled the price. Or compare that to todays 8 GB USB Flash stick in terms of space and price. One thing that (USB) Flash memories have obsoleted are floppy discs, with their maximum capacity of 1.44 MB.
antialias_physorg
not rated yet Jul 23, 2012
SDDs are nice (got a 64GB at home for the operating system and programming related activities and a large HDD for storage of non-timecrucial data like music, movies, images, games, etc. )

But I was under the impression that SDDs had limited rewrites (i.e. they degrade over time). While this isn't much of a problem for the average home user I would expect that in large data centers the usage on these suckers would be pretty heavy. Lifetimes might not be that great in those circumstances.
SteveL
not rated yet Jul 23, 2012
It is mindboggling to think how much HDDs have changed in less than 30 years. I still should have somewhere that 5.25 inch full height (=4 inch) HDD with capacity of 10 Megabytes. Yes, 10 MB, not GB or TB as they are today. And then one would also need the ISA bus MFM HDD controller card that doubled the price. Or compare that to todays 8 GB USB Flash stick in terms of space and price. One thing that (USB) Flash memories have obsoleted are floppy discs, with their maximum capacity of 1.44 MB.
Yes the old IBM PC came with a 10 meg HDD. It was expected to be more memory than a user would ever need. Of course they said the same about the 16k of available RAM. I suppose they didn't know about bloatware yet.
bhiestand
not rated yet Jul 23, 2012
It is mindboggling to think how much HDDs have changed in less than 30 years. I still should have somewhere that 5.25 inch full height (=4 inch) HDD with capacity of 10 Megabytes. Yes, 10 MB, not GB or TB as they are today. And then one would also need the ISA bus MFM HDD controller card that doubled the price. Or compare that to todays 8 GB USB Flash stick in terms of space and price. One thing that (USB) Flash memories have obsoleted are floppy discs, with their maximum capacity of 1.44 MB.

1.44 MB? You must be using one of those newfangled "stiff floppies".
eachus
not rated yet Jul 23, 2012
So is it just a new and advanced caching technique?


It isn't simple, nothing is today in computers due to the amount of complexity we are trying to manage. The toy programs in CS100 or whatever introductory computing class you get teach nothing about large memory management.

In this case, what they are doing is replacing a DRAM and hard disk virtual memory management model with one better tuned for today's VM and Flash memory. To grossly simplify what they are doing, when managing millions, billions, or even trillions of objects, they pretend to the user's program that each allocated object has a (64-bit really 48-bit on most hardware today) page-aligned byte address. The rest of the software is doing the VM support to make those addresses work.

Part of "making the VM work" is that objects that "fall out" of the in-memory cache get written to the next available space on the SSD, eliminating the erases and rewrites necessary to put an object back where it came from.