120 petabytes: IBM building largest data storage array ever

August 29, 2011 by Bob Yirka report

(PhysOrg.com) -- As computers big and small grow ever faster, new and better ways to store more data must be developed as well to keep up with the demand. It wasn’t all that long ago that a 1 gigabyte hard drive on a personal computer seemed the stuff of science fiction. On the higher end, ever faster supercomputers require not just more data storage but an ability to save and retrieve data faster as well, otherwise they’d spend more and more time dedicated to doing nothing but searching for that data. To address the situation, IBM is apparently hard at work assembling the largest data array ever, which will utilize the fastest data storage and retrieval system ever devised, allowing for the storage of 120 petabytes of data (120 million gigabytes) using its newly refined GPFS file system that is capable of indexing 10 billion files in just forty three minutes.

Housed in IBM’s Almaden, California research facility, the new as yet unnamed data array is reportedly being underwritten by an unnamed client with a lot of money to spend. Based on the situation with the U.S. government’s financial situation, it seems more likely the client is someone like Microsoft, Apple or Google, though for what purpose such a huge array would serve is a mystery. Traditionally, supercomputers and their huge data servers have been used to model weather, military or physics experiments. Now however, the focus might be on ways to crunch ever more data in a faster fashion, which sounds a lot like something Google might be interested in doing, especially when you consider that so much data is moving onto the cloud.

The array works by connecting 200,000 standard (likely 1 terabyte drives) together in a traditional warehouse, though more tightly compacted together than usual in special extra wide drawers. The drives will be kept cool by normal air-conditioning in conjunction with a water cooling system, something has been using for years with its supercomputers; when finished the array will dwarf current systems, which are typically able to store just 15 petabytes of data. To make sure drive failures don’t slow down computations on the supercomputer, data is stored in a redundant fashion over multiple drives, which are striped to allow for accessing large chunks of data simultaneously. When a drive dies, data is moved slowly to its replacement, spreading processing time over a relatively long period, allowing the processor to focus on its main tasks.

The new array, while seemingly for the moment to be almost overkill, is likely to be followed by other even larger systems as more and more data storage becomes necessary. The hope though, is that some new breakthrough will occur that will allow for huge increases in capacity that won’t mean simply adding more and more conventional medium to existing technology.

More information: via Technology Review

© 2011 PhysOrg.com

Filter


Move the slider to adjust rank threshold, so that you can hide some of the comments.


Display comments: newest first

antialias_physorg
Aug 29, 2011

Rank: 4.1 / 5 (12)
This is a hot contender for "most vague article, ever"

- We don't know who orderd it [insert speculation about some companies].
- We don't what it's for [insert speculation about some uses].
- And it's "likely to be followed by even larger systems" [insert speculation that there may be future technologies in the future...Duh].

Result: I don't know what I'm supposed to do with an article like that [insert speculation about the aptitude of the author for journalism].
frajo
Aug 29, 2011

Rank: not rated yet
Information retrieval:
IBM - and not - will be building the largest data storage system ever. May other companies jostle for the largest share of media hype and personality cult, IBM delivers.
Non-bloated PR as a pleasant touch of sobriety.
gimpypoet
Aug 29, 2011

Rank: 1 / 5 (2)
The singularity has already happened and needs the storage and speed increase to wipe us out.this is evident with all the other "unexplainable" happenings within computer controlled happenings such as speed of light trading in the stock market.this accounts for the unknowns above.
finitesolutions
Aug 29, 2011

Rank: 1 / 5 (2)
this is information technology evolution. HD surveillance cameras will fill an 1TB disk in hours. supercomputers need superdata capacity storage. 128bit processors will happen also in the future.
ekim
Aug 29, 2011

Rank: 1 / 5 (2)
The information stored in a 100-trillion-synapse brain?
Mapping of a persons brain after death could lead to immortality.
Vendicar_Decarian
Aug 30, 2011

Rank: 5 / 5 (3)
Largest porn collection ever.
antialias_physorg
Aug 30, 2011

Rank: not rated yet
Mapping of a persons brain after death could lead to immortality.

Making a (working) copy still means that the (biological) original dies at some point. So from the point of view of the individual copied he hasn't gained anything.
Ricochet
Aug 30, 2011

Rank: not rated yet
Yes, this new system could hold all the information in the library of congress several thousand times over, or about 1/8 of all the porn on the internet.
ekim
Aug 30, 2011

Rank: not rated yet
Mapping of a persons brain after death could lead to immortality.

Making a (working) copy still means that the (biological) original dies at some point. So from the point of view of the individual copied he hasn't gained anything.

I doubt a "working" model is possible using current technology. However in another decade or so a digital version might work containing all the memories and thoughts of the deceased. Further ahead, nano technology may be able to reproduce the brain and body using this saved data and stored DNA. Cryogenics for the digital age.
abhishekbt
Sep 01, 2011

Rank: not rated yet
...allowing for the storage of 120 petabytes of data (120 million gigabytes)


Correct me if I am wrong, but 1Petabyte is 1024Gigabyte.
So, 120PB = 120 * 1024GB which equals only 122,880 GigaBytes

Where did the author get 120 million? That's a thousand times more! This article has poor quality written all over it.
antialias_physorg
Sep 02, 2011

Rank: not rated yet
1Petabyte is 1024Gigabyte

No. 1 terabyte is 1000 gigabytes. 1 petabyte is indeed a million gigabytes.

(As an aside: If you go by powers of two then it's not gigabytes (GB), terabytes (TB), etc. but gibibytes, tebibytes, ... and get the abbreviations GiB, TiB, etc. )
Ricochet
Sep 02, 2011

Rank: not rated yet
The only time 1024 comes into it is when you list the actual bytes.
1024 bytes = 1kB
So when you're converting from kB to MB to GB and wherever from there, it's all 1 to 1.
But when you go back down to bytes, you need to add the extra 24 bytes for each kB.
So, 1PB = 1,000,000,000,000,000,000kB = 1,024,000,000,000,000,000,000 bytes

In actuality, it's all translating down to bytes, and to binary bits from there. the kB, MB, GB TB, PB, etc., are just for our convenience.
CHollman82
Sep 02, 2011

Rank: not rated yet
The only time 1024 comes into it is when you list the actual bytes.
1024 bytes = 1kB
So when you're converting from kB to MB to GB and wherever from there, it's all 1 to 1.
But when you go back down to bytes, you need to add the extra 24 bytes for each kB.
So, 1PB = 1,000,000,000,000,000,000kB = 1,024,000,000,000,000,000,000 bytes


False.

Binary values are powers of 2. 2^10 = 1k, 2^20 = 1m... 2^20 = 1024 * 1024... not 1024 * 1000...

kilo = 2^10
mega = 2^20
giga = 2^30
tera = 2^40
peta = 2^50

I often wonder where people get their information...
CHollman82
Sep 02, 2011

Rank: 5 / 5 (1)
The reason this is done is for computational efficiency. A division/multiplication by 1024 is a simple 10 bit left/right shift that can be done in hardware using a barrel shifter. A division/multiplication by 1000 takes much longer to accomplish by the processor and is done in a sequence of several steps.
antialias_physorg
Sep 02, 2011

Rank: 5 / 5 (1)
kilo = 2^10
mega = 2^20
giga = 2^30
tera = 2^40
peta = 2^50

kilo literally means "a thousand" (not 2^10). It's derived from a greek word.

("mega" just means "big", but in SI convention mega, giga, etc. are reserved for powers of ten, not two)

CHollman82
Sep 02, 2011

Rank: not rated yet
kilo = 2^10
mega = 2^20
giga = 2^30
tera = 2^40
peta = 2^50

kilo literally means "a thousand" (not 2^10). It's derived from a greek word.

("mega" just means "big", but in SI convention mega, giga, etc. are reserved for powers of ten, not two)



You're dropping context... I was not giving a lesson on etymology.
antialias_physorg
Sep 02, 2011

Rank: not rated yet
Manufacturers use the powers of ten for their product (otherwise they'd have to give you more memory for the same label). So you can bet that these terabyte drives mentioned in the article do not contain 2^20 bytes but merely 10^12 bytes each.
CHollman82
Sep 02, 2011

Rank: not rated yet
Manufacturers use the powers of ten for their product (otherwise they'd have to give you more memory for the same label). So you can bet that these terabyte drives mentioned in the article do not contain 2^20 bytes but merely 10^12 bytes each.


I wouldn't be surprised... but engineers don't have that "ripping off the customer" bias.
Ricochet
Sep 02, 2011

Rank: not rated yet
The reason this is done is for computational efficiency. A division/multiplication by 1024 is a simple 10 bit left/right shift that can be done in hardware using a barrel shifter. A division/multiplication by 1000 takes much longer to accomplish by the processor and is done in a sequence of several steps.


Actually, it's because 1024 is divisble by eight, whereas 1000 is not. Eight bits in a byte... Don't put more into it than there actually is.
antialias_physorg
Sep 02, 2011

Rank: not rated yet
I wouldn't be surprised... but engineers don't have that "ripping off the customer" bias.
Unfortunately (we) engineers aren't the ones who decide stuff like that. I've seen it often enough. Giving the customer maximum benefit is NEVER a winning strategy in a company - making maximum profit is

(Just look at Apple. They have perfected the sell-overpriced-junk-through-media-hype marketing ploy. Their trinkets are fully 60% profit and the specs are abominable compared to other products on the market. )
TheGhostofOtto1923
Sep 04, 2011

Rank: not rated yet
This is a hot contender for "most vague article, ever"

- We don't know who orderd it [insert speculation about some companies].
- We don't what it's for [insert speculation about some uses].
- And it's "likely to be followed by even larger systems" [insert speculation that there may be future technologies in the future...Duh].

Result: I don't know what I'm supposed to do with an article like that [insert speculation about the aptitude of the author for journalism].
Hey auntie did you check the link at the end of the article, or did you just jump at the chance to dis somebody? [insert speculation about the nature of aunties gut bile and lack of energy for exploring references]
antialias_physorg
Sep 04, 2011

Rank: not rated yet
Yes, I did check it, and that article actually has some substance - my gripe was that the type of 'writeup' that the physorg editor did here is completely useless. A naked link would have been equally informative.
abhishekbt
Sep 04, 2011

Rank: not rated yet
Thanks for correcting me!
I forgot there's Tera after Giga and before Peta!
CHollman82
Sep 06, 2011

Rank: not rated yet
The reason this is done is for computational efficiency. A division/multiplication by 1024 is a simple 10 bit left/right shift that can be done in hardware using a barrel shifter. A division/multiplication by 1000 takes much longer to accomplish by the processor and is done in a sequence of several steps.


Actually, it's because 1024 is divisble by eight, whereas 1000 is not. Eight bits in a byte... Don't put more into it than there actually is.


And why are there 8 bits in a byte instead of, say, 10?... because it is a binary power of two.

The answer is that 1024 is a power of 2, and 1000 is not... as I said.

I'm a software and firmware engineer with 15 years of experience.
Rank 4 /5 (6 votes)
Relevant PhysicsForums posts

More news stories

Nvidia trumpets Tegra 3 phone design wins for 2012

(Phys.org) -- Nvidia’s competitive war paint has a name, Tegra 3. On the heels of Nvidia announcements about lowering costs of its Tegra 3 processors and Nvidia-enabled tablets running Android Ice Cream ...

Electronics / Hardware

created 7 hours ago | popularity 5 / 5 (2) | comments 1 | with audio podcast report

Dell tablet leak: 10.1-inch display, two-battery choice

(Phys.org) -- Headline after headline talks about vendors’ tablets in the wings as likely number-one contenders for the iPad. Such claims have justifiably been taken with a grain of salt, considering ...

Electronics / Consumer & Gadgets

created 18 hours ago | popularity 5 / 5 (3) | comments 8 | with audio podcast report

Nvidia says Kai platform will turn price tide for tablets

(Phys.org) -- In March, Nvidia gave some signs that they were working to lower the cost of their Tegra 3 processors and they suggested consumers might see prices for Android tablets as low as $199. Connect ...

Electronics / Hardware

created May 24, 2012 | popularity 4.3 / 5 (4) | comments 3 | with audio podcast report

OmniVision tops up sensors for cameras, phones

(Phys.org) -- OmniVision has announced two high-resolution image sensors for the digital still and digital video camera market (DS/DVC) and higher end smartphones. In end-user language, it is a claim for superior ...

Electronics / Hardware

created May 25, 2012 | popularity 5 / 5 (6) | comments 3 | with audio podcast report

MIT researchers devise new means to synchronize a group of robots (w/ Video)

(Phys.org) -- For several years, roboticists have been working out ways to get a group of robots to perform synchronized activities as demonstrated most often in dance routines. It’s not just about trying ...

Electronics / Robotics

created May 25, 2012 | popularity 5 / 5 (1) | comments 1 | with audio podcast report


Browser wars flare in mobile space

The browser wars are heating up again, but this time the fight is for dominance of the mobile Internet.

Scientist: Evolution debate will soon be history

(AP) -- Richard Leakey predicts skepticism over evolution will soon be history. Not that the avowed atheist has any doubts himself.

Keep food safety in mind this memorial day weekend

(HealthDay) -- Picnics, parades and cookouts are as much a part of Memorial Day weekend as tributes to the United States' war veterans.

Social welfare cuts ultimately come with heavy price, researchers say

(Phys.org) -- Slashing government funding for Medicaid, food stamps and other programs that serve the poor – while politically popular with some lawmakers and many conservatives – may do more harm ...

Is a classical electrodynamics law incompatible with special relativity?

(Phys.org) -- The laws of classical electromagnetism that were developed in the 19th century are the same laws that scientists use today. They include Maxwell’s four equations along with the Lorentz la ...

Landmark calculation clears the way to answering how matter is formed

(Phys.org) -- An international collaboration of scientists, including Thomas Blum, associate professor of physics, is reporting in landmark detail the decay process of a subatomic particle called a kaon – ...