Experts ponder era of big data

June 17, 2011 By John Boudreau

The era of big data has arrived. Last year, consumers and businesses around the world are estimated to have stored more than 13 exabytes of information on PCs, laptops and other devices - the equivalent of more than 52,000 times the information housed in the Library of Congress. An exabyte is 1 followed by 18 zeros, or a billion gigabytes.

And the amount of data stored in such "technological memories" is growing 25 percent a year, said Martin Hilbert, a researcher at the University of Southern California.

He joined an eclectic group of experts in Santa Clara, Calif., recently for a two-day conference sponsored by The Economist magazine to debate what this means for individuals, businesses and society. The collective conclusion: It means greater government and corporate transparency - and threats to personal privacy.

technology is a boon for businesses that know how to crunch data for their competitive advantage. But the constant bombardment of tweets, text messages and beeping devices may also be an impediment to the ability of people to focus their attention and form deep thoughts, some participants said.

"It's like fire," said John Perry Barlow, co-founder of the . "It will have all manner of consequences."

Businesses that analyze data about customers increasingly have an edge over their competitors, said James Manyika, director of the McKinsey Global Institute.

He estimates that in 2009, nearly all companies in the United States with at least 1,000 employees on average stored at least 200 terabyes of data - twice the size of Wal-Mart's data warehouse in 1999.

In the past decade, "Our ability to digitize data has gone up dramatically," said Manyika. "Storage capabilities to capture data have changed dramatically. And the Internet has made it very easy to share data."

Retail companies that are savvy about sifting through their data can improve operating margins by 60 percent, he said. But in coming years, Manyika added, the United States will face a severe shortage of workers with deep analytical skills, as well as managers and analysts with the capabilities to make business sense of this information.

"In the next two years, the sexy job will be statistician," he said.

The data deluge is changing all areas of society.

"This is an age of network intelligence, an age of collaboration and an age where there are profound changes fueled by data. Companies and organizations are being scrutinized like never before," said Don Tapscott, author of "Macrowikinomics: Rebooting Businesses and the World."

The digital information era is making government and corporations more transparent, he said.

Yet individuals, wittingly or not, are giving up their privacy, Tapscott said. While advocating for more open government and corporations, Tapscott said he does not accept the Facebook ethos that, "If we are all more open, we will be better people. I don't buy this (notion that) privacy is dead and we should all get used to it. We have governments collecting dossiers on us. Government is not always benevolent."

Internet technology, in a sense, is returning America to an earlier time, when many people lived in small communities where neighbors knew much more about one another, said Naveen Jain, founder and CEO of Intelius, a service that digs up information on individuals.

Throughout the day, people leave technological fingerprints - from video cameras monitoring banks, streets and malls to software that tracks online behavior, such as search results, he said.

"If you want privacy," he added, "you can't expect Google to give you information for free."

Said Barlow: "Everything you do in life leads to a digital slime trail."

Nicholas Carr, author of "The Shallows," a book that argues the Internet hurts the way people think, said multitasking and the constant craving for more information make workers less productive. "If you never pay attention to information closely, you never create that wealth of internal connections that give way to thoughts," he said.

"People who multitask a lot think they are being productive because they are taking in a lot of information," Carr said. "But when you test their productivity, it's actually lower than people who aren't multitasking. There is no such thing as doing two things simultaneously."

The information age is upending life as we know it - much like previous technological changes did, researcher Hilbert said.

"When you are in the middle of it, it's as disruptive as you've ever seen," he said. When automobiles first appeared, "People were killing each other. There was no DMV. There were no driver's licenses. Now we have to create these institutions for the information age."

Explore further: Coalition wants US privacy law revamped for Internet Age


Related Stories

The case for oversharing

June 24, 2010

There's been a lot of squealing of late over our rapidly eroding privacy on the Web, with the finger of accusation pointed directly at those sneaky Facebook folks. Founder Mark Zuckerberg is now the Snidely Whiplash of the ...

Globe Talk: Money power vs. the NSA

May 19, 2006

Outraged that some telecom companies are selling out their customers to the government? Then vote with your wallet, or so argues one activist group. But the seemingly good idea may be rather difficult to follow through.

US senators introduce online privacy 'bill of rights'

April 12, 2011

US senators John Kerry and John McCain introduced an online privacy bill Tuesday that seeks to strike a balance between protecting the personal information of Web users and the needs of businesses to conduct electronic commerce.

We can have ‘win-win’ on security vs. privacy

March 26, 2007

People think there has to be a choice between privacy and security; that increased security means more collection and processing of personal private information. However, in a challenging report to be published on Monday ...

Recommended for you

New method analyzes corn kernel characteristics

November 17, 2017

An ear of corn averages about 800 kernels. A traditional field method to estimate the number of kernels on the ear is to manually count the number of rows and multiply by the number of kernels in one length of the ear. With ...

Optically tunable microwave antennas for 5G applications

November 16, 2017

Multiband tunable antennas are a critical part of many communication and radar systems. New research by engineers at the University of Bristol has shown significant advances in antennas by using optically induced plasmas ...


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (1) Jun 17, 2011
I'm sorry but you are arguing for internet censorship and institution? Again no thanks, your advocacy of this is a threat to my freedom and sovereignty. You're right the government isn't always benevolent, so we definitely shouldn't be trusting them with a task of this great import! Otherwise we will pay a really big price, we will have just handed the system a golden ticket to a potential tyranny. I don't suggest we head down this road.
not rated yet Jun 17, 2011
13 exabytes seems a little low. Using 3TB hard drives, figuring them at a slightly generous 105mm x 27mm x 210mm, times 4,333,333 drives, that's 2580 cubic meters, or a 13.7m (45ft) cube which would cost about $563M at retail. On the other hand, I wonder how much of that data is duplicates, and what the optimally compressed amount of data would be.
not rated yet Jun 18, 2011
"The study goes on further to explain that only a mere 3% of the total internet capacity is not related to porn"

Good stuff.
not rated yet Jun 20, 2011
Big data = Opportunity for big knowledge. The problem we face is that our institutions are failing to evolve as fast as our technology. Kurtzweil's Law of Accelerating Returns has no counterpart in organizational theory.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.