Experts ponder era of big data

And the amount of data stored in such "technological memories" is growing 25 percent a year, said Martin Hilbert, a researcher at the University of Southern California.

He joined an eclectic group of experts in Santa Clara, Calif., recently for a two-day conference sponsored by The Economist magazine to debate what this information overload means for individuals, businesses and society. The collective conclusion: It means greater government and corporate transparency - and threats to personal privacy.

Information technology is a boon for businesses that know how to crunch data for their competitive advantage. But the constant bombardment of tweets, text messages and beeping devices may also be an impediment to the ability of people to focus their attention and form deep thoughts, some participants said.

"It's like fire," said John Perry Barlow, co-founder of the Electronic Frontier Foundation. "It will have all manner of consequences."

Businesses that analyze data about customers increasingly have an edge over their competitors, said James Manyika, director of the McKinsey Global Institute.

He estimates that in 2009, nearly all companies in the United States with at least 1,000 employees on average stored at least 200 terabyes of data - twice the size of Wal-Mart's data warehouse in 1999.

In the past decade, "Our ability to digitize data has gone up dramatically," said Manyika. "Storage capabilities to capture data have changed dramatically. And the Internet has made it very easy to share data."

Retail companies that are savvy about sifting through their data can improve operating margins by 60 percent, he said. But in coming years, Manyika added, the United States will face a severe shortage of workers with deep analytical skills, as well as managers and analysts with the capabilities to make business sense of this information.

"In the next two years, the sexy job will be statistician," he said.

The data deluge is changing all areas of society.

"This is an age of network intelligence, an age of collaboration and an age where there are profound changes fueled by data. Companies and organizations are being scrutinized like never before," said Don Tapscott, author of "Macrowikinomics: Rebooting Businesses and the World."

The digital information era is making government and corporations more transparent, he said.

Yet individuals, wittingly or not, are giving up their privacy, Tapscott said. While advocating for more open government and corporations, Tapscott said he does not accept the Facebook ethos that, "If we are all more open, we will be better people. I don't buy this (notion that) privacy is dead and we should all get used to it. We have governments collecting dossiers on us. Government is not always benevolent."

Internet technology, in a sense, is returning America to an earlier time, when many people lived in small communities where neighbors knew much more about one another, said Naveen Jain, founder and CEO of Intelius, a service that digs up information on individuals.

Throughout the day, people leave technological fingerprints - from video cameras monitoring banks, streets and malls to software that tracks online behavior, such as search results, he said.

"If you want privacy," he added, "you can't expect Google to give you information for free."

Said Barlow: "Everything you do in life leads to a digital slime trail."

Nicholas Carr, author of "The Shallows," a book that argues the Internet hurts the way people think, said multitasking and the constant craving for more information make workers less productive. "If you never pay attention to information closely, you never create that wealth of internal connections that give way to thoughts," he said.

"People who multitask a lot think they are being productive because they are taking in a lot of information," Carr said. "But when you test their productivity, it's actually lower than people who aren't multitasking. There is no such thing as doing two things simultaneously."

The information age is upending life as we know it - much like previous technological changes did, researcher Hilbert said.

"When you are in the middle of it, it's as disruptive as you've ever seen," he said. When automobiles first appeared, "People were killing each other. There was no DMV. There were no driver's licenses. Now we have to create these institutions for the information age."

(c) 2011, San Jose Mercury News (San Jose, Calif.).
Distributed by McClatchy-Tribune Information Services.