Sitting in a cafe in Sydney awhile back, Danny Kennedy and Andrew Birch mused about all the valuable free data Microsoft and Google float out on the Internet. In particular, they wondered if they could tap into the tech giants' mapping services to remotely design rooftop solar energy systems using satellite images and aerial photos.
Turns out they could. Four years after that "aha" moment, Kennedy and Birch operate Sungevity, a thriving solar energy firm with 100 employees involved in designing residential solar energy systems in three states - without leaving the company's Oakland, Calif., office.
They're doing it using modern business intelligence, or BI. Analyzing large sets of data to rev up business operations is as old as arithmetic. But two big variables have changed. First, the sheer volume of data amassed over the past decade is not just unprecedented, it's mind-boggling.
This year the amount of digital information created and replicated is expected to hit an astounding 1.2 zettabytes. That's 1 trillion gigabytes, enough data to fill a stack of DVDs reaching from Earth to the moon and back, according to researcher IDC. That number will approach 35 zettabytes by 2020; picture a stack of DVDs reaching halfway to Mars.
Second, the general populace has become tech savvy. Most people are at ease surfing the Internet and using mobile devices. Combine the two variables, and you find BI exploding on several fronts.
"It's a very exciting new world," says Vivek Ranadive, CEO of Tibco, a Palo Alto, Calif.-based supplier of BI infrastructure and tools. "The potential exists for 21st-century business analytics to solve problems big and small and make the world a better place."
At a grass-roots level, building inspectors, real estate agents, law enforcers and entrepreneurs dreaming up unique start-ups are tapping public data and doing simple correlations that give them an edge.
Sungevity, for example, relies on Bing Bird's-eye View images - digital photos taken with a special high-resolution camera attached to airplanes hired by Microsoft to fly grid patterns above populated areas around the globe. Shot from 10,000 feet at a 45-degree angle, the images show depth and height of buildings relative to the ground.
Sungevity created special software to calculate roof dimensions and orientation to the sun using Microsoft's photos, no ladders or tape measures needed.
"We're leveraging this technology to protect people from falls and make it easier to scale up the solar energy industry sensibly," Kennedy says.
At another level, companies such as Tibco and Seattle-based Tableau Software are introducing visualization tools capable of quickly intermingling data from multiple sources and displaying correlations in eye-catching, endlessly adjustable charts, timelines and maps.
Not to be left behind, IBM, Oracle, SAP and EMC are acquiring and updating older BI technologies, and touting expanded use of traditional BI systems to help run a widening array of complex operations much more efficiently.
BI may be hot. But it's not always easy, especially in larger organizations. Extracting profitable insights from mountains of data requires getting disparate tech systems to blend, an intrinsically complex endeavor, says Boris Evelson, BI industry analyst at Forrester Research.
What's more, key executives and managers must reach consensus about the wisest correlations to pursue, then engage software developers to design custom programs to get it done. That can be time-consuming - and spark turf wars.
"Treating data as a shared enterprise asset is a new concept," says Bill Hewitt, CEO of Kalido, a Burlington, Mass.-based supplier of BI infrastructure. "It requires everyone involved to be committed to a new way of doing things."
Yet the potential payoffs seem clear. Forrester estimates global spending on BI software will ring in at $9.4 billion this year and rise to $14.1 billion by 2014. That does not include a comparable amount likely to be spent on BI consulting services. "BI is nirvana for software vendors and consultants," says Evelson. "Tons of opportunity for everyone."
IBM, in particular, is betting big on BI. In the past five years, it has spent $14 billion to make 24 BI acquisitions. Its payroll now includes 10,000 BI software developers, 8,000 BI consultants and 200 BI mathematicians. "IBM is dedicated to making business analytics mainstream," says Rob Ashe, general manager of IBM's business analytics and optimization division.
The company generates $9 billion in annual revenue from BI software and consulting services. Its small army of salespeople, consultants and technicians have marching orders to push that figure to $16 billion by 2015.
Testimonials from folks such as Debbie Croshaw, a child protective services supervisor in Clark County, Nev., show how BI systems can improve worker productivity and customer service.
Croshaw used to spend a good chunk of her day hand-tallying entries on reports filed by caseworkers. She would then input the information into spreadsheets of her own creation.
At any given time, Croshaw knew she might be asked to cobble together a report regarding one aspect or another of child neglect and abuse cases in her sector of Greater Las Vegas, a city were residents move frequently. This process often led to confusion and errors.
Then the Clark County Department of Family Services purchased IBM's Cognos analytics system. Consultants arrived like the cavalry to centralize data collection and set up a system to create uniform reports relatively quickly.
This freed Croshaw from her makeshift data-entry chores. And it gave senior officials more direct access to data useful for monitoring key aspects of open cases, such as follow-up visits by caseworkers and the status of children removed from their homes.
"I now have more time I can spend with my staff to focus on our priority of making sure children are safe," says Croshaw.
Having a cleaner, more readily accessible set of data paid off in another way. A data analyst discovered that the county could qualify for a federal Medicaid grant if it kept track of state benefits payments more closely. The IBM Cognos analytics system derived the necessary report, and now Clark County gets $7 million a year in federal funding to help protect children.
"He identified the additional revenue source on his own," says Eboni Washington, Clark County senior management analyst. "Cognos gave us the ability to build the necessary report to submit for claims."
Historical reports have long been the bread-and-butter of traditional BI systems. But Ed Goldfinger, CFO of Cambridge, Mass.-based Zipcar, was looking for a more nimble tool. He had plenty of summaries showing the average daily, weekly and monthly utilization of the car-sharing company's fleet of 8,860 vehicles.
But Goldfinger was frustrated with limitations in manipulating historical data trapped in rigidly formatted reports. "You could try to piece it together with Excel, but it's not visual, it's hard to set up, and a lot of the information isn't that helpful," he recalls.
So Zipcar purchased Tibco Spotfire, a data visualization tool designed to enable ordinary workers to display and analyze information previously accessible to them only with the hand-holding of a tech geek.
After a week's training and a month's experience using Spotfire, Goldfinger and his team of fleet managers became proficient at designing colorful, interactive charts and using them to slice and dice data to their hearts' content. They began to see what every single car was doing every hour of the day in 14 metro areas and at 200 universities.
"What smart people want to do is keep asking questions of the data and have a conversation with the data," Goldfinger says.
The fleet managers created discount offers to encourage evening use, as well as a program to entice business travelers to use Zipcars as an alternative to taxis or rental cars. "It allowed us to offer more interesting deals and improve the efficiency of our business," says Goldfinger.
One big reason data visualization tools such as Tibco Spotfire are gaining traction is that the workforce is more at ease with technology, says Christian Chabot, CEO of Tableau Software, which makes another type of data visualization technology.
Tableau's year-over-year revenue grew 123 percent in the quarter ending Sept. 30, making it the fastest-growing software company in the $10 million to $30 million range in the world, according to Software Magazine. Founded in 2003, Tableau has 182 employees and plans to add 100 more over the next 12 months.
Chabot contends Tableau is being championed by workers in their 20s and early 30s steeped in using cutting-edge online services - supplied gratis by the likes of Google, Amazon and Facebook - to do research, to shop and to socialize.
"The younger people coming into the workforce are looking at the way corporations do things and at the technologies corporations are using and basically laughing," Chabot says.
Tableau has been getting rave reviews from clients such as shipping giant UPS, which has been using Tableau Desktop to correlate route lines and container loads on an interactive map. That allows transportation planners to discover patterns and combine partial loads to save money.
Picture CNN newsman John King spontaneously pulling up cool visuals of voting patterns on a digital map as he did during the 2008 U.S presidential elections, and you get the idea.
"We look at pictures rather than reports," says Martin Click, recently retired UPS senior director for transportation planning. "By applying the tool and seeing the data visually, we saved $2.5 million."
Explore further: Is big data still big news?