Inside the hidden web

Looking for information? Google it, right? Maybe not. Senior Lecturer in the Faculty of Arts and Social Sciences (FASS) Maureen Henninger is helping journalists, writers and information knowledge management professionals use data mining and data visualisation to unearth new ideas.

Google will help you find "less than 50 per cent of the information that's out there," says Henninger. The rest, she says, is tucked away on "the hidden web".

It's an area Henninger has researched for more than 20 years. And it's at the heart of a subject she has developed and teaches at UTS – Investigative Research in the Digital Environment, or IRDE as her students call it.

"We offered it for the first time three years ago, and it's proved very popular. So much so, that it runs both semesters, and, to my knowledge, it was the first subject of its kind in Australia." Last year, Henninger received a UTS Learning and Teaching citation for the subject, which is offered as an elective in FASS.

IRDE deals with two types of information – unstructured (like text and photographs) and quantitative (like statistics) – which is extracted and re-used. "To re-use it, it has to be analysed and re-written. What is exciting about this particular subject is that one of the ways we re-use it is to visualise it."

Data mining and data visualisation are growth areas, says Henninger. Not only among information knowledge management professionals who design and manage information in social and business contexts, but also among journalists and writers.

The move is being driven by an increasing access to data. "More and more, because of the web and because governments are being pushed to make these things more transparent, journalists have begun to listen to people like me and realise you can actually take that data and use it in a visual way to support a story.

"What's more interesting, and this is what I am most interested in, is taking the data, visualising it and finding the story in the visualisation."

Reporter with Background Briefing on the ABC's Radio National Wendy Carlisle agrees. "This is the future of journalism."

She enrolled in IRDE last year because she "realised there was a lot of stuff on the internet I didn't know how to get my hands on.

"I've been doing journalism for two decades, and after the first class I walked out of there and I said to someone standing next to me, 'Oh gee, I thought I was a pretty good researcher but I've just learned more in two hours than I've learned in the last 20 years'."

During the first five weeks of the subject, students learn new ways of finding information. Then they're taught how to use more complex tools to visualise the data.

"Along the way they have to learn about what we call data integrity or graphical integrity," says Henninger. "You can do a graphic that tells lies and Excel can tell you all sorts of lies if you want it to," she laughs.

There are also three assignments. The first is "strictly finding and visualising data". The second is to deconstruct a story Henninger has redacted and develop a model for finding the data it refers to. The third is to write a story and visualise it using the model from assignment two and computer aided tools.

Already IRDE students have uncovered some surprising stories on US military spending, Australian coal seam gas and more. The latter, pitched to the ABC by Carlisle, became Coal Seam Gas By The Numbers, an exposé launched online last November.

"We built an interactive map of Australia which accurately showed where every coal seam gas well was and people could search by postcode or by type. So it was really the very first time this information had been visually mapped for people," says Carlisle.

The project, which employed Henninger as a data consultant, was the first of its kind in Australia. And Carlisle says, "the ripple effect of that is quite significant in the organisation.

"Because we're living in a digital world, places like the ABC, with its multiple platforms, can use these skills to tell stories in a way that you can't do on radio or on television. It's a whole new space and because there's so much of this information out there, unless you have the tools to understand it and look at it, I don't think you can do your job effectively as a journalist anymore."

Henninger agrees. "This is exciting stuff. Digital recovery, digital re-use is cutting edge and I'm enthusiastic about it and I want my students to be.

"I'm very, very curious, I have a great sense of wonder and I'm very, very enthusiastic and I think that translates."

Carlisle concurs. "Maureen's highly specialised and she's got an incredible knowledge of the internet, its history and how to search. She understands it really intimately and she's intuitive about it.
 
"She's very witty and she's incredibly well-prepared and engaging." And because a lot of the class is a workshop, "it's very hands-on; it's not chalk and talk".

The ever-changing nature of the digital environment also means Henninger has to work hard to ensure the subject content remains current.  "Every one of my lectures has to be re-written every time I teach it.

"I'm a nerdy IRDE or an IRDE nerdy," she laughs.
 
Her favourite part of the subject? "Well there's probably two. One is diabolical, and that is finding a question Google can't answer. But the other is watching students run with it and seeing what they come up with."

Henninger's outlook is one reason Carlisle is such a strong advocate for the subject. "Everybody in any media should do it, because they've got no idea what they don't know right now.

"It will be the hardest course you've ever done in your life because you actually have to learn new techniques to search databases and directories and it's a new language. But it is incredibly exciting.

"My fervent hope is that her course becomes a core subject in journalism. No one should walk out of the UTS school of journalism without that subject under their belt."