Reducing big data using ideas from quantum theory makes it easier to interpret

April 23, 2015

Analysing the large volumes of data gathered by modern businesses and public services is problematic. Traditionally, relationships between the different parts of a network have been represented as simple links, regardless of how many ways they can actually interact, potentially loosing precious information. Only recently a more general framework has been proposed to represent social, technological and biological systems as multilayer networks, piles of 'layers' with each one representing a different type of interaction. This approach allows a more comprehensive description of different real-world systems, from transportation networks to societies, but has the drawback of requiring more complex techniques for data analysis and representation.

A new method, developed by mathematicians at Queen Mary University of London (QMUL), and researchers at Universitat Rovira e Virgili in Tarragona (Spain), borrows from ' well tested techniques for understanding the difference between two quantum states, and applies them to understanding which relationships in a system are similar enough to be considered redundant. This can drastically reduce the amount of information that has to be displayed and analysed separately and make it easier to understand.

The new method also reduces computing power needed to process large amounts of multidimensional relational data by providing a simple technique of cutting down redundant layers of information, reducing the amount of data to be processed.

The researchers applied their method to several large publicly available data sets about the genetic interactions in a variety of animals, a terrorist network, scientific collaboration systems, worldwide food import-export networks, continental airline networks and the London Underground. It could also be used by businesses trying to more readily understand the interactions between their different locations or departments, by policymakers understanding how citizens use services or anywhere that there are large numbers of different interactions between things.

Dr Vincenzo Nicosia, co-author of the paper from QMUL's School of Mathematics, said:

"We've been trying to find ways of simplifying the way big data is represented and processed and we were inspired by the way that the complex relationships in quantum theory are understood.

"With so much data being gathered by companies and governments nowadays, we hope this method will make it easier to analyse and make sense of it, as well as reducing computing costs by cutting down the amount of processing required to extract useful information."

Explore further: Uncovering complex network structures in nature

More information: M. De Domenico, V. Nicosia, A. Arenas, V. Latora, 'Structural reducibility of multilayer networks' (2015) will appear in Nature Communications on Thursday 23 April 2015.

Related Stories

Uncovering complex network structures in nature

December 10, 2014

The global spread of Ebola is due to the complex interactions between individuals, societies, and transportation and trade networks. Understanding and building appropriate statistical and mathematical models of these interactions ...

How big data can be used to understand major events

March 4, 2015

With the most unpredictable UK general election looming in modern times, how can big data be used to understand how elections are covered by the media? New research has for the first time analysed over 130,000 online news ...

Patterns in large data show how information travels

February 16, 2015

According to Fariba Karimi, network scientist at UmeƄ University, analysis of massive online data can reveal what information matters to us and with whom we have most in common. For example, networks can be used to study ...

Recommended for you

Volumetric 3-D printing builds on need for speed

December 11, 2017

While additive manufacturing (AM), commonly known as 3-D printing, is enabling engineers and scientists to build parts in configurations and designs never before possible, the impact of the technology has been limited by ...

Tech titans ramp up tools to win over children

December 10, 2017

From smartphone messaging tailored for tikes to computers for classrooms, technology titans are weaving their way into childhoods to form lifelong bonds, raising hackles of advocacy groups.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

zakaria
not rated yet Apr 28, 2015
1 Weird Trick That Forces Your Eyes Into
perfect 20/20 Vision In Just 7 Days"
With Practically No Effort
So, why wait ?
Click here it watch the free tutorial and details
www bit.ly/EyesIntoperfect

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.