Big data shows people's collective behavior follows strong periodic patterns

November 8, 2016
social network
Social network diagram. Credit: Daniel Tenerife/Wikipedia

New research has revealed that by using big data to analyse massive data sets of modern and historical news, social media and Wikipedia page views, periodic patterns in the collective behaviour of the population can be observed that could otherwise go unnoticed.

Academics from the University of Bristol's ThinkBIG project, led by Nello Cristianini, Professor of Artificial Intelligence, have published two papers that have analysed periodic patterns in daily media content and consumption: the first investigated historical newspapers, the second Twitter posts and Wikipedia visits.

The two sets of findings, taken together, show that people's follows strong periodic patterns and is more predictable than previously thought. However, these patterns can often only be revealed when analysing the activities of a large number of people for a very long time, and until recently this has been a very difficult task.

By using technologies it is now possible to obtain a unified look at newspaper content, for dozens of newspapers at the same time, spanning several decades or to analyse the contents posted on Twitter by large numbers of users, or even the Wikipedia pages visited.

Professor Nello Cristianini, from the Department of Engineering Mathematics, said: "What emerges is a glimpse at the regularities in our behaviour that are hidden behind the day-to-day variations in our lives.

"Our two papers have shown that by analysing massive data sets of modern and historical news, and Wikipedia page views, we can obtain an unprecedented look at our collective behaviour, revealing cycles that we certainly suspected, but that have never been observed before."

The first paper, published in the journal PLOS ONE, analysed 87 years of US and UK newspapers between 1836 and 1922. The researchers found people's leisure and work were strongly regulated by the weather and seasons, with words like picnic or excursion consistently peaking every summer in the UK and US.

Much of our diet was influenced by the seasons too, with very predictable peak times for different fruits and foods, and even flowers, in the historical news. The same was found for diseases, such as the peak season for measles in both countries was found to be in late March to early April. Interestingly, a strong indicator was provided by the very periodic re-appearance of gooseberries every June, which is no longer found in modern news, along with many other lost traditions.

This may seem obvious, but the research team also noticed that certain activities that used to be highly regular, like Christmas lectures, have now all but disappeared, and have been replaced by other periodic activities, like football, Ibiza, Oktoberfest. In some ways, the TV has partly replaced the weather as a major factor of synchronisation of people's lives.

In the second paper, to be presented next month at a workshop at the 2016 IEEE International Conference on Data Mining (ICDM), the researchers discovered that seasons may also have strong effects on . The team analysed the aggregate sentiment in Twitter in the UK, plus aggregate Wikipedia access over four years. They found that negative sentiment is overexpressed in the winter, peaking in November, and anxiety and anger are overexpressed between September and April.

At the same time, an analysis of Wikipedia visits for mental health pages, globally but strongly dominated by northern hemisphere traffic, showed clear seasonality in searches for specific forms of mental issues. For example, visits to the page on seasonal affective disorder peaks in late December and panic disorder visits peak in April, at the same time as visits to the page on acute stress disorder.

Together, these two articles show that the use of multiple sources of big data can enable researchers to look at the collective behaviour, and even the mood and mental health, of large populations, revealing cycles for the first time that have been suspected but were difficult to observe.

Explore further: Public interest in plane crashes only predicted 'if death toll is 50 or higher'

More information: 'Discovering periodic patterns in historical news' by Nello Cristianini et al in PLOS ONE.

'Seasonal fluctuations in collective mood', by Fabon Dzogang, Thomas Lansdall-Welfare and Nello Cristianini to be presented at a workshop at the 2016 IEEE International Conference on Data Mining (ICDM).

Related Stories

Women are seen more than heard in online news

February 3, 2016

It has long been argued that women are under-represented and marginalised in relation to men in the world's news media. New research, using artificial intelligence (AI), has analysed over two million articles to find out ...

Study finds major U.S. newspapers are warming up to Wikipedia

April 12, 2011

( -- Major newspapers in the United States are referencing Wikipedia more often and framing the online encyclopedia more positively in stories, according to a study by a pair of faculty researchers in the Virginia ...

Tracking flu levels with Wikipedia

April 17, 2014

Can monitoring Wikipedia hits show how many people have the flu? Researchers at Boston Children's Hospital, USA, have developed a method of estimating levels of influenza-like illness in the American population by analysing ...

Recommended for you

Scientists write 'traps' for light with tiny ink droplets

October 23, 2017

A microscopic 'pen' that is able to write structures small enough to trap and harness light using a commercially available printing technique could be used for sensing, biotechnology, lasers, and studying the interaction ...

When words, structured data are placed on single canvas

October 22, 2017

If "ugh" is your favorite word to describe entering, amending and correcting data on the rows and columns on spreadsheets you are not alone. Coda, a new name in the document business, feels it's time for a change. This is ...

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (2) Nov 09, 2016
A better title to this article: Using big data to discover the obvious.
Nov 09, 2016
This comment has been removed by a moderator.
not rated yet Nov 10, 2016
Isaac Asimov's Foundation series is a bit better grounded than previously thought. Coming from an engineering career before crossing over to economics due to disability, a statistical basis for behavior *should* sense (econometrics) but the tools weren't there yet. Indeed, Dr. F. N. David's model for prison populations worked even though the explanation just wasn't there, yet. [My teen mentor.] This does make me wonder about further, deeper analytics. Bit frightening as well.

hawkingsbrother: no, what they tend do is keep the current crop of politicians in charge when their "pocketbooks" are doing okay to well. They [attempt] to toss the bums out when they do badly. The "issue" that keeps getting cited by the Trump voters (I'm not one, even though agree) is that the countryside (non-city) population have been screwed for decades, not just recently in "The Great Recesion." More of them turned out than for the other lot, thus the result. Perception.
not rated yet Nov 12, 2016
Jack: re- non-city populations feeling screwed, agreed. However I question if justified? Perhaps that's a clue to puzzling slogans like "Make America Great Again!" Is it an appeal to return to e.g. Edwardian days when the British upper class lived primarily on their estates in the country (the landholdings being the basis of their wealth) and kept a Townhouse only for use on occasional visits?

If that's it then it's never coming back.
not rated yet Nov 14, 2016
A better title to this article: Using big data to discover the obvious.

Indeed! Another historical breakthrough of Dr. O.B. Vious and his team.

I think he should be nominated for the Nobel Prize of Ikebana !

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.