Why the global survey on safety is deeply flawed

August 7, 2018 by Guy Lamb, The Conversation
Credit: CC0 Public Domain

There has been a rise in global statistical initiatives that measure and rank countries in terms of various aspects of the human condition. Some of the more prominent examples include the Human Development Index, the World Governance Indicators, the Global Peace Index and the Corruption Perceptions Index.

Each ranks countries according to a series of indicators, or a composite indicator, and tracks their progress or decline over time.

One of the most recent global indicator projects is the Gallup Law and Order Index. It ranks 142 countries based on a perception survey relating to personal safety and policing, from a representative sample of 1000 people in each country. Knowing how secure, or insecure people feel is important because insecurity affects economic growth and undermines development.

According to the recently released 2018 law and order index, South Africa ranks high in the insecurity index – 137 out 142 countries. This means that South Africans would have expressed high levels of insecurity as well as fear that they were likely to, or had already, fallen victim to crime.

The ranking suggests that South Africans consider themselves to be more insecure, and having lower levels of confidence in the police, compared to people in Yemen, the Democratic Republic of Congo (DRC), the Central African Republic (CAR), Libya and Mali. These are all unstable states affected by violent conflict and high levels of instability.

This is surprising given that South Africa is not in state of armed conflict and is relatively stable. The possible reason for such a questionable ranking is that the survey, like many global perception surveys, doesn't adequately account for the extent to which people will provide unreliable information about sensitive issues. To improve accuracy, surveys like this should factor in differences in context.

The rankings

The rankings are based on an index score derived from responses to the following questions:

  • In the city or area where you live, do you have confidence in the local police force?
  • Do you feel safe walking alone at night in the city or area where you live?
  • Within the last 12 months, have you had money or property stolen from you or another household member?
  • Within the past 12 months, have you been assaulted or mugged?

It's undeniable that South Africa has high levels of insecurity and interpersonal crime. And, there's a significant trust deficit between citizens and the police.

For example, the StatsSA 2016/17 Victims of Crime Survey showed that only 30% of South African's reported feeling safe walking at night in their neighbourhoods. Only 57% of households reported that they were "satisfied" with the police in their communities. And the country has very high levels of crime.

Nevertheless, it seems odd that South Africa is ranked below countries like Yemen, which has been in the throes of an intense civil war for several years, the Central African Republic and Libya, which have been acutely affected by insurgency, criminality and weak law and order institutions.

Ranking South Africa below the DRC, Mali and Libya is also questionable given that the security forces and militias in those countries have been widely regarded as predatory and highly abusive.

So what's missing?


Firstly, context is key.

A key shortcoming of using survey data about crime and insecurity to construct indices and rankings is that people won't always reply to questions honestly and accurately.

In stable democracies respondents will often give precise and truthful responses as there is little or no fear of reprisals from the state. Conversely, in unstable countries that have repressive governments, research shows that citizens are less willing to provide accurate information about personal experiences of crime and policing. This is because they fear there may be negative repercussions for them and their families.

Secondly, as research method experts have argued, survey responses can also be influenced by a variety of societal norms, particularly those related to privacy and dignity, in which sensitive matters aren't easily discussed with strangers.

In South Africa, citizens are generally willing to talk openly about crime and to criticise the police. But, this isn't the case in many other African and Latin American countries that were rated as being safer. These include DRC, Libya, Honduras and Mexico.

This is not to say that constructing indices about crime victimisation and policing on a country basis is irrelevant. But the danger of indicators like this, and adopting a ranking approach without careful consideration of the context in which the data is gathered, is that it could lead to wrong perceptions about crime and policing. That may even reinforce the use of militarised policing strategies, which will further undermine human security over less aggressive and more integrated approaches to crime prevention. Examples of where this has happened include Brazil, Mexico and South Africa.

To improve the accuracy of indices like this, it would be advisable to develop a quantifiable weighting for the reliability of and insecurity survey data for each country, and then apply the weighting to the overall index score. For example, in countries with more authoritarian governments, respondents are likely to under report their levels of trust in the and sense of personal insecurity.

Applying a reliability weighting would adjust the overall insecurity index score to better reflect people's lived reality. Such a weighting can be developed by including additional questions in the survey, for example about how willing respondents are to talk to strangers about sensitive information, including views about their governments.

Explore further: How people view crime depends on the politics of when they were growing up

Related Stories

Murder rates highest in countries that lack due process

July 17, 2018

Governments that do the best job protecting the rights of the accused have the lowest murder rates, while those that neglect due process have the highest, according to a five-year study of 89 countries by CU Boulder political ...

Study suggests proactive policing may do more harm than good

September 26, 2017

(Phys.org)—A pair of researchers, one with Louisiana State University, the other the University of Michigan has conducted a study on the benefits of proactive policing and have found an example where removing it appeared ...

Model suggests fear of crime is contagious

July 12, 2017

(Phys.org)—A pair of researchers at University College London has found evidence suggesting that fear of crime is contagious. In their paper published in Proceedings of the Royal Society A, Rafael Prieto Curiel and Steven ...

Recommended for you

Sensual fresco discovered in ancient Pompeii bedroom

November 19, 2018

Archaeologists have found a fresco in an ancient Pompeii bedroom that depicts a sensual scene of the Roman god Jupiter, disguised as a swan, and a legendary queen of Sparta from Greek mythology.

Excavators find tombs buried in Bolivia 500 years ago

November 17, 2018

Archaeologists say they found tombs at a Bolivian quarry containing remains from more than 500 years ago that give an insight into the interaction of various peoples with the expanding Inca empire.

Preventing chemical weapons as sciences converge

November 15, 2018

Alarming examples of the dangers from chemical weapons have been seen recently in the use of industrial chemicals and the nerve agent sarin against civilians in Syria, and in the targeted assassination operations using VX ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.