IHS, which describes itself as the “largest global economics consultancy in the world”, released a statement on 31 October in which it claimed that “crime in South Africa is at its lowest level in 15 years” and that violent crime is “at the lowest level seen in a decade, declining some 40% between 2002 and 2013”.

The same day, the South African minister of police, Nathi Mthethwa, was quoted in a statement as saying that IHS’s findings “vindicate government’s recently-released national crime statistics”. Mthethwa added that “whilst many local analysts were sceptical in their analysis of the [official] statistics, it is gratifying that an internationally-recognised, credible and globally-respected institution has now affirmed the fact that crime is decreasing in South Africa”.

Mthethwa’s statement received wide coverage in South Africa and internationally. The New Age, News24, The Herald, eNCA, Cape Argus, Mail and Guardian, Witness and The Star all ran versions of a report published by the South African Press Association. They made no attempt to interrogate the police’s claims, or IHS’s findings and methodology. The Pretoria News and Business Day were rare exceptions in explaining how the index was calculated.

But was the IHS correct to conclude that crime has decreased in South Africa? And does the crime index really vindicate the government’s 2012/13 crime statistics?

Police miscalculated crime ratios based on outdated population estimates

South Africa’s latest crime statistics have been marred by controversy since their release on 19 September 2013. The Institute for Security Studies (ISS), an independent research organisation, has said that police used outdated population estimates to calculate the change in crime rates between 2011/12 and 2012/13. The 2011/12 crime rates were calculated using the 2001 census population estimate of 50,6 million people, not the more recent 2011 census figures which showed a population of 52,3 million.

When police statisticians calculated changes in crime rates between 2011/12 and 2012/13 they should have recalculated the 2011/12 crime rate using the most recent census data. Gareth Newham, head of the ISS’s governance, crime and justice division, argues that their failure to do so has resulted in statistically and factually flawed crime statistics.

For instance, according to the police’s calculations, the murder rate only increased by 0.6% over the past two financial years. However calculations done by the ISS, using the most recent population estimates for 2011/12, found that the murder rate had actually increased by 2.6%. Similarly, police claimed that sexual offences cases had decreased by 0.4% when they had actually increased by 1.5%.

Crime index incomparable with crime statistics

David Wilson, a senior analyst at IHS Global Insight Southern Africa, the regional office which compiles the crime index, told Africa Check that the police and most press reports had ignored “important nuances” in the IHS’s index. As Africa Check discovered, the way in which the crime index is calculated makes comparison with the 2012/13 crime statistics impossible.

The IHS uses the crime index to advise their clients on economic risk levels in South Africa. The index is calculated in two steps.

First, the IHS calculates crime rates based on raw police data of the number of crimes committed in a given year, together with population estimates. This results in a crime ratio of the number of crimes committed per 100,000 people. (A previous Africa Check report highlighted the importance of crime ratios which allow for an accurate measure of how crime rates compare across different areas with different population sizes)

The IHS uses its own population estimates based on census data. However, it also considers other available survey data including mortality statistics, birth and death registrations, the number of registered voters and the number of registered school children. At this point, the IHS, ISS and police are using largely the same method, although the IHS’s population estimates differ slightly.

Crime weightings have no objective basis

Significantly, the IHS’s process involves a second step where they weight the crimes. Statistical weighting is used to give certain factors more influence than others.

Neither the ISS nor the police statisticians undertake this step. The IHS weights crimes to emphasise their economic impact and the perceived seriousness of some crimes over others.

Wilson said they elect to weight crimes because the manner in which the police calculate the crime rate is insufficient for the needs of IHS’s commercial clients. The IHS views certain crimes as having a larger impact on the economic risk of investing in South Africa. “For example, a trebling of shoplifting isn’t important to our needs,” he said.  As a result, shoplifting is weighted to count less than other crimes.

The IHS then combines the weighted crimes into a single index figure. The IHS crime index score for South Africa in 2012/13 was 110.11. This was a 1.68 point decrease on 2011/12’s figure of 111.79. According to Wilson, “The actual index number is really, really close to flat [compared to 2011/12]. But when all the crimes were combined it brought the index below last year’s level.”

The ISS and the police do not give statistical weightings to their calculations. “This can lead to incorrect conclusions such as crime decreasing when the opposite is true,” said Newham.  “Also there are no single international standards for weighting crime so it is impossible to then make meaningful comparisons with official statistics and other data sources.”

Tom Moultrie, an associate professor of demography at the University of Cape Town and director of the Centre for Actuarial Research, told Africa Check that the weighting method used by the IHS was questionable. “You could give murder a weighting of ten and mugging a weighting of one, and then ten muggings would be the equivalent of one murder. But there is no objective basis for concluding that,” he explained.

Is crime really at a fifteen year low?

The IHS claims that there has been an overall decrease in crime in the last 15 years. However, the ISS notes that while over the “past eight years between 2002 and 2011 there have been notable decreases in most crime categories”, 2012/13 saw a reversal in this trend. Last year was one of the “worst years in a number of years from a crime reduction point of view”.

And while the IHS claimed that violent crime was “at the lowest level seen in a decade” certain violent crimes increased in 2012/13. Murder, attempted murder, sexual crimes, common robbery, robbery with aggravating circumstances, kidnapping and public violence all increased last year. The IHS crime index appears to overlook these increases.

According to Newham, the crime index is not suitable for analysing crime in South Africa: “They are using a model for the specific purpose of evaluating economic risk in South Africa…and while they are very good at what they do, they are not crime analysts. They should engage with crime experts before making public statements about crime in South Africa.”

Are the government’s 2012/13 crime statistics vindicated?

The police were wrong to claim that the IHS crime index vindicates their crime statistics. Firstly, the crime index is incomparable with normal crime statistics because of its weighting system. It does not accurately reflect the state of crime in South Africa. Secondly, the IHS has expressed it support of the ISS’s criticism of the police’s use of outdated population estimates.

“[The ISS’s] recalculations, taking into account the corrected population figures for 2011/12, are accurate and the right way to do it,” Wilson told Africa said.

“There is a right way to calculate numbers and a wrong way…The police have made a mistake by not taking into account accurate population growth estimates,” he said.

Africa Check contacted police spokesman Zweli Mnisi. When told that the institution the police had described as an “internationally-recognised, credible and globally-respected” supported the ISS’s criticisms of the 2012/13 crime statistics, he replied: “So, what do you want me to say?”

Mnisi refused to comment on the IHS’s support of the ISS findings. “Let the IHS issue a statement and we will respond,” he said.

Conclusion – The police can’t have their cake and eat it too

Police were clutching at straws when they claimed that the IHS crime index vindicated the official crime statistics. The crime index was calculated in a very different manner to the 2012/13 crime statistics. It was calculated to reflect economic risk in South Africa, not accurately reflect the state of crime in the country.

As a result the index and the official crime statistics cannot be meaningfully compared. The IHS gave a statistical weighting to crimes based on their economic impact and perceived seriousness, which could create the misleading impression that crime, is down when it has actually increased.  The index also obscures the increase in violent crimes in 2012/13.

The police continue to dismiss the ISS’s criticisms. But the institute it hailed as “internationally-recognized, credible and globally-respected” has now voiced the same concerns. These concerns must be addressed. The formal independent inquiry suggested by the ISS would be an appropriate manner to do so.

The accuracy of crime statistics in any country must be beyond reproach. By refusing to correct their calculations, or institute a formal independent inquiry, the police risks further damaging its relationship with the public. DM

Africa Check, a non-partisan organisation which promotes accuracy in public debate” and mention our address: www.africacheck.org.

Photo Police officers patrol the streets during violent service delivery protests in Bekkersdal, west of Johannesburg. (REUTERS/Siphiwe Sibeko)