No, Volkswagen Polo drivers are not ‘the worst’ drivers in SA

South African Volkswagen Polo owners have been vindicated and are no longer the worst drivers in the country, according to a thorough fact check. 

Africa Check explained that a widely reported on press release titled ‘Volkswagen Polo Drivers are the worst, a study confirms’, misinterpreted a study of fatal crashes in South Africa. The press release was issued by the Road Traffic Management Corporation (RTMC) last month.

The RTMC’s study, South African Fatal Crashes in Context, analysed fatal crash data from 1 October 2017 to 30 June 2021, as well as Aarto infringement data from 1 January 2019 to 20 June 2021. 

But the study reported on by various media houses including The Citizen did not actually find Polo drivers to be the worst drivers. 

The claim is a statistical blunder and should never have been reported as widely as it was, Africa Check said. 

Polo is a common car 

Instead of revealing the ability of Polo drivers, the RTMC’s study found a less exciting fact: the Polo is a common car to own in South Africa. 

Lightstone Auto is a market analytics company and records vehicle sales data and other information. A spokesperson from the company gave Africa Check its estimates of the most popular cars in South Africa. The Polo was the second and the Polo Vivo the fourth most common model in the country. (The RTMC did not distinguish between the two.) 

Together, these two models were more popular than any other single model of car and made up around 6.36% of all “units in circulation”.

Dr Leanne Scott, an associate professor of statistics at the University of Cape Town, told Africa Check that for a study to reliably determine which drivers are “the worst”, it would need to control for several factors. 

To begin with, “the study would need to take into account the base rates of car ownership”. This refers to the proportion of drivers that drive each type of car. That the Polo is so popular could have a significant impact on the results.

The report acknowledged this, though it used the National Administration Traffic Information System (Natis), a national register which stores, among other information, the licence and registration details of South African vehicles, to estimate the total number of vehicles on the roads and which were involved in crashes.

In its breakdown of the makes and models of cars involved in the highest number of fatal crashes, the RTMC estimated that Polos made up roughly 17.0% of all cars by model, and were involved in 16.7% of all fatal crashes.

This showed that the number of Polos in fatal accidents was not out of proportion with the number on the roads. The study actually noted that Polos were not overrepresented in fatal crash statistics, only in recorded speeding infringements.

But this wasn’t the only factor the study failed to control for.

Laundry list of issues with study

The RTMC could link only 66.9% of cars involved in fatal crashes to a vehicle in the Natis database. This places an obvious early limitation on the accuracy of the study and its conclusion. 

It is possible to draw reasonable conclusions from an incomplete data set. However, the RTMC study has other issues which make it impossible to determine which South African drivers are “the worst”.

Dr Sebnem Er, also a researcher and lecturer in the statistics department at the University of Cape Town, told Africa Check that “lots of other factors need to be considered when dealing with this type of data”.

A particular type of car, for example, might be more likely to be involved in a fatal crash, regardless of driver ability. Er pointed to the Toyota Quantum as an example.

The Quantum was involved in the third highest proportion of fatal crashes of any model of car in South Africa, but Er explained that it shouldn’t be included in the same category as cars like the Vivo. Er said that “almost all Toyota Quantums in South Africa are minibus taxis”, a common form of public transportation in the country. According to Er, this – whether a vehicle is a taxi or not – “needs to be controlled [for] first”.

The study did note some of these factors, but did not control for them in its analysis, or compare only the same types of vehicle in its comparison of makes and models.

Why can’t some types of vehicle be compared to others? One South African media outlet, which reported accurately on the RTMC study, explained it well.

The Mail & Guardian’s head of digital Adam Oxford wrote: “A taxi that is on the road for 10 hours a day or more is obviously more likely to be involved in an incident than a Polo that does a 20-minute commute twice every 24 hours.”

Oxford found that the Toyota Quantum accounted for more fatal crashes per 1,000 vehicles than any other model. But this still tells us nothing about the ability of Quantum drivers. Controlling for other factors, like the amount of time spent driving each day, would almost certainly produce a different result.

Counting only fatal accidents introduces new issues. Scott told Africa Check that “being involved in a fatal accident does not imply that the driver was at fault, or driving badly”. 

Even if they had the same likelihood of being involved in an accident, a car model with fewer safety features would be more likely to be involved in a fatal accident than a car with more or better safety features.

The RTMC did not control for any of these factors, and the study did not conclude that drivers of a particular model of car were “worse” than any other drivers.

The worst data crime of all? Confusion is the RTMC’s fault! 

The Mail & Guardian rightly pointed to the importance of studies like this in helping us better understand road accidents in South Africa. The RTMC can be commended for this.

But the corporation is also responsible for the misleading press release which misrepresented its own study’s findings and called Polo drivers “the worst”. 

Some blame also lies with the media’s reporting on the press release. It’s a clear example of why journalists should do more than read the press release about a study. Luckily we have a guide to help, on how to craft accurate science journalism out of press releases.

Edited by Nica Richards.

This report was written by Africa Check, a non-partisan fact-checking organisation. View the original piece on their website.