South Africa may be at an inflection point. After years of load shedding, collapsing infrastructure and falling investor confidence, recent commentary by Adrian Enthoven and Johan Fourie highlights that the country could finally be turning the corner.
Electricity availability has improved, key logistics reforms are advancing and business confidence, long suppressed, is beginning to thaw. With Operation Vulindlela pushing structural reforms and renewed cooperation between government and business, a sense of cautious optimism is emerging, and there appears to be a change in momentum (a “vibe shift” as Fourie puts it).
But moments like these are precarious. The question now is whether South Africa can convert early signs of progress into sustained improvement, or whether we will regress into stagnation.
Because, even with momentum, the underlying challenges are immense. Economic growth is still sluggish and unemployment is catastrophically high at 31.9%. At the same time, reading-for-meaning among young learners remains shockingly low, and the country is one of the most, if not the most, unequal in the world. These challenges are deep and longstanding.
At a time when fiscal constraints are tight and more than 20% of government revenue goes to debt service costs, we cannot afford to spend resources on programmes that feel right but do not deliver. If South Africa has even a narrow window of opportunity, we must use it wisely. That starts with knowing what works and what doesn’t.
This means rigorously testing and evaluating programmes as they are implemented, and being willing to change course when necessary. It is often difficult to predict whether a policy will achieve its goals. In fact, many policies, despite good intentions, are ineffective. For example, international evidence suggests that many job training programmes have no measurable effect on employment outcomes.
In South Africa, a study on the government’s learnership programme, which combines classroom learning and on-the-job experience, found that initial gains faded quickly, with no long-term advantage for participants.
This is why rigorous evaluations of programmes and policies are so important. We need to build evidence about what works and what doesn’t and use that information to guide decision-making and accountability.
One proven way to do this is to embed impact evaluations into programmes before scaling them, using randomised controlled trials (RCTs). For example, in the first phase of a job training or education initiative, participants can be randomly assigned to either receive or not receive the intervention. This approach is especially apt when programmes are rolled out in phases or when there are more applicants than available spaces.
With randomisation, the two groups are, on average, identical except for programme participation, so any observed differences in outcomes can be attributed to the intervention itself. This allows policymakers to accurately measure the causal impacts of the programme before making larger investments. Did it help people get better jobs? Did learning outcomes improve? If not, the programme may need to be redesigned before being rolled out on a larger scale.
This method has been used effectively in other countries. For instance, in Botswana, the NGO Youth Impact partnered with the Ministry of Basic Education to test the Teaching at the Right Level approach. A small-scale randomised pilot in 2016 showed that grouping children by learning level, rather than grade, and providing targeted instruction led to improved learning outcomes. Based on this evidence, the programme was scaled up and now operates in more than 20% of public primary schools.
A/B testing
However, running RCT evaluations like these takes time and resources — both of which are in short supply in South Africa. The urgency of our challenges related to unemployment, education and inequality means that we may need to try new policy responses immediately on a large scale, and a complementary evaluation approach may be more suitable: A/B testing.
A/B testing, widely used by tech companies such as Google and Amazon, is a rapid, low-cost way to compare different versions of a product or programme and continuously optimise performance. In essence, two versions of a programme — version A and version B — are tested in parallel to see which works better.
For example, a tech company may randomly expose customers to two different versions of a website interface, and whichever generates more clicks or revenue becomes the new standard. This culture of rapid, ongoing experimentation has produced striking results. One study found that firms adopting A/B testing saw performance improvements of 30-100% after a year.
The same logic can be applied to social programmes to assess cost-effectiveness and scalability — by comparing a standard model (option A) to an optimised version (option B). As in an RCT, A/B testing also relies on random assignment, but instead of comparing a programme to a pure control group that receives nothing, it compares two different versions of the programme itself.
For example, one group of learners might receive a remedial reading programme in its current form, while another group receives the same programme with a simple enhancement — such as parents receiving weekly SMS messages with short reading tips or reminders to read with their child for 10 minutes a day. Success is then measured by comparing learning outcomes between groups A and B.
Importantly, such A/B tests can often be run using existing monitoring and evaluation data already collected by implementing partners. For example, if schools already collect learning data every 9–12 weeks, they can use the data to run A/B tests over that period and iterate accordingly. This process can be repeated every term, allowing for rapid learning and ongoing improvement.
Unlike traditional evaluations like RCTs that usually aim to answer the question, “Does this programme work?”, A/B testing focuses on a different but equally important question: “Which version of this programme works best and at the lowest cost?”
This type of rigorous learning enables implementers to adopt the best-performing, most cost-effective, and most scalable programme model. It’s an approach that is well suited to the urgency of South Africa’s social and economic challenges — and to a political landscape where resources are scarce and results are needed fast. Not every test will yield large effects, and some changes will show no improvement. But over time, these small, frequent tests can lead to significant gains.
Ultimately, we need to become a country that learns, quickly and deliberately, from what works and what doesn’t. If South Africa is indeed turning the corner, the worst thing we could do now is assume that momentum alone will carry us. Transformations stall when decision-makers fly blind; they accelerate when evidence tells us which paths to scale, which to abandon, and which to redesign. DM
Richard Freund is a research assistant at Young Lives, an international study of childhood poverty coordinated out of the Department of International Development at the University of Oxford.