Since the annual national assessments (ANAs) were introduced in 2011, President Jacob Zuma has called them “a powerful tool of assessing the health of our education system”. That system is struggling and the more understanding of what’s wrong and what needs to be fixed, the better. That’s why improving ANAs are a key aspect of the National Development Plan’s medium-term strategic framework targets for education.

“I love the ANAs very much. The ANAs are quite helpful in terms of assessing the progress we’re making in improving the quality of education,” the African National Congress’s Gwede Mantashe said on Monday.

So when unions such as the South African Democratic Teachers’ Union (Sadtu) refuse to administer the tests, they’re selfishly sabotaging efforts to improve education, right? It’s a bit more complicated. A line has been drawn between unions and the Department of Basic Education on the issue, but the effectiveness of the ANAs remains in dispute.

Here’s something you won’t hear the Democratic Alliance (DA) say often: “Sadtu is right.” The opposition, fond of saying unions are holding development to ransom, agreed last year that the tests promote rote learning, are not being used as a diagnostic tool to address students’ struggles, and lead to teachers and schools being labelled ‘under-performing’, and that the education system doesn’t have the required resources to implement changes. (A more recent DA statement was titled ‘Sadtu must stop disgraceful boycott’).

Sadtu represents teachers and doesn’t always put education improvements first. A cynical view of its refusal to administer ANAs would suggest its members don’t want to do the extra work and don’t want to be held accountable for substandard teaching. Sadtu’s grievance about unconstructive finger pointing might be justified, but its eternal fight against imposed authority is not.

The union’s criticism, however, includes valid points and it’s not just Sadtu complaining but other education unions – the National Professional Teachers’ Organisation of South Africa (Naptosa), the South African Teachers’ Union, the National Teachers’ Union, the Professional Educators’ Union. In a joint statement on Monday, they said they were participating in a task team to rework the tests, but did not commit to a date on which to hold them, calling the Department of Basic Education’s announcement that the ANAs would be held in December “regrettable and confrontational”.

“The understanding was that once the remodelling process was complete within the set time frame, the parties would then further engage on the time lines to launch a remodelled systemic and diagnostic tool. Under no circumstances would we have agreed to the roll-out of the ANA in its current form even if it would have been at a later date as unilaterally decided by the Department of Basic Education,” said the unions.

They are not against the ANAs ever being held, but will not proceed without reforms. The unions plan to mobilise members against any attempt to hold tests in December. They won’t administer them, which might be nice if you’re a student wanting to ditch a quiz, but catastrophic for the government’s tough stance.

“As the Department of Basic Education, we advise all teachers to ignore the messages being circulated and instead prepare to administer ANA in December in the best interests of learners and the sector as a whole,” the department responded on Monday. “A volatile atmosphere created by unnecessary rhetoric needs to be avoided at all costs. We have agreed to engage with unions on a monthly basis for the remainder of the year to constructively deal with the concerns they have raised. Beyond that we have agreed to meet with unions on a quarterly basis to avoid any further communication lapses.”

Basic Education Minister Angie Motshekga showed a willingness to respond to the unions’ concerns when she announced the ANAs would be postponed, but now the department is firm that they will go ahead this year after provincial education MECs said they are ready for the tests. It accused the unions of reneging on an agreement to delay the ANAs and encouraged them to work within the task team established to review the system. To bolster support, the department said on Tuesday that it had spoken to school governing body groups, parent associations and student organisations which want the ANAs to go ahead.

The assessments don’t count towards whether a student will pass or fail, but are like matric results in that the state holds them up as an example of improvement. Last year, 7.3-million students wrote the national assessments testing numeracy and literacy in Grades 1-6 and 9, but the system, like much standardised testing across the world, has been regularly criticised.

Sadtu has said the ANAs shouldn’t be used to negatively label teachers and it wants the tests to take place every three years so that feedback can be implemented and monitored. Naptosa has expressed concern that students will learn by rote for the tests and said there is little time to implement changes after the results.

“In testing 7-million children, the department has bitten off more than it can chew and, in the process, undermined its own technical credibility,” Stellenbosch University researcher Nic Spaull, currently on sabbatical at Stanford, said last year. He noted implausibly high increases in results, a preference for higher, unverified results to be used in public, and a lack of psychometric testing which would help in comparing results over the years. “If we could trade ambition for competence, we may have a test that was actually telling us something clear instead of the muddled mess that is the national assessment for 2013. This testing must and should go on, but for heaven’s sake do it properly.”

A response from the Department of Basic Education’s Mweli Matanzima agreed that there are challenges as the tests were only recently introduced, but he said measures have been implemented to improve accuracy and that already ANAs are helping monitor the quality of teaching and learning.

Learners’ organisation Equal Education has also been critical. In a statement this month it supported the idea of administering the tests every two or three years. It said the ANAs can lead to teachers focusing on the test rather than the curriculum, that they can promote superficial rather than substantial improvements, and that it’s still too hard to gauge year-on-year improvements under the current system.

Equal Education general secretary Tshepo Motsepe claimed on Monday that the ANAs are part of the basic education minister’s performance indicators, which is why she is pushing them despite the resistance. “For the minister it seems like a tick in the right box,” said Motsepe.

Professor Ruksana Osman, dean of humanities at the University of the Witwatersrand, said: “ANA is meant to measure or provide a snapshot of grade-related curricular content. (It) provides progressively less useful diagnostic information in SA as increasing proportions of children seem unable to work at the grade level of the specified curriculum. Our research shows that early grade ANAs assess for the wrong things.” She added that the current confrontation is worrying “and will certainly disrupt schooling if not kept in check”.

Crain Soudien, CEO of the Human Sciences Research Council and deputy vice-chancellor of the University of Cape Town, has a long history of studying education issues. Asked if the ANAs are effective, he said: “Yes and no. The ‘no’ in some ways is more important because there are some methodological issues that are deeply worrying.” Soudien said there is no standard list of data to standardise the tests to measure them consistently year on year. That’s a “deep problem” because it’s difficult to understand the reliability and value of the results without a standard assessment practice.

“Where the system is right now is that the system needs a degree of formalisation and a robustness about what it thinks is valuable and desirable,” said Soudien. He also believes it’s not helpful to administer the tests every year and says treating the tests like rituals might lessen the value of what they should be trying to achieve. “I think there is an over-reliance on testing where we should be focusing on fixing up (the system).”

Soudien was also critical of the unions’ motivations and called for a frank discussion between education stakeholders, including those who could help fix the challenges in the ANAs, on how to proceed.

Professor Mellony Graven, South African numeracy chairperson at Rhodes University, has worked extensively in mathematics teacher education. “There are enormous challenges and the system needs to find a way for the ANAs to be helpful for these teachers rather than sort of get in the way,” she said. Currently the tests might not be the best use of money, which could be better used on intervention, she added.

One problem, she explained, is that from Grade 3 onwards teachers can’t mediate the language of a question and students might understand it better if they could hear it in their mother tongue. Another is the diagnostic impact of the tests. Graven explained that one of the key issues in improving mathematics results is filling the gaps in learning from previous grades, but ANA results don’t help teachers understand what level a student is working at because they only test students at their grade level.

So when the average mark for Grade 9s in mathematics is 11%, it’s clear the majority aren’t up to the desired standard, but it’s unclear how teachers should intervene for the different students.

“It tells us the same thing every year, over and over,” said Graven. If the tests helped identify a student’s level of proficiency then teachers could address the gaps using multi-level workbooks. Currently, however, for those Grade 9 mathematics students, we only know many of them scored zero on many questions, which doesn’t help teachers trying to plug the gap. “I don’t think the tool as it is is working for the purpose of, ‘Lets try and do something about these terrible results’,” said Graven. DM

Photo: School children play outside the Khanyakhwezi Primary School in Inanda near Durban, South Africa, 22 September 2010. EPA/JON HRUSA

Read more:

• Annual National Assessments report 2014