Maverick Life

South Africa, Maverick Life

GroundUp Analysis: Why have annual national assessments?

This year’s annual national assessments, which are administered in literacy and numeracy to all learners in Grades 1 to 6 and 9, have been postponed till December following opposition from teachers' unions. How should we understand the value of these assessments, the reasons for the opposition from unions and how the assessments can be improved for the future? By STEPHEN TAYLOR for GROUNDUP.

One thing typically lacking in discussions about the annual national assessments (ANAs) is a clear understanding of their purpose. There are three broad goals one might want a system of national assessments to achieve:

First the ANAs should be seen as a tool for teachers to use in their own assessment processes in order to diagnose learning problems and design remedial strategies (Goal 1).

Second, the ANAs could be used to measure the progress of the country (or of smaller sub-groups such as provinces) towards improving literacy and numeracy outcomes (Goal 2).

Third, the ANAs could be used to promote accountability throughout the system for achieving learning outcomes (Goal 3). Here I use a broad definition of accountability, which includes empowering parents and communities with information about pupils’ and schools’ performance.

All three of these goals are envisaged in the Department of Basic Education’s sector plan, albeit not in the exact same words.

To some extent the ANAs have made limited contributions to all three of these goals over the last few years. On goal 1, research just prior to the introduction of the ANAs had revealed extremely weak assessment practices within the majority of our schools, thus pointing to the need to provide teachers with good assessment tools to use in the process of teaching and learning. Therefore, even if the test development process in the ANAs could be improved, the government has at least provided new assessments that are an improvement on what was previously available.

On goal 2, the ANAs have confirmed the low and unequal nature of learning outcomes within the primary school sector, highlighting the urgent need to address foundational literacy and numeracy.

On goal 3, there is now a greater emphasis on academic outcomes within primary schools, something that previously only existed at the matric level when it is too late to address fundamental learning gaps.

However, these goals are only bluntly or partially achieved by the ANAs in their current design. On goal 1, many teachers do not yet effectively incorporate the ANAs into their learning programme, and sometimes experience them as a distraction. On goal 2, we cannot yet compare the overall performance in the ANAs from one year to the next since a new set of test items is developed each year. On goal 3, one cannot exert any strong accountability if teachers themselves administer and mark the tests – any attempts to label particular teachers will therefore not improve accountability but will simply provide an incentive for teachers to inflate performance in the ANAs.

Importantly, each of these goals, if they are to be achieved, requires different design features within the ANAs. And there is a real risk that if we aim for the ANAs to serve all three goals we will end up achieving none of them. For example, if teachers administer and mark the ANAs themselves since they are expected to use them as a classroom assessment tool, but those teachers also face rewards and sanctions based on the test results, then we may end up in a situation where teachers don’t use the tests effectively and the test results will not be reliable for accountability purposes.

To provide teachers with good assessment tools, teachers across all grades should receive the ANAs every year. If the assessments are to function effectively for this purpose, they should be administered and marked by the teacher, and they should remain low-stakes so that teachers administer and mark them without any fear of negative consequences. For this purpose, the department could introduce training opportunities and tools for teachers dealing with how to use the ANAs to diagnose learning difficulties and inform remedial strategies. To some extent, these sorts of initiatives are already being introduced by the department, but over time they need to be refined and implemented on a larger scale. There may also be a need to investigate the curriculum content covered in the ANAs relative to the timing of when the tests are administered in order to optimise the alignment of the ANAs with lesson planning and curriculum sequencing within schools.

For goal 2, the best way to measure changes in learning outcomes within the system is to test a representative sample of students, as is presently pursued through the Verification ANA process. This process requires independent fieldworkers to administer and mark the tests. In order to compare performance from one year to the next, the tests administered to this sample of students need to contain some test items that are common across the years. These items must therefore be kept secure. The department has been exploring possibilities for achieving this but it is a somewhat technical process and has not yet been finalised. For this purpose, it is not necessary for ANAs to test every grade; one could focus only on certain priority grades.

Using national assessments to hold schools or teachers accountable for learning outcomes is a far more ambitious pursuit. This would require independently administered assessments for all schools, which would have major logistical implications and may not be compatible with the current design where assessments are administered simultaneously in all schools. Even then, it is a statistically complex task to isolate the impact of a teacher on learning outcomes over and above the influence of other factors such as the home backgrounds of students. This is of course not to mention the objections teachers’ unions would probably have to holding teachers accountable in this way. My view is that we should rather be content with ‘soft’ and ‘generic’ accountability whereby the importance of achieving learning outcomes in the early grades of schooling is recognised by all parties – teachers, pupils, parents and the government.

The ANAs are potentially a very important intervention in our school system. All the research indicates that to meaningfully improve educational opportunities in South Africa we have to get learning foundations in the early grades of school right. The assessments help raise awareness throughout society of the importance of achieving these early learning foundations.

So why is there opposition to the ANAs from teachers’ unions?

I am sympathetic to the desire to improve the design and implementation of the ANAs in order to increase their usefulness. However, I suspect the opposition to the ANAs is at least partly motivated by a general anti-testing view I often encounter. The anti-testing sentiment is strongly influenced by experiences and educators in the US. In South Africa, the context is very different. Prior to the ANAs, there was effectively no measure of school performance aside from the matric examinations, leaving primary schools in the dark. Arguments like “weighing the pig doesn’t make it any fatter” are of course ridiculous – if this was applied to medicine there would be no place for diagnosis. Besides, assessment is fundamental to learning and teaching. The argument that teachers are teaching to the test would perhaps be valid if there was an unhealthy over-emphasis on academic outcomes in South Africa relative to a more holistic education. But in a situation where the majority of primary school pupils are failing to acquire basic literacy and numeracy skills as defined by international benchmarks, encouraging teachers to prepare explicitly for a literacy or numeracy test is hardly a bad thing.

The argument that the money spent on the ANAs could perhaps rather be spent on teacher training opportunities is a red herring and a distraction. Given that the ANAs cost less than 0.2% of what the government spends on teachers’ salaries, the ANAs are one of the most cost-effective ways in which we spend on education. Improving opportunities for the professional development of teachers is of course important, and a lot of work is happening in this area, but this should not be seen as an alternative to the ANAs, which are a key systemic intervention.

But the ANAs clearly need to be improved in their design and implementation. This improvement will have to begin with a clear and shared understanding of the purposes of the ANAs. The ANAs of 2015 have unfortunately been negatively affected. Administering the ANAs in December is obviously not ideal: there will be less opportunity for teachers to use the tests to inform their teaching and the marking and data collection may have to occur during the school holidays.

On a positive note, the recent concerns raised by teachers’ unions could prompt the government to fast-track its efforts to improve the various design features of ANAs. In the end, we must use this situation to redesign and improve the ANAs in the interests of one key stakeholder – the children. DM

Taylor is a researcher in the South African Department of Basic Education. His work includes impact evaluation of education interventions, measuring educational performance and equity in educational outcomes. In 2010 he completed a PhD in economics at the University of Stellenbosch, analysing educational outcomes of poor South African children. This article is written in his personal capacity.

Photo: Teacher Reginald Sikhwari poses for a picture with his class of grade 11 students at Sekano-Ntoane school in Soweto, South Africa, September 17, 2015. REUTERS/Siphiwe Sibeko

Gallery

Please peer review 3 community comments before your comment can be posted