The website of the statutory quality assurance body for higher education, the Council on Higher Education (CHE), records that: “On the 2nd of February 2017, the Council on Higher Education launched the National Reviews [sic] of Doctoral Studies, it held its first ‘Stakeholder Consultative Forum’ at the National Research Foundation (NRF) in Pretoria.”

This announcement was followed by two years of planning for the review of all doctoral qualifications across all universities. As the review was originally scheduled for completion in April 2020, universities can expect lots of review activity this year.

The story that follows tracks developments to the present. It’s a troubling story, as we shall see. At the outset though, to the credit of the CHE, the doctoral review process in fact got off to a good start. In line with the principle of peer review, the “Qualification Standard for Doctoral Degrees, November 2018” (the qualification standard), was compiled with the input of a reference group of 19 eminent professors from a representative range of universities. An earlier draft had also gone out for public comment. We thus have credible benchmarks for the review, established through a legitimate process.

In providing benchmarks for assessing quality, the qualification standard follows a clear logic comprising:

• Purpose of the doctoral qualification;
• Graduate attributes that manifest that purpose; and
• The contexts and conditions for the assessment of those attributes.

The qualification standard was followed by the CHE’s “Institutional Self-Evaluation Report Template, April” (the SER template). This is supported by a further CHE document, the “National Review Manual: Doctoral Qualifications”. Review procedures are straightforward enough. Using the SER template, each institution develops a self-evaluation report of its doctoral qualifications; a peer review panel conducts a site visit to interrogate the institutions’ claims of compliance with the benchmarks; institutions then receive a report from the CHE. If the report identifies shortcomings, the institution is required to submit an improvement plan to the council.

Perhaps it’s the familiarity of this entirely legitimate background that explains why there’s not been much public comment on a review that can have major implications for the reputation and status of each university. With few exceptions, comment has been generally supportive, with perhaps mild reservations about issues of implementation.

Anyway, of course, everybody agrees on the importance of the doctorate. It’s the most prestigious and most generously state-subsidised qualification at the apex of the higher education pyramid, so why not review it? And as Sioux McKenna further points out, poor quality doctoral education can also have damaging implications beyond the doctorate itself: It can set the scene for the nature of knowledge creation and dissemination in the entire university.

But it’s time to take McKenna’s argument to its next logical step. If the CHE’s review is itself of poor quality, it can have damaging implications for the entire higher education sector. It’s time to ask the age-old question: Quis custodiet ipsos custodes? – who will guard the guards themselves?

Let’s begin with the characteristics and purpose of the doctoral degree.

The CHE Qualification Standard confirms that in terms of national regulation, the doctoral degree is a level 10 qualification carrying 360 credit points on the National Qualifications Framework (NQF). In the case of a doctoral degree awarded entirely by research – and this is the type offered at most universities – all 360 credit points are allocated to the thesis. The thesis must present a significant and original contribution at the frontiers of knowledge. It has always been understood that the degree is awarded after a student’s thesis has been approved through a rigorous process of examination. There is no more to it than that.

To this time-honoured intrinsic PhD purpose as an academic research endeavour in line with the NQF’s level descriptor, the qualification standard adds a new layer of extrinsic purpose in the form of graduate attributes that are deeply personal and professional. Graduates should have the skills, predisposition and understanding of professional codes to enable them to serve as custodians and stewards in their field of expertise; be able to adapt to changing contexts; act as agents of intellectual advancement at levels from the local to international, and so on. These attributes have relevance to the role graduates are expected to play in the world of work.

There is good sense in socialising doctoral students into communities of practice. Many universities have developed innovative supervisory practices that do exactly this. It is also true that international standards such as the 2015 UK Quality Code for Higher Education include the professional development of students as an important criterion for doctoral quality.

However, compared with its international counterparts, the CHE qualification standard’s version of students’ professional development has a notable distinctiveness that can be traced to the reason why the National Research Foundation proposed a national review in the first place. According to the CHE Manual (p4): “In fulfilling its mandate of Human Capital Development (HCD), the NRF has several funding instruments that support the doctoral degree. However, in making these investments, the NRF needs to be assured that Doctoral qualifications offered by South African public and private higher education institutions meet national quality standards for Doctoral degrees.”

In other words, the NRF needed to be sure that the PhD was an instrument fit for its own human capital purpose. Consistent with this, the qualification standard has a “Preamble and Rationale” highlighting the need for “a steady supply of high-level new knowledge for innovation and sustained growth”. Imperatives of the Department of Science and Technology’s “Ten Year Innovation Plan 2008 – 2018” and the 2012 National Development Plan are cited.

Never mind that research has seriously questioned human capital theory so beloved by Unesco and the OECD. By making explicit the extrinsic purposes of doctoral study with reference to national policies at this moment of our history, the qualification standard infuses our understanding of doctoral quality with a dose of instrumentalism.

But instrumentalism has a place of honour in commissioned research and, to a lesser extent, in qualifications other than the doctorate. The doctorate has its own integrity. It is an opportunity for individual students to explore the world, and truth is different to usefulness.

Genuinely high-level original research has also moved beyond the Newtonian world that saw the universe as rationalistic, deterministic and of clockwork order. Complexity theories have emerged since the publication of Thomas Kuhn’s book, The Structure of Scientific Revolutions almost 60 years ago. Kuhn discredited the idea of science developing through steady, incremental research steps. Real progress comes about not in “normal” science, but in “revolutionary” phases of science. When scientists encounter anomalies in current theories (often fortuitously), communities of specialists in the field are plunged into crises of uncertainty, turmoil and angst. It is these revolutionary phases – such as the transition from Newtonian mechanics to quantum physics – that represent conceptual breakthroughs (Naughton, 2012). 

High-level research is more likely to have an impact when it explores what Donald Rumsfeld famously called the “known unknowns”: the things we know we don’t know. If research does that, we have a chance of learning about the “unknown knowns”: the things we don’t know that we know, like the long-term trends and anomalies quietly bubbling away behind the scenes. Such a non-instrumental approach, based on a view of the world as organic and non-linear rather than clockwork, calls for researcher curiosity in exploring uncertainties.

It’s a pity the qualification standard says nothing about philosophies of research. Because it’s so firmly grounded in instrumentalism, it seems to better serve the cause of the NRF and the CHE rather than the real long-term quality of the doctorate. This qualification standard can have no future anyway: it’s too context- and time-bound in the politics and policies of present-day South Africa.

But back to the present. The layer of extrinsic purpose in the quality benchmarks creates an immediate problem for universities now completing their self-evaluations. If national regulation requires that all 360 credit points for the doctorate are assigned to the thesis, how then should a university provide evidence of compliance with a layer of personal and professional graduate attributes beyond the thesis? Where does students’ socialisation into communities of practice fit in? At present, it cannot be a legalistic requirement for the award of the degree.

By its very nature, doctoral education is an individual experience. In practice, professional development opportunities for students vary greatly across disciplines, departments and faculties. Some students may make the most of opportunities universities provide for socialisation into epistemic and professional communities of practice, but others – particularly those whose personal circumstances do not permit such luxuries – will simply bury themselves in their research.

Perhaps the framers of the qualification standard intended a “soft” interpretation of the extrinsic qualification purpose. In any event, it’s the job of the manual and the SER template to interpret benchmarks and to guide institutional responses.

But if the compilers of these documents were aware of the conundrum presented by the twofold purpose of the PhD, they do not address it at all. They do not even reduce the descriptive narrative in the CHE qualification standard purpose statement to the kinds of analytic categories that make templates useful instruments. For example, they could have at least categorised the twofold purpose of the PhD in terms of “intrinsic” and “extrinsic” qualities; and those categorisations could then have provided a basis for guidelines in respect of relative weighting and so on. Instead, the SER template simply passes the buck by referring universities to the CHE qualification standard.

Things get worse when we shift our gaze to the simple mechanics of review implementation. In translating the logic of the qualification standard into an SER template, the CHE presents universities with confusion in respect of two of its three strands of logic: Graduate Attributes and Qualification Purpose.

The qualification standard explains Graduate Attributes thus:

“The purpose and level of the qualification will have been achieved when the following attributes are evident. The attributes are assessed within the context of the Purpose of the qualification.” Attributes listed below this explanation cover nine areas of knowledge and skills, each beginning with an achievement-based statement such as, “The graduate demonstrates …”, or, “The graduate shows evidence of …”.

On the other hand, the SER template says:

“Describe and evaluate how your institution prepares candidates to attain the graduate attributes (knowledge and skills) set out in the qualification standard for Doctoral Degrees.”

At the simplest level of what words mean, achievement-based evidence in respect of what graduates can do, or demonstrate (as in the qualification standard), is different to the means through which a qualification is designed to develop the sought-after attributes (as in the SER template). Should institutions take their cue from the qualification standard, or from the SER template?

Against that background of confusion, the SER template concludes rather ominously with tables on drop-out statistics, time-to-degree statistics and graduation rates. Let’s hope that peer review judgements are not clouded by such indicators of efficiency. Quality research can take time and it would be unfortunate if universities were, in future, to feel the need to safeguard their interests by encouraging doctoral students to tackle what is easily “doable” rather than what is intriguing.

In conclusion, at this stage we don’t yet know what sense universities will make of all this; and whether the CHE will then have sufficient common understanding across institutions to enable it to draw valid conclusions across the sector. A lot depends on the peer reviewers who will do the site visit rounds later this year. Let’s hope that scholarship and good professional judgement can mitigate the CHE’s shortcomings.

But from what we have seen so far, there are menacing signs of a phenomenon called “meliorism” (if you’re English); or the more descriptive “verschlimmbesserung” (if you’re German). Either way, it means that the doctoral review is on course for an outcome that is likely to subvert its own noble intentions of assuring – and developing – quality.

How did we get to this unpromising state of affairs? All CHE documentation is approved by its own statutory bodies. If that is not alarming enough within the context of our question about who guards the guardians, consider too that universities have been informed that: “The necessary review instruments have been developed and approved with the support and close involvement of the National Research Foundation.” DM

Ken Harley is Emeritus Professor, University of KwaZulu-Natal. He is a former Head of the School of Education and Interim Dean at the University of Natal/KwaZulu-Natal. He was editor of the Journal of Education for 12 years; manager of the CHE’s National Teacher Education Review; Team Leader of the EU-funded HIV Pilot Project in Teacher Education; and as a consultant, he has conducted 11 evaluations of funded Open Education Resource (OER) projects, mainly in East Africa and Ghana. He is co-editor of a COL/UNESCO book on OER.