The Scottish government is currently holding a review of what it calls ‘the Future of Qualifications and Assessment’, in other words school exams. The review is led by Professor Louise Hayward of Glasgow University. It issued a consultation paper at the end of last week. Comments on the questions which it raises have to be submitted by 16 December, and Professor Hayward will make her final recommendations in March next year. The purpose of this blog is not to debate the substantive issues. That is done in detail in papers from Reform Scotland’s Commission on School Reform (in June 2021 and March 2021). My focus is on the process.
On the face of it, this seems an innocuously open approach to a matter of great importance. But it isn’t. Despite the undoubted good intentions of everyone involved, the consultation is, in truth, a conversation of the governing elites with each other. Only those who are invited set the criteria by which the outcomes are chosen. Only insiders have a real chance to influence the options offered in public for change. Only they get to set the agenda of debate.
Before I explain how the process is being constructed, let me consider an example of one of the questions that the consultation raises to show how an ostensibly open approach actually hides the real ways in which decisions are taken. Question 4 asks for views on ‘what a “better balanced” assessment system would look like’. It means by this mainly the relative importance of examinations on the one hand against coursework on the other.
Whatever your views on this matter, imagine being invited into a room to join a discussion of it and how differently you would prepare under two contrasting scenarios:
- where those already in the room quite like the current balance of exams and coursework – which at Higher, for example, means that most subjects assign between a fifth and a third of marks to coursework and the rest to exams of the familiar kind;
- where the presumption in the room is against exams altogether as unfair and as unable to test candidates’ knowledge and skills.
The problem with the consultation on this question is that it is impossible to know the review’s starting point – the beliefs of the people in the room. It might be reasonable to assume that it is scenario (1), because that is, after all, the currently existing state of affairs, in which coursework has gradually expanded its share over the past four decades. On the other hand, the consultation paper cites as authoritative a review of Scottish assessment by an academic from Oxford University that is quite resolutely hostile to any kind of exam, and also embarrassingly ill-informed about several important details of how Scottish exams have evolved over the past century and a half. If you entered the room assuming scenario (1), and assuming a fair degree of support for exams, you would be rather taken aback to find yourself having to defend their very existence.
That is the essence of the problem here. This review is shrouded in secrecy while pretending not to be. It has a rather byzantine structure. At the head is Professor Hayward, who is served by a secretariat that is provided by the Scottish government. She is advised by what is called the Independent Review Group, whose 22 members are drawn from the usual range of civic organisations that constitute such review groups in education – students, teachers, parents, employers, academic researchers, and the Scottish Qualifications Authority. Each individual doubtless has a great deal to offer, and the organisations they represent ought indeed to be among those which should be consulted. But how the specific organisations or the individuals were chosen is not explained. More importantly, what they are talking about in their monthly meetings is not shown publicly. Although we are promised minutes, these do not yet seem to be available despite the meetings’ having been going on since late spring. In any case, I expect these will be the kind of sketchy notes with which government minute-takers adeptly conceal anything controversial.
At least, though, these 22 are named, and there might be minutes. Beyond these people are 13 ‘Collaborative Community Groups’, which are described as ‘recognis[ing] and embrac[ing] the diversity of Scotland’s learners and communities’. No details are given of who is on these groups, of how they operate, or of the papers and controversies that they might be addressing. So the chances of anyone outside this invited circle being able to distinguish between, for example, scenarios (1) and (2) above are not good.
If each of the 13 Groups also contains about 20 members, this policy-making process involves perhaps around 300 people, fewer than 10% of whom are named, and almost all of whose procedures are invisible. That number 300 is probably larger than in most consultations by the Scottish government, but it is perhaps 0.02% of those currently involved in Scottish education. In other words, the agenda of debate here is being set by about 0.008% of the Scottish population age 16 or older.
I should declare an interest here. Back in August, I was invited to join one of the 13 Groups. I asked for an assurance that everything that was tabled and discussed at the Group would be fully public. My argument was that, as a member of the Group, I would want to be able to enter into public debate of the kind that would provoke me into thinking carefully about the questions that were being raised, and especially into questioning my own presuppositions. After all, Professor Hayward had the 22 members of the Independent Review Group to stimulate her thoughts. Each of these 22 people, likewise, could turn to one of the 13 Collaborative Community Groups for similar opportunities to sound out ideas. Therefore, I reasoned, as a member of one of these Groups, I would want to be able to air ideas outside. If every member of these Groups did that, then a full range of options could be debated in public – including some very controversial opinions – rather than only those options which the review officially wanted to have debated.
But no such assurance of public openness was forthcoming, and so my invitation did not translate into membership.
One result of this ostensibly open but actually opaque process is the one-sided and rather flimsy consultation paper which was issued last week. Nowhere in that paper, or on the review’s website, is there any indication of anything like the full range of views about assessment that are debated worldwide. Nor is there any accurate history of Scottish exams, or of the ways in which, in the past, they helped to establish the reputation of Scottish education as broad and rigorous. There is no acknowledgement of the role which exams played in ensuring the credibility of Scottish comprehensive education. There is apparently complete ignorance of the way in which exams have provided better opportunities to social groups that have suffered from invidious discrimation – girls, Catholics, minority ethnic groups, low-status social classes. The consultation paper thus presents a mainly negative view of the existing arrangements. And that partiality is the result of the secrecy of the review’s processes.
Secrecy of this kind was supposed to have been set aside by the pluralism of the Scottish parliament. Yet that is not what has happened since its advent in 1999. Consultations have had a veneer of openness, but a reality of invitation-only introspection. Professor Walter Humes has called this the ‘iron cage of educational bureaucracy’, the capturing of the policy process by second-rate managers of educational quangos. An investigation by the Times newspaper (15 September 2022) of the current reforms to Scottish education confirmed this, finding that change was being insidiously guided by the people in charge of the institutions that are being reformed.
When this consultation on assessment is completed, government ministers will claim that the way ahead has been widely consulted on. Yet all the really difficult discussions will have happened in private, if they have happened at all, with only selected snippets given out for public debate. Many difficult questions will have been ignored because only a limited range of points of view will have been at the table where decisions about recommendations are made. This consultation paper itself is such an example, with its list of ex cathedra questions presented without any explanation of how they arose, and without any prior opportunity for anyone to debate publicly what should be asked about.
These criticisms are not intended to impugn the integrity or expertise or public-spiritedness of any of the people involved in either the Independent Review Group or the Collaborative Community Groups. The problem is not with them, but with a process that lacks the robustness that can come only from unfettered public debate.
Lindsay Paterson is Professor emeritus of education policy in the School of Social and Political Science at Edinburgh University.