Suddenly everyone is talking about exams. After two years of upheaval as a result of COVID-19 the old certainties are starting to crumble and some surprising people are beginning to think the previously unthinkable and say the previously unsayable. It’s as if they’ve suddenly had to admit that the emperor might in fact be stark naked and that the supposed gold standard of high stakes exams might always have been just the glitter of fools’ gold. The OECD report is sceptical and the Scottish Government’s International Council of Education Advisors puts it bluntly
“High school examinations are essentially an out-of-date 19th and 20th century technology operating in a 21st century environment of teaching and learning.”
Mind you, not everyone seems to get it. The Times last week reported Carrie Lindsay, President of the Association of Directors of Education, bemoaning the fact that “it had been challenging to find a consistent and accurate measurement of happiness, confidence and contentment among pupils”.
Given how challenging it has proven to be over the last few years even to find a consistent and accurate measurement of mathematics attainment at Higher, I really don’t envy the team that will presumably soon be tasked by Scottish Government to rectify this manifest failing in our system. Defining happiness, confidence and contentment would be a start. Personally, I would include a vigorous walk, a long warm shower, an opera by Mozart, blue cheese, half a bottle of good Lebanese red and the love my wife and children. I would argue that they have all been part of my broad general education, but I accept that some might not readily lend themselves to assessment in the senior phase.
What is needed is not more assessment but more humility about the value of assessment. It would help if we actually understood how reliable (or, in fact not) our assessment has been all this time. There is shockingly little data on examination reliability in Scotland but what we see from England doesn’t exactly inspire happiness, confidence or contentment.
On 2nd September 2020, Ofqual’s then Acting Chief Regulator, Dame Glenys Stacey acknowledged to the House of Commons Education Select Committee that exam grades “are reliable to one grade either way”. As my colleague Dennis Sherwood who has researched these data extensively puts it, “That sounds quite reassuring until you realise that this statement is the same as saying ‘an A level certificate showing ABB actually means any set of grades from A*AA to BCC, but no one knows which’”.
What is worse is that when I recently quoted that to a representative of leading UK universities, they had not the first clue that there might have been a problem. The myth that exams provide useful and reliable information about young people is so central to the way that we conceive of education in the UK that even some of those who advocate change persist in seeing the emperor in his finery.
So, before we get too far into the discussion about new approaches to assessment, let alone the desirability or otherwise of assessing happiness, confidence and contentment, let’s be honest about the limitations of any assessment system.
The Ofqual data suggests that across all subjects in England about 1 grade in every 4 is wrong. I would be delighted if SQA could produce data showing the situation in Scotland to be much better.
We wouldn’t accept such poor reliability in totting up our bill in a supermarket, let alone in medicine, or thankfully in the testing of aero engines. But if it’s just young people’s futures at stake, it’s more or less fine.
The answer to the problem of “out-of-date 19th and 20th century technology operating in a 21st century environment of teaching and learning” is not to assess more things. It is to work towards a system where we can be happy, confident and content that what we need to assess can be assessed reliably and in ways that support young people’s learning. But where most of what is truly valuable about education is not assessed at all.
Melvyn Roffe is Principal of George Walton’s College, Edinburgh and writes about education and related topics in a personal capacity