Scotland’s independent think tank
Scotland’s independent think tank

Challenge Paper: Reliable Data


Scotland needs a major rethink in how it measures educational performance, a group of experts announces today

In a new paper, the independent Commission on School Reform (CSR) argues that a lack of reliable data makes it difficult to tell whether standards in Scottish school education are rising or falling. The Scottish Government has scrapped domestic surveys of pupil performance and withdrawn Scotland from international surveys except PISA, which was published last week and is now the only survey of information about how standards in Scottish education compares with other countries. 

The consequence, according to the Commission, is that “we know less now about the performance of Scotland’s schools than at any time since the 1950s”.

Keir Bloomer, chair of the CSR, says: 

“The Scottish Government has proclaimed education its highest priority. Other political parties share the view that it is vital to the future of the individual and to that of society and the economy. Yet there is legitimate concern about whether the system’s performance is satisfactory.

“A debate rages about how our schools, teachers and pupils are performing, with many statistics exchanged but few incontrovertible conclusions being reached. The continuing disagreements say something important and profoundly unsatisfactory about the information that is available. There ought to be a solid foundation of factual information that will allow constructive discussion to take place. Unfortunately, no such foundation exists.  PISA is the only example of international comparison available. Yet, even when the message was as damning as last week’s results, there is an official unwillingness to face facts.  

“Has there been a narrowing of subject choice in S4? Has it been accompanied by a fall in standards? Is performance in Higher rising or falling? Is the proportion of young people leaving school without any qualifications reaching an alarming level?

“We know less now about the performance of Scotland’s schools than at any time since the 1950s. Unfortunately, there seems to be little thought given to which measures are best able to provide valid and relevant answers.”

Professor Lindsay Paterson, co-author and member of the CSR added:

“The advent of devolution two decades ago raised hopes that policy-making in Scottish education would become more evidence-based, and that the evidence would be more reliable and relevant. 

“In practice, the evidence base for Scottish education has deteriorated drastically. Scottish education policy is now based on speculation, ideological whim, and partisan rivalry. No worthwhile policy making is possible in such a context.”

The CSR calls for the following steps to be taken to restore confidence in Scottish educational data and aid improvement to the system:

  • A Scottish Government commitment to maximise the amount of objective data that is available in relation to the performance of Scotland’s school education system
  • The introduction of a new sample survey of performance in key curricular areas during the phase of broad general education
  • Scotland should rejoin international surveys such as the Third International Mathematics and Science Survey and the Progress in International Reading Literacy Study
  • The Government should develop in consultation a set of measures of performance in the senior phase and at the point of leaving school that will properly reflect the success or otherwise of the system in improving the life chances of all young people.


CSR Challenge Paper – reliable data