Fixing Scotland’s Misleading Education Statistics

The plethora of statistics on Scottish school education that has appeared in the last week is a reminder of how inadequate the current provision is. The First Minister, John Swinney, and the leader of the Scottish Conservatives, Russell Findlay, fell into the statistical morass yesterday, as First Minister’s Questions deteriorated into a slagging match of dodgy numbers. Reform Scotland has repeatedly pointed out the problems, most recently in its paper on what needs to change. Yet nothing does change. Here’s a sample of what is wrong and the Scottish Government’s panglossic media claims to the contrary:

The Scottish government says: ‘95.7 per cent of 2023-24 school leavers were in a positive destination’

This is good news only if we accept the undifferentiated definition of ‘positive’, as ‘higher education, further education, training, employment, voluntary work, [or] Personal Skills Development’. Employment can be anything from a stable, full-time job with good prospects to a zero-hours arrangement with no actual contract and no offer of training or any other kind of career advice. ‘Personal Skills Development’ is too amorphous to be meaningful – any activity to ‘improve employability’. The data are collected vicariously (via Skills Development Scotland) rather than directly from the leavers themselves. Setting these vague categories aside, we find that only 71% of leavers were in the only three destinations here that have any kind of quality assurance – higher and further education, and training. The percentage in these destinations was as low as 62% of people who left school after fourth year (the number of which is itself rising). The chasm between these percentages and the 95.7% implicitly cited by Mr Swinney yesterday left open the space for Mr Findlay to propose that the students who occupy that dark hole would be better not in school at all.

The Scottish government says: ‘57.4 per cent of school leavers left with 1 pass or more at SCQF Level 6 or better’

Level 6 means Highers but also anything else that is classified as being at that level, such as national units (which are essentially bits of Highers courses), national progression awards, Scottish Vocational Qualifications at (confusingly) Level 3, and much more. Many of these non-Highers qualifications are valuable, but they are not Highers. Specifically, the highest-status university courses will not accept them as part of entry qualifications. Why doesn’t the government sub-divide this information into these more meaningful components? If they did, then Mr Findlay would have to recognise that there are courses in school – such as the Foundation Apprenticeships – which do serve the non-academic students quite well. But the Scottish government can hardly complain about his assault on their policies when they don’t provide comprehensive statistical information about their operation.

The Scottish government says: ‘The gap between the proportion of school leavers from the most and least deprived [fifths of] areas attaining 1 pass or more at SCQF Level 6 or better was 38.4 percentage points’

This gap is enormous, but expressed like that is almost wholly uninformative in an educational useful way. The importance of the gap depends on the baseline. For students who did not get beyond level 6, the gap is 44 against 31, so that the advantaged proportion is about 1.4 times the disadvantaged proportion. For those who reached level 7, the advantaged percentage is 34 against 9 for the disadvantaged, which is a smaller gap in an absolute sense but better expressed as nearly four times greater. These re-calculations can of course be done by the reader of the statistical tables, although it would be more helpful if some inkling of them were given in the headlines. More serious is the absence of longitudinal data that would explain why the gap widens so much between courses mainly taken in school fifth year and those taken a year after. More generally, the absence of longitudinal data of this kind makes all these statistics quite useless in a pedagogical sense.

Part of the problem is also the inadequacy of the measure ‘deprived area’. It is not a valid measure for individual students. Around one half of families with low income do not live in the most deprived fifth of neighbourhoods as measured by the SIMD. So these statistics that seem to show us something about the impact of social inequality on attainment give us no evidence about what works in helping young people in different circumstances.

In the same vein, infecting all the statistics here is the complete invisibility of pupils at independent schools: in Scotland, the official picture simply pretends that they don’t exist. It is as if having parents who are wealthy enough to afford a place at an independent school deprives a student of membership of Scottish society.

Ethnicity

Deprivation is not the only unsatisfactorily measured social characteristic. Another is ethnicity. On this, the Scottish government choses to say nothing at all in the form of a headline, which is itself rather surprising when groups generally called ‘minority ethnic’ constitute about 10% of all pupils in local authority schools. Scottish statisticians still stick with the rather dated ethnic categories that have been used for decades. These are becoming far too crude for a society that is increasingly diverse. For example, the proportion gaining at least one pass at level 6 was 84% in the category which the Scottish datasets call ‘African/Black/Caribbean’. In the ‘white – Scottish’ category, it was 65%. The conclusion might seem straightforward: black students are doing better than white students. But the inadequacy of this may be seen from the better data on ethnicity in England. There, it is clear now that ‘African’ and ‘Caribbean’ measure very different educational experiences. In GCSE, for example, consider the proportion gaining an award at Grade 4 or better (analogous to a National 5 pass at levels A-C). Among students labelled ‘African’, that proportion was 69%, but for students labelled ‘Caribbean’ it was 52%. Students classified as ‘white British’ had 64%, and so any inferences that might be drawn about ethnic disparities are complex (as has often been pointed out). There is no reason to expect the patterns to be much different in Scotland. The Scottish data environment still does not seem to have learnt that crude ethnic labels can conceal as much as they reveal.

These are merely a few topical examples to illustrate the annual disappointment of Scottish educational data. Official statistics ought to give us insights into how our society is changing, and how the public services are contributing to that. In Scottish education, such illumination is still far away.

Lindsay Paterson is Professor Emeritus of Education Policy at the University of Edinburgh