Planning v competition in education : outcomes and efficiency

Governments have a responsibility to secure the best return on their investments in services. This being the case, the institutions that are involved in organising the supply of these services are a topic of interest and concern. In the case of educational services in Wales, particular concern surrounds the performance of school pupils as measured by international tests, in which the recent record has been disappointing. This paper examines this record and evaluates reform measures that have been put in place.


Governments
have a responsibility to secure the best return on their investments in services.This being the case, the institutions that are involved in organising the supply of these services are a topic of interest and concern.In the case of educational services in Wales, particular concern surrounds the performance of school pupils as measured by international tests, in which the recent record has been disappointing.This paper examines this record and evaluates reform measures that have been put in place.
The remainder of the paper is structured as follows.In the next section, I look at some data that allows comparison of educational outcomes in Wales and England.This is followed by a broader discussion about the system of education.The final section draws together the main ideas of the paper in a conclusion.

Data analysis
The data used here come from the Programme for International Student Assessment (PISA) undertaken by the Organisation for Economic Cooperation and Development (OECD).Every three years, samples of 15 year old pupils from schools in OECD countries and elsewhere undertake tests in mathematics, reading and science.More than 80 countries have participated in PISA; in the case of the United Kingdom, the data are collected by the National Foundation for Educational Research (NFER) and are enhanced by the presence of separately identifiable samples from each of England, Wales, Scotland and Northern Ireland.The latest available data, used here, are for 2015.The microdata include information about pupil and school characteristics as well as about pupil performance on the tests, and are made freely available to researchers on the OECD website 1 .
The PISA report for 2015 2 provides data on average scores achieved on tests in mathematics, reading and science that are in each case higher in England (493, 500 and 512 respectively) than in Wales (478,477 and 485 respectively).These mean values are based on samples of pupils, of course, but it should be noted that the England and Wales 95% confidence intervals do not overlap in the case of any of the three subject areas.This follows performance in earlier rounds of PISA testing that likewise indicated lower scores in Wales than elsewhere in the United Kingdom.This being the case, the media has shown interest 3 and the results have been the subject of debate both in the Welsh Assembly and the House of Commons 4 .Various reforms have been introduced in Wales over recent years, and while it is hoped that these will have positive impact it is likely that their effects on pupil performance are only now beginning to be realised (Reynolds, 2016).
Average scores based on outputs (test results) can provide a misleading picture of the educational process, however.The literature on educational production functions (Hanushek andWoessmann, 2011a, 2011b;Glewwe et al., 2013) provides a wealth of information about how various characteristics of pupils, schools and systems influence performance.The differences that we observe between educational output as measured by PISA performance across England and Wales are likely due, in part at least, to differences in inputsincluding, for example, socio-economic composition.
To interrogate this further, I investigate the PISA data for England and Wales using methods drawn from the literature on data envelopment analysis (DEA).This method was developed by Charnes et al. (1978) following earlier work by Farrell (1957), and uses linear programming techniques to identify, separately for each decisionmaking unit in the data, a frontier against which can be evaluated the efficiency with which the unit maps inputs onto outputs.Specifically in the present context, I use the DEA approach of meta-frontiers pioneered by Charnes et al. (1981)  Two important conclusions may be drawn from this analysis.
First, once allowance is made for the different characteristics of pupils, spatial variation of performance in standardised tests may not be as great as at first appears.This echoes the findings of Rees and Taylor (2014), who find that the gap in measured performance between Welsh and English schools is much reduced when they undertake a comparison of schools in Wales with those in the English local education authorities whose characteristics most closely match those of Wales.Secondly, the shape of the production frontier likely differs across space so that it is not necessarily the case that all pupils are best served by the technology  2011) that school autonomy is beneficial in developed systems of education, but less so elsewherethere appear to be horses for courses.This is particularly significant in the context of Wales, given the political commitment, arguably more pronounced there than in England, to equality and social justice (Power, 2016).

Discussion
The third Thatcher government introduced a wide range of policies aimed at transforming the provision of public services by way of the introduction of quasimarkets (Le Grand, 1991).The new arrangements separated out the funding of services by the state from their provision.Providerswhether in the public or private sectorwould compete with each other to provide services, and would be remunerated on the basis of the amount of custom they attracted.In education, for example, the 1988 Education Reform Act replaced block grant funding by formula funding where monies followed pupils and, crucially, where pupils (and their parents) were given new freedoms to choose the school that they attended.At the same time, greater managerial autonomy was granted to schools, which for the first time could opt out of local authority control.In some areas, such as the provision of nursery education, similar principles were applied but in a yet more vivid fashionthrough the introduction of vouchers (Sparkes and West, 1998).
Much early evidence on the impact of quasi-market reforms on school performance in England is consistent with the view that competition enhances performance (Levačić, 1994;Bradley et al., 2000;Bradley et al., 2001).More recent evidence provided by Burgess et al. ( 2013) is particularly interesting because it uses the abolition of school rankings in Wales as a natural experiment with which to test the hypothesis that publication of such rankings improves school effectiveness.The evidence supports the hypothesis, albeit in a way that varies substantially across the distribution of schools.This is an important point, not least because of the emphasis placed in Wales on equity as a desired outcome from the education system; hence not only average effects of policy but the impact of policy throughout the distribution of experience should be considered.Indeed, in the Burgess et al. study, there is no evidence of an effect on performance of the introduction of quasi-market mechanisms on the top quartile of schools.Goldstein and Leckie (2016) confirm that the reform was followed by a relative decline in average achievement in schools in Wales, though they argue that this may be as much due to pupils' lack of practice in high-stakes tests as to competitive effects.Likewise West and Pennell (1997) document improvements due to the greater freedom to choose schools, but (in contrast to Burgess et al.) argue that the greatest benefits were realised by those drawn from the higher end of the income distribution.
Yet the fundamental assumption underpinning competition in the provision of school places deserves some interrogation.Schools are not shareholder-owned firms in which management faces an imperative to maximise profit, and the freedom of entry and exit essential to a truly competitive climate, is absent.Moreover, until the advent, in England but not in Wales, of academies in the early years of this century, adherence to a national curriculum further limited competition.
Schools in reality face a plethora of incentives and respond to a multiplicity of social needs.They offer their customer base an experience goodone where the quality of the match between a school and pupil can only be assessed over time and through the commitment of engagement.Schooling takes time, school quality is not static, and choice made on the basis of necessarily retrospective data may not be optimal ex post.Furthermore, schooling is a club goodone that is provided simultaneously, on a non-rival basis, to many beneficiaries; individuals cannot make decisions about their schooling without this impacting on others, and in such a context markets cannot operate perfectly 7 .
Following the establishment of the Welsh Government in 1999, the approach to policy in the sphere of education has certainly differed markedly between Wales and England.Power (2016) notes the tendency for government to be viewed in England as part of the problem, while in Wales it is more usually seen as part of the solution.Relatively few schools in Wales have become grant maintained, so that local authorities retain a strong voice here in the provision of education.Likewise, the curriculum, or at least the way in which it is delivered, has developed in ways that are more progressive in Wales than in England.Critically, there is less standardised testing in Wales, where league tables have not been produced since 2001.This means that the principle of school choice does not work in the same way in Wales as in Englandparents do not have the same information sets.It is in any event not clear that school choice can work the same way across the two countries; in mid-2016, England had a population density of 424 per km2, while the corresponding figure for Wales was just 150 8 .
As noted by Swaffield (2017), the relatively poor ranking achieved by Wales has resulted in a 'PISA shock', with politicians responding with the plea that we should 'never waste a crisis'.Hopkins (2016) has identified a number of issues at systemic level that have prevented the Welsh system from achieving to its potential.In particular, he notes that 'accountability systems are still relatively crude and not linked to increased performance, there is a lack of a secure pedagogy that reliably leads to enhanced student learning, the standards of entrants to the teaching profession are lower than in equivalent systems and the architecture for sustained self-improvement is missing'.He also notes a mismatch between the needs of the system and the solutions proposed by politicians, the latter tending to be more bureaucratic than effectivea criticism also noted by Grigg (2016).
Recent reforms in Wales have aimed to tackle these shortcomings, starting out from the 20 point plan outlined by Andrews (2011).This focuses on four areas (Pont et al., 2017).The first concerns the quality of teaching provision.This follows criticism by Estyn (2016) of the standard of teaching in many secondary schools.This is being addressed through Teach First, and through the various recommendations of Furlong literacy; mathematics; health and wellbeing; and science.How the change programme advocated in this review translates into practice will be crucial in determining the success of the reform, and evaluation will also be crucial in this area.
Thirdly, the emphasis on equity in education is particularly strong in Wales (Welsh Government, 2016), and this distinguishes the system in this part of the UK from that in, say, England.The DEA results reported earlier confirm this, in showing that the system in Wales serves those in the bottom ESCS decile better than does that of England.Nevertheless, questions remain about the extent to which the oversight that the Welsh government has on resource allocation allows good decision making.Local authorities, working alongside regional consortia set up following the National Model for Regional Working (Welsh Government, 2014), devise funding formulae for the allocation of resource to schools.As has been the case in England, there is an open question about how equitable this allocation might be, and about the extent to which local decision making results in a 'postcode lottery'.This relates to the tension between the desirability of equity and the principle underpinning subsidiaritythat experts at local level are best placed to assess the local situation.An interesting literature on subsidiarity appeals to property rights and shows that local authorities likely underinvest in circumstances where there are spillovers in the returns to the investment (Lülfesmann, 2002).
Specifically in the context of education, the optimal locus of decision making has been the subject of research by Johnes (1995) and De Witte and Schiltz (2016), the latter of which in particular indicates the presence of substantial economies of scalethat is, that decisions are better made at higher levels.
Fourthly, as has been noted by Bloom et al. (2015) and McCormack et al. (2014), leadership is a key determinant of the success of educational institutions.School leadership remains an area that is of concern to the OECD (Pont et al., 2017), although the National Academy of Educational Leadership 9 launched in 2018 offers promisebut again will need to be evaluated.
A common thread running through these areas of activity is the recognition that reform is needed in order to improve the experience of young people as they progress through education in Wales.While PISA results have been a driver, they should not be (and have not been) the only driver.While recognising the mobility of labour within the UK and beyond, the development of the education system in Wales needs to be cognisant of the needs of Wales itself, and specifically of (existing and prospective) employers in Wales.The distance between employers and education has been a challenge, not only within Wales but more widely in the UK, and opportunities to shrink this distance need to be grasped.

Table 1 Average value of pupil-level efficiencies
But the fact that the score for England is less than unity indicates that there are some pupils whose characteristics are such that, for them, the frontier in Wales dominates that in England.It is not, therefore, unambiguously the case that the English system is performing better (or worse) than the Welsh.Put simply, it depends who you are.