Summary evaluations of multi-academy trusts

The scrutiny of multi-academy trust (MAT) performance by Ofsted has recently undergone some operational changes. The regime previously known as “focused inspections” has been replaced by “MAT summary evaluations”. In this note, we comment on the revised approach and identify a number of key elements for MATs to reflect on.

Background

The practice of MATs themselves being subject to scrutiny by the inspectorate is not a new one. In 2015, “a concerted programme of action to establish the effectiveness of multi-academy trusts in supporting and challenging academy schools within individual chains”, known as focused inspections, was launched. Under that regime, Ofsted mainly concentrated on the largest MATs at the weakest and strongest ends of the spectrum in terms of performance, but it was never entirely clear how those inspections should run or exactly how MATs would be selected for them because no guidance was ever issued.

Ofsted’s position now is that, having trialled the new approach in 2018, the system of summary inspections will improve and clarify the experience for all.

Interestingly, Ofsted also released an operational note for inspectors to support this updated process.

As with the predecessor regime, the Secretary of State wrote to HM Chief Inspector to give the nod to the new process, including emphasising the point that MATs are not being inspected in the strict sense of the word – it remains the case that there is no legal basis for Ofsted inspecting and grading the MAT entity independently under the inspection framework.

Summary evaluation – overview of process

As mentioned above, Ofsted has taken the opportunity to publish an “operational note for inspectors” to support the new system. Rather than summarise and repeat that note’s contents here, we share some key observations below.

There was no such guidance under the previous era of focused inspections and it is fair to say that the new document is welcome and does suggest greater transparency on process. It looks, perhaps not accidentally, like a mini framework and there is a sense that Ofsted is attempting to ease the continuing tension between its role in the formal inspection of academies and reviewing the direct part that MATs play in the quality of education.

At the core of the summary evaluation regime is a two-stage process. Under stage one, Ofsted will carry out “batched inspections”, inspecting under section five or section eight, as applicable, some of the individual academies within the MAT. Importantly, the new approach offers more flexibility around scheduling so that, although in some cases the inspections could take place within the same week, it is also possible for them to be stretched out over as long as two terms. Stage two occurs after the publication of those inspection reports – again, this is seen as a key improvement as it will enable the MAT to reflect properly on the findings from the batched inspections. The second stage will include meeting with MAT personnel, site visits to academies and telephone surveys with any academies not inspected under stage one. Ofsted draws on those findings to ultimately produce the summary evaluation letter, which will follow a prescribed format and highlight areas of strength and improvement. The outcome is solely narrative as opposed to any graded judgement but is published on Ofsted’s website.

Some observations

Away from the detail of the process which can be found in the 17 pages of the operational note, some interesting points are:

• Although Ofsted clearly retains its legal powers to inspect individual academies as part of stage one, Ofsted has no power to insist that MATs engage with summary evaluations under stage two. Similarly, the ability for inspectors to carry out site visits under stage two is at the sole discretion of the MAT (this was reinforced in the Secretary of State’s letter). MATs are also able to submit a deferral request if they have genuine concerns about timing. Refusing to co-operate with a summary evaluation without any sound basis would not be advisable, but it is a reminder of the lack of legal status that the process does require MAT engagement.
• A MAT will not know if they will be asked to engage in the summary evaluation process until the end of stage one. Ofsted will take an initial decision to carry out the batched inspections but will then use its discretion and likely refer to the criteria listed in paragraph seven of the operational note to decide if it will proceed with stage two.
• If a MAT is unhappy with any aspect of the summary evaluation process, it is able to complain in accordance with Ofsted’s usual complaints process. The fact that there is now a published process for these reviews perhaps makes the complaint procedure more tangible in this context, as the operational note suggests certain standards and expectations. In addition, the MAT will have five days to review and comment on a draft of the summary evaluation letter before it is published, with the operational note suggesting that such comments do not have to be limited to points of factual accuracy.
• Ofsted plans to select a “broad range” of MATs and will not solely focus on large underperforming groups. One of the common questions regarding focused inspections was how Ofsted made its selections, but now the factors Ofsted will typically take into account are listed at paragraph 13 of the operational note. The list doesn’t give too much away; for example, there is no prescribed minimum size of MAT which is subject to summary evaluation. Indeed, so far we have seen summary evaluations of MATs with anything from 10 to 30-plus academies. However, it seems clear that cause for concern (whether due to volume of complaints, significant decline in results, warning notices or safeguarding concerns) is likely to bump up the odds of a summary evaluation and local intelligence will still be relevant. In reality, the number of MATs that are subject to a summary evaluation will be small - we expect about 15 per year - but the low number also adds gravitas to the process and brings into focus the value of self-scrutiny at MAT level. Interestingly, Ofsted’s letter on the first summary evaluation published made clear that the trust was selected because of the high number of its schools that were performing well. A focus on all sizes and performance levels should be welcomed and, if delivered on, should provide trusts with more opportunities to gain insights from the new regime.

How can you prepare?

Paragraph 39 of the operational note sets out a list of areas upon which inspectors are likely to concentrate and is certainly worth referring to. Paragraph 35 also highlights the type of evidence that will be relevant. Neither list is exhaustive. There is no expectation that evidence is prepared in a certain format for the purpose of the process, but this is very clearly the MAT’s opportunity to present evidence beyond that which has been obtained by inspectors under stage one in order to demonstrate areas of focus, the impact the MAT is having across its academies and the overall quality of education.

Looking back at Ofsted’s findings under the focused inspections regime, as well as the summary evaluations which have taken place to date, we know, for example, that Ofsted sees strength where MATs are able to identify their “weaknesses”, the solutions they have in place to tackle areas of improvement, and how these solutions are being implemented and subsequently monitored. When it comes to strategy, performance, structure and impact, there are various trigger questions which MATs may want to ask themselves in preparation. For example:

• What impact are you having on your schools – both in terms of promoting strong performance and tackling underperformance?
• Can you explain your governance structure and demonstrate that it is effective?
• How are you avoiding common governance pitfalls, such as poor communication and unclear relationships between the board, the executive and local governing bodies?
• How do you identify and deploy leadership expertise within your trust?
• What is your school improvement strategy?
• Do you understand the strengths and weaknesses of all your schools?
• Does your local governing body (or equivalent) value the trust and the support it provides?
• Are you a learning organisation?
• How do the MAT’s values translate meaningfully into practices at all levels?

This is merely a sample of considerations which you would encourage MATs to reflect on, but which demonstrate the point that, ultimately, Ofsted is continuing to review whether your MAT is an effective organisation.

Learning the lessons

Looking at the summary evaluations which have taken place to date, MATs have been credited, amongst other areas, for:

• embedding their vision so that it guides decision-making at all levels of the MAT consistently
• having effective central functions (for example, HR, finance and estate management) that closely support schools to allow school leaders to retain focus on school improvement
• being a trust that “knows itself well”, enabling expertise and resources to be accurately focused
• being a MAT that schools want to join or are proud to belong to, for example because the trust demonstrates a genuine commitment to ensuring that schools can retain their individual characteristics
• consistency of approach on curriculum, polices, systems and particularly safeguarding across the schools, fostering collaboration rather than competition
• a measured approach to growth
• the CEO, chair of trustees and executive leaders having “their fingers on the pulse of every school”
• high-quality and well-targeted professional development opportunities for staff.

Areas of challenge which have been focused on have been:

• pupil attendance across the MAT, including where this is specific to a particular phase (for example, secondary) or group (for example, disadvantaged pupils)
• the need to develop a wider curriculum and/or identify the sequence of knowledge acquisition
• ensuring that practice on providing support to disadvantaged pupils impacted right across the MAT
• further embedding school improvement strategies to increase the rate of improvement in some schools
• high levels of exclusion.

We continue to monitor the outcome letters of summary evaluations to ensure that any trends feed into our advice, consultancy services and training for MATs. We also welcome hearing about your own experiences of the process.