Use of evidence-based practices in schools: National snapshot

Introduction

This report focuses on 4 evidence-based practices with a well-documented evidence base that they can positively impact student outcomes (Archer and Hughes 2011; Black and Wiliam 1998; Heitink et al. 2016; Kulik et al. 1990; Chaffee et al. 2017).

These practices are:

• explicit instruction
• mastery learning
• formative assessment 
• focused classrooms/classroom management. 

Findings in this snapshot come from a range of data sources:

• AERO’s evidence use survey
• A rapid review of use of EBPs in Australia
• Analysis of data collected in Australian schools through 4 international education surveys – the Progress in International Reading Literacy Study (PIRLS), the Programme for International Student Assessment (PISA), the Teaching and Learning International Study (TALIS) and the Trends in International Mathematics and Science Study (TIMSS).

This snapshot provides critical information about the use of evidence-based practices

Given the impact that EBPs can have on student outcomes, it’s important to know how and if these practices are used in schools, and to what extent.

This snapshot:

• provides baseline information on the use of EBPs against which changes over time can be described, monitored, tracked and compared
• identifies what hinders and what supports use of EBPs
• identifies gaps in data collection for accurate measurement of use of EBPs in schools in Australia
• provides information that helps improve student outcomes through effective use of EBPs.

Key findings

• Over three quarters of teachers report regularly using EBPs in their classroom, with some EBPs such as explicit instruction or formative assessment more frequently reported than other practices.
• Classroom management, through the consistent enforcement of rules and routines at classroom and school level is the least implemented among the EBPs investigated. ‘Focused classrooms’ are reported by around half of Australian students and teachers.
• Teachers do not always use EBPs in their entirety, with early indications that some teachers may not be making use of the full range of strategies these practices encompass.
• Teachers also report using teaching practices that are not supported by evidence. These include designing lessons based on students’ learning styles (71%) and using unguided instruction or independent inquiry time to allow students to discover answers for themselves (36%).
• Teachers report using EBPs more when there is collaboration at a school level, support from school leaders, teacher capacities and attitudes, designated time to regularly include EBPs in the classroom, and professional learning opportunities, particularly those involving coaching.
• Time is the most frequently identified barrier to using EBP. Teachers report limited opportunities for collaboration with peers or to implement specific techniques in the classroom.
• Teachers seem to be using EBPs more frequently in classrooms with younger students, but there isn’t any evidence to suggest EBPs are less effective in secondary settings.
• Existing surveys of students and teachers rarely ask about the use of EBPs, and so there is not much data on their use in Australian classrooms. Generating and bringing together more data from teachers and students will provide complementary insights into how teaching is implemented and experienced. Together, these insights can also help teachers and school leaders better plan to improve teaching and learning.
• Teachers report using EBPs more than students report experiencing them, indicating a range of reporting biases, for both students and teachers. We need better measures of how much and how well teachers are actually using the suite of teaching techniques comprising EBPs and measures of EBP suitable for student reporting.

Explicit instruction

Explicit instruction involves fully explaining and effectively demonstrating what students need to learn. In the classroom, this can include strategies like:

• actively supervising and interacting with students as they practise their skills
• specifically outlining learning objectives and success criteria
• using worked examples to demonstrate the steps needed to complete a task.

Explicit instruction strategies are a common theme in high performing schools and effective classrooms (CESE 2013; Louden 2015).

In AERO’s evidence use survey, teachers and leaders report that explicit instructional strategies occur frequently in Australian classrooms. In “most” or “every lesson”:

• 91% of teachers interact with students as they work, providing immediate elaboration and explanations as needed
• 78% of teachers clearly outline what students will learn and how they know they have learned it
• 78% present and explain worked examples when introducing students to new or unfamiliar problems.

These findings align with results from the analysis of Australian data from international education surveys. We looked at several activities that contribute to explicit instruction across the 4 surveys:

• Review of previous learning
• Presentation of new learning
• Correction and teacher support.

Review of previous learning

Many teachers and students (67% to 85%) indicate that activities associated with the review of previous learning, such as explaining how new and old topics are related, or linking new content to prior knowledge, is something that happens frequently in their classroom (Figure 1). The only activity that was not common was presenting a summary of the previous lesson at the beginning of each class (43%). While this activity may ensure previous learning is established before new knowledge, it is not something that would be expected to happen in every lesson.

Figure 1: Explicit instruction – review of previous learning

Figure 1 notes

Presentation of new learning

Activities such as setting goals at the beginning of instruction or explaining what the teacher expects students to learn are often implemented, according to 72% to 95% of teachers and students (Figure 2).

Figure 2: Explicit instruction – presentation of new learning

Figure 2 notes

Correction and teacher support

Teachers are expected to provide students with in-class support to understand and master difficult concepts (CESE 2020). Many activities associated with correction and feedback, such as the teacher providing extra help when students need it, re-explaining a topic when students don’t understand, or asking students to explain their answers, occur “frequently”, “always”, or in “most or every lesson” more than 60% of the time. Generally, students report that teachers give extra help when students need it, or they explain topics again when students don’t understand. This is more so the case with younger students (64% to 70%) compared with older students (only 44% to 52%). Similarly, a higher proportion of teachers of younger (77%) compared to older students (61%) report they ask students to explain their answers every or almost every lesson. Only 31% of Year 10 students report that teachers provides individual help when a student has difficulties understanding a topic or task (Figure 3).

Figure 3: Explicit instruction – correction and teacher support

Figure 3 notes

Formative assessment

Formative assessment refers to the variety of methods teachers use to gather and interpret information about student learning as learning is taking place. Putting this into practice involves, among other things:

• identifying what students already know and what they need to learn (setting goals)
• providing timely feedback using developmental rubrics and/or worked samples so students understand what is expected
• using simple, low-key assessments to regularly check for student understanding and identify any misconceptions.

Some studies indicate that Australian teachers use formative assessment practices more frequently than the Organisation for Economic Co-operation and Development (OECD) average and point to the increasing emphasis on formative assessment in policy (CESE 2014; Hegazy and Barton 2017). Respondents to the AERO evidence use survey report that a range of formative assessment methods occur frequently in Australian classrooms. In “most” or “every lesson”:

• 73% of teachers assess students’ understanding of the content and make adjustments accordingly
• 67% of teachers design lessons based on data they have gathered regarding students’ prior knowledge and experience.

The analysis of Australian data from international education surveys provides information about elements of formative assessment:

• Setting goals
• Checking understanding
• Provision of feedback

Results from these analyses largely align with findings from the AERO evidence use survey.

Setting goals

Research into effective formative assessment practices consistently highlight the importance of setting learning goals, ideally in co-operation with students (Hegazy and Barton 2017; Fletcher 2018; Schildkamp 2020). Most teachers (85% to 95%) indicate they always or frequently explain what they expect students to learn, and they observe students when working on tasks. While to a lesser extent, students also generally agree the teacher sets goals for their learning and asks questions to check whether students understood what was taught (72% to 77%) (Figure 4).

Figure 4: Formative assessment – setting goals

Figure 4 notes

Checking understanding

An important element of formative assessment is that teachers use a mixture of observation and questioning to check how well students have understood a topic. Across the international education surveys analysed, around two thirds to three quarters (61% to77%) of teachers in Australian schools report that in most or every lesson they check students’ understanding in the classroom by asking them to explain their answers, asking questions to check if they understood what was taught. Most students (81%) answering the PISA Year 10 survey also indicate that in most or every lesson, teachers ask questions to check whether students understood what was taught (Figure 5). Surprisingly, only 61% of teachers answering TIMSS Year 8 survey indicate they ask students to explain their answers in every or almost every lesson.

Figure 5: Formative assessment – checking understanding

Figure 5 notes

Provision of feedback

The success of formative assessment relies on effective feedback (Hegazy and Barton 2017; Bellert 2015; Brooks et al. 2021a) which should ideally be delivered as targeted explicit instruction, based on individual student learning data (Hegazy and Barton 2017; Bellert 2015; Hoogland et al. 2016). The feedback should focus on correcting conceptual misunderstandings rather than reiterating processes or content, and should clarify success, check progress and plan improvement through actionable suggestions (Hoogland et al. 2016; Bellert 2015; Brooks et al. 2021a). The provision of feedback should be as soon as possible (Bellert 2015; CESE 2015). Delivering feedback face-to-face is ideal, so that misunderstandings can be quickly resolved; this also ensures students are likely to be more attentive to the suggestions and feel that the feedback is individualised (van der Kleij 2019). Unfortunately, some studies found that in practice, feedback from teachers is often not clear or actionable to students (van der Kleij 2019). However, almost three quarters (71%) of respondents to the PIRLS Year 4 student survey agree “a lot” that their teacher tells them how to do better when they make a mistake. Further, teachers commonly report they observe students when working on particular tasks and provide immediate feedback (82% to 92%). Over a quarter (29%) of teachers give individualised feedback to each student in every/almost every lesson (Figure 6). This lower figure is not surprising, given time constraints and what can be achieved in one lesson. It is difficult to manage individualised feedback to each student in each or almost every lesson and around 47% of Year 4 students answering PIRLS said their teachers do this in about half the lessons.

Figure 6: Formative assessment – provision of feedback

Figure 6 notes

Classroom management and focused classrooms

Focused classrooms maximise students’ on-task learning time by minimising disruptive behaviour and disengagement. Classroom management, or strategies for focused classrooms, include teaching and modelling appropriate behaviour so students know how to perform the roles expected of them and establishing a clear system of classroom rules and routines, so students have predictability and structure. The AERO evidence use survey suggests that most teachers and leaders adopt strategies for classroom management:

• 76% of teachers and leaders explicitly teach rules and routines for how to participate effectively in class
• 90% of teachers and leaders model appropriate behaviours, such as not raising their voice and following the rules.

Although research suggests that schoolwide strategies and support are important for fostering ‘focused classrooms’ (Aditomo and Köhler 2020; Goss et al. 2017), classroom-level practices are more important than factors at the school-level (Hepburn et al. 2021; Hepburn and Beamish 2019; Murphy 2014). Effective school environments flexibly allocate resources to meet classroom management needs, implement schoolwide attendance strategies, systematically record and utilise behavioural data, provide administrative support for teacher’s disciplinary decisions, and maintain a focus on improving academic outcomes (Hepburn and Beamish 2020; Murphy 2014).

The suite of international education surveys we analysed inquired about rules and routines at school level, as well as engagement with classroom rules, modelling appropriate behaviour and levels of disruption in the classroom. The following elements of classroom management to achieve a focused classroom are presented below:

• Rules and routines are established and enforced at school level.
• Rules and routines are established and enforced at classroom level.

Focused classrooms are measured through:

• Respectful students
• (Lack of) disruption.

Rules and routines are established and enforced at school level

Findings from international education surveys indicate that younger students and teachers of younger students mostly agree “a lot” that their school has clear rules about student conduct (79% and 69% respectively), and around two thirds agree “a lot” that the school’s rules are enforced in a fair and consistent manner (60% of teachers and 66% of students). A lower proportion of teachers of older students agree “a lot” that the school has clear rules about student conduct (55%) or that the school rules are enforced in a fair and consistent manner (38%) (Figure 7).

Figure 7: Classroom management – rules and routines are established and enforced at school level

Figure 7 notes

Rules and routines are established and enforced at classroom level

Only a small amount of information is available across international education surveys about the enforcement of rules and routines at classroom level. This data shows that about two thirds of teachers answering the TALIS survey report they “always” or “frequently” establish rules and routines such as regularly asking students to follow classroom rules, or to quiet down as the lessons begins (61% to 68%) (Figure 8).

Teachers have a critical role in classroom management. The teacher’s choice of management strategies is more important for students’ behavioural outcomes than teacher education or experience. (Hepburn 2020; Romi et al. 2013; Harbaugh and Cavanagh 2012; Aditomo and Köhler 2020). Specific practices can be proactive or reactive, and although some research recommends the use of proactive strategies (in particular Hepburn 2020; Hepburn and Beamish 2019), positive reactive (sometimes called responsive) strategies have been shown to be effective as well (Goss et al. 2017). The degree of classroom disruption reported in the AERO evidence use survey may indicate that teachers are relying on reactive practices, responding to student behaviours as they arise. Alternatively, these findings may reflect the widely reported concerns that teachers do not feel adequately prepared for classroom management by either preservice education or the professional learning opportunities which are available to them (Hepburn 2020; Stephenson et al. 2013; Hepburn and Beamish 2019; Murphy 2014).

Figure 8: Classroom management – rules and routines at class level

Respectful students

While there are numerous reports detailing school practices around suspension and expulsion, there is evidence of an overreliance on these exclusionary disciplinary techniques (Hepburn and Beamish 2019), and there are fewer studies detailing teacher practices in proactively working towards focused classrooms and respectful students.

Just over half of younger students report students are respectful of teachers and school property and behave in an orderly manner (54% to 58% agree “a lot” this is the case). This is also reflected in reports from their teachers (43% to 51% agree “a lot”). Only 30% to 43% of teachers of older students agree “a lot” that students respect school property, are respectful of the teachers and that students behave in an orderly manner (Figure 9).

Some proactive classroom management strategies which have been highlighted in prior research are establishing rules and expectations, organising and delivering content to minimise disengagement, the provision of lesson rationales, and consistent routines (Watt et al. 2017; Hepburn et al. 2021; Hepburn and Beamish 2020). Praise can also be effective and should be used to encourage positive or expected behaviours, rather than reserved for good work (Goss et al. 2017). Data on these practices were, however, not available in current datasets and should be included in future measurement projects.

Figure 9: Focused classrooms – respectful students

Figure 9 notes

Disruption

Disruptive students impact learning across the classroom, hindering peers who might otherwise be engaged students (Sullivan et al. 2014).

A quarter of teachers answering TALIS 2018 “agree” or “strongly agree” that there is disruptive noise in their classroom and that they need to wait a long time for students to quiet down. Almost a third (32%) indicate they lose quite a bit of time because of students interrupting the lessons (Figure 10). Over a third of students, and slightly more older students, report there is noise and disorder, and students don’t listen to what the teacher says (33% to 43%) (Figure 11).

Figure 10: Focused classrooms – teachers' reports of disruption

Figure 10 notes

Figure 11: Focused classroom – students’ report of disruption

Figure 11 notes

Research indicates that most behavioural concerns are minor infringements such as noncompliance or talking
out of turn, but that the extreme frequency of these issues places a substantial strain on educators (Sullivan et
al. 2014; Goss et al. 2017). Between a quarter and a third of students agree or strongly agree that students interrupt the teacher, they don’t listen to what the teacher says, and the teacher has to wait a long time for students
to quiet down (Figure 12).

Figure 12: Focused classroom – students’ report of interruption

Figure 12 notes

Mastery learning

Mastery learning is a way of designing units of work so that:

• each task (or set of tasks) focuses on a particular learning objective
• students must master a task to move onto the next one.

Teachers use formal or informal assessments to monitor students’ progress and provide additional support to students who have not yet mastered the learning objectives.

Using mastery goals is associated with improved classroom behaviour, deeper learning, and higher levels of intrinsic motivation for learning (Watt et al. 2017; King et al. 2017; Pudelko and Boon 2014).

A few items in the suite of education surveys we analysed inquired about aspects of mastery learning:

• Clear objectives
• Revision and enrichment

Clear objectives

Most teachers answering the TALIS 2018 survey indicate they always or frequently explain what they expect students to learn (95%) and set goals at the beginning of instruction (85%). Three quarters of students answering PISA 2010 survey indicated that teachers set objectives and goals for their learning (72% and 77% respectively) (Figure 13).

Figure 13 Mastery learning – clear objectives

Figure 13 notes

Revision and enrichment

Students’ reports of whether their teacher explains a topic or continues teaching until the students understand range between 44% and 75%. This might be due to respondents referring to teachers in general (PISA) or teachers of specific subjects (for example, Maths or Science in TIMSS). Younger students are more likely than older students (64% and 70% versus 44% and 52%) to agree “a lot” that their teachers explain a topic again if students don’t understand (Figure 14). While the amount of time spent on revision is important, opportunities can be difficult to find, given curriculum pressures. Fortunately, it appears that the quality of the revision and the success of students in their revision tasks is more impactful than the total time spent on revision (Evidence for Learning 2016).

Figure 14 Mastery learning – revision and enrichment

Figure 14 notes

Enablers of and barriers to evidence‑based practices

A range of factors that support use of EBP were identified from the studies reviewed. The absence of these factors was often identified as a barrier to EBP. The most consistently highlighted enabler of EBP is quality professional learning, and the most consistently highlighted barrier is a lack of time.

Professional learning enables the use of evidence-based practice

Targeted professional learning opportunities, particularly those involving coaching, are effective at increasing the usage and quality of a variety of EBPs (O’Neill and Stephenson 2013; Goss et al. 2017; Hammond and Moore 2018; Hepburn and Beamish 2020; Brooks et al. 2021b). Coaching is relatively common with just under two-thirds (64%) of respondents to the AERO evidence use survey indicating that coaching is available at their school to help them use evidence to change their practice. Unfortunately, there are frequent reports that professional learning opportunities are not sufficiently targeted, either to the right skills, or to the staff that would benefit most, and that coaching is often prohibitively costly (Murphy 2014; Louden 2015; Hepburn and Beamish 2019).

Professional learning activities can be internally or externally facilitated. In general, professional learning should be spaced out over longer periods to leave time for reflection and experimentation between sessions and should use content embedded in the curriculum to minimise the translation required for the classroom (CESE 2014; Hammond and Moore 2018; Brooks et al. 2021b). However, this may not often be possible, due to limited time available for such activities.

Time is a consistently identified barrier to using evidence-based practices

Time is a barrier across multiple levels in schools. At the school level, there is limited time for educator collaboration with peers to improve or discuss their practice (Hoogland et al. 2016), and in the classroom there is often limited time for teachers to implement the specific EBP (Bellert 2015; van der Kleij 2019; CESE 2020; Daffern and Fleet 2021). Especially in later years, curriculum pressures can exacerbate this issue, shifting the focus to student performance, rather than learning, with the coverage of curriculum content taking up a lot of time (Hoogland et al. 2016; Romero-Celis 2017; Brooks et al. 2021b). Finally, depending on the practice, the amount of time that educators have with a specific group of students can be another limiting factor, reducing contact and making the individualisation recommended by many practices challenging (Hepburn et al. 2021).

Collaboration among teaching staff supports the use of evidence‑based practices

At the school level, collaboration is a frequently identified factor supporting the use of EBPs, with increased collaborative planning associated with greater classroom-level uptake (CESE 2013; Hammond and Moore 2018; Schildkamp 2020). Teachers should have opportunities to collaborate with their peers, coming up with ways to collect, implement and interpret student learning data (Hoogland et al. 2016; Brooks et al. 2021b). The TALIS and PISA 2018 (teacher) surveys show that just over half (58%) of teachers engage in discussions about the learning development of specific students.

The AERO Evidence use survey found that teachers and leaders report good levels of collaboration:

• 86% of respondents agree or strongly agree that at their school colleagues discuss evidence that could improve their practice
• 80% of respondents agree or strongly agree that at their school leaders share and discuss evidence that could improve their practice
• 75% of respondents agree or strongly agree that at their school they have set aside regular times or meetings to discuss evidence that could improve their practice.

Effective use of student learning data enables the use of evidence‑based practices

Using student learning data (data collected by teachers in their daily practice) is another frequently highlighted enabler of EBPs. In addition to enabling specific practices such as formative assessment or classroom management to be tailored to students’ needs (Murphy 2014; Hoogland et al. 2016; Hegazy and Barton 2017; Brooks et al. 2021a), using learning data also allows teachers to see the impact of any changes to their practice, making them more likely to continue using an EBP that has positive effects (that might not be otherwise apparent, such as delayed effects; CESE 2013; McMullen and Madelaine 2014; Bellert 2015; Hammond and Moore 2018; Centre for Evidence and Implementation 2020). Using learning data as evidence of an effective classroom can help demonstrate the usefulness of EBPs, providing motivation for change at a school‑level (Louden 2015).

To use evidence-based practices well, teachers need the right knowledge, capabilities and attitudes

Teachers first need to be aware which practices are evidence-based (Hepburn and Beamish 2019; Hepburn et al. 2021). Preservice training may not always cover specific EBPs or may present them alongside non-EBPs with little differentiation around the practices that are best supported by evidence (Hepburn 2020; O’Neill and Stephenson 2013; Hepburn and Beamish 2019). In addition to using EBPs, respondents to the AERO evidence use survey also report using in their classrooms strategies that are not supported by rigorous causal evidence. For example, in ‘most’ or ‘every’ lesson:

• 36% of survey respondents allow unguided instruction or independent inquiry time for students to discover answers for themselves
• 71% design lessons that match the different learning styles of their students.

To put EBPs into effect, deeper knowledge of the specific practice is required, as well as content knowledge and additional skills such as assessment or data literacy (as required by the specific EBP; Hoogland et al. 2016; Schildkamp 2020; Brooks et al. 2021b). Teachers also need enough confidence and autonomy to implement EBPs in their classroom (Romi et al. 2013; Hoogland et al. 2016; Watt et al. 2017; Schildkamp 2020).

School leaders can have a big influence on the use of evidence‑based practices

School leadership should be stable and promote the schoolwide implementation of EBPs (Louden 2015; Hegazy and Barton 2017; Brooks et al. 2021), ideally also championing specific EBPs through non‑evaluative channels (Hammond and Moore 2018). Leaders should possess depth of knowledge of the specific EBPs they want to implement, but in practice, this is often lacking (Hoogland et al. 2016; Fletcher 2018). While classroom-level factors tend to have a greater effect, there is a wealth of evidence to suggest that schoolwide policy is still very important (CESE 2013; Murphy 2014; Goss et al. 2017; Watt et al. 2017; Hepburn and Beamish 2019; Aditomo and Köhler 2020). However, schoolwide policies are not enough for the consistent implementation of instructional practice. Ongoing investment in improvement is required, through channels such as professional development or coaching (CESE 2015; Hammond and Moore 2018).

Recommendations and next steps

The baseline has provided several next steps for organisations who engage with teachers and leaders, as well as AERO.

1. Develop measures of evidence‑based practices that are comprehensive, reliable and relevant

While analyses of Australian data from international education surveys provide an indication of how much EBPs are used, these surveys were not designed to measure specific practices, meaning that our understanding is incomplete. Further, data from student surveys about teaching practices can be open to interpretation as it is not always clear whether the students are able to understand the survey items.

Next steps for AERO

• Continue our work on rubrics to help teachers and leaders to understand their own use of EBPs. The tools will also allow AERO to comprehensively measure the use of these practices.
• Investigate ways that student surveys can be better used to describe use of EBPs.

2. Focus on quality use of evidence-based practices

It is important to assess the depth or quality of use of EBPs. Some teachers and leaders report they use a practice, but they may be only using aspects of that practice. In addition, surveys may over-state the use of EBPs, as survey respondents tend to answer questions in a manner that will be viewed favourably by others. This may create a distorted image of EBPs being frequently used if there is no investigation of the quality of how they’re used.

Next steps for AERO

• Expand information on what specific EBPs look like, including guidance on how to use them in the classroom
• Further examine the quality or depth of use of evidence.

3. Further investigate the topic of classroom management and develop measures to track change

International education surveys inquire about elements of focused classrooms (respectful students, lack of disruption) and some classroom management practices. These measures only scratch the surface of what contributes to focused classrooms. The topic of classroom management and focused classrooms needs to be further investigated to understand how low levels of respect and high levels of disruption can be addressed.

Next steps for AERO

• Inquire about activities that contribute to focused classrooms, to understand what works to achieve a focused classroom.
• Develop measures to monitor and track change in classroom management and focused classroom.

4. Better understand how schools, teachers and leaders can successfully implement evidence‑based practices and sustain their use

The characteristics of schools, teachers and leaders who use EBPs well are not fully understood. Understanding individual- and school-level factors will allow for more nuanced and comprehensive support for schools.

Next steps for AERO

• Continue to learn about the ways that characteristics already linked to use of EBPs (such as confidence, leadership and collaboration) can further enhance use of evidence in practice.
• Continue to identify characteristics and enablers that help schools and individuals to implement and sustain use of EBPs, including how these interact with each other.

Limitations

We acknowledge a range of limitations in this research.

• Data from the international education surveys were not collected specifically for this snapshot, hence results are only an indication of the frequency of use of EBPs.
• There may be an over-estimation of how much EBPs are used, due to reliance on survey data and limited information from teacher interviews.

References

Keywords: educational datasets, data analysis