The guide (PDF, 97KB) helps educators, teachers, leaders and policymakers reflect on a piece of research that provides evidence about the effectiveness of a particular policy, program or practice (that is, an approach), which they may be considering implementing.
Using the guide
First identify a piece of research evidence on a particular approach that you are considering implementing. Then, answer the series of guiding questions below that will prompt you to consider: what the research says; how relevant the research is to your context; whether you should implement the approach; and what you can do to ensure successful implementation.
The guide can be used individually or in a group as part of a community of practice.
context (or contextual factors)
Context is the social, cultural and environmental factors found in research settings. Taking context into account in research studies is important because context can affect the outcomes of research (i.e. evidence generated in one context may not necessarily apply to a different context). Evidence is most relevant when it has been generated in a context similar to the context in which it will be applied. Examples of ‘context’ may include location, demographics of research participants, or the level of organisational support for the particular approach being researched.
Relevant evidence is evidence produced in contexts that are similar to one’s own context. Evidence can also be considered relevant when it is derived from a large number of studies conducted over a wide range of contexts.
Data is information that is collected and analysed in order to produce findings and/or to inform decision-making. Data can be qualitative (for example, teacher observations or quotes from students) or quantitative (for example, student test scores or attendance data).
An educational approach is effective if it causes (see causation above) a desired change in a particular outcome. This desired change can be an increase in an outcome (for example, increases in student achievement) or it can be a decrease in an outcome (for example, reduction in student absenteeism).
association (or correlation)
An association is when there is a relationship between two elements, factors or events, but the association cannot be proved or explained. Associations can be positive (for example, higher socioeconomic status is associated with higher student achievement) or negative (for example, higher student absenteeism is associated with lower student achievement).
Evaluation is the systematic and objective assessment of an approach. Evaluation provides evidence of what has been done well, what could be done better, the extent to which objectives have been achieved and/or the impact of the approach. This evidence can then be used to inform ongoing decision-making regarding the approach.
An outcome measure is an observation that can be used to measure the effect of a particular approach. Outcome measures can be qualitative (such as quotes or observations) or quantitative (such as test scores). For example, when examining whether a particular approach helps students understand a concept, a teacher could set an assessment. The student assessment score could then be used as an outcome measure of student understanding.
causation (or cause(s)/causal/causal evidence)
Causation is when one element, factor or event is known to cause another (for example, a particular teaching practice is known to lead to improvements in student test scores). To prove causation between two things (let’s call them A and B), researchers need to show: 1. that there is an association between A and B; 2. that A happens before B; and 3. that B is not caused by a third thing (that is, C or D). In education settings, proving causation is often challenging because of the many influences on teacher and student outcomes.
The worksheet (PDF, 392KB) is designed for reflecting on primary studies, which are individual studies reporting on data collected and analysed by the researchers themselves. It isn’t designed for reflecting on research that summarises a body of evidence (for example, a literature review).
If you’re an educator or teacher, using this resource to reflect on research can help you to make decisions about your practice. If you’re a leader, you can use this resource to support your team to engage with evidence as part of their ongoing professional development.
Ways to use this resource
- Personal professional learning to become more familiar with research.
- Professional learning in a group, such as a community of practice – use the completed worksheets to discuss the education approach as team.
- Keep the completed worksheet as a record of decision-making about a particular approach.
- Revisit the completed worksheet as a reminder of the questions you may still have about an approach (and to focus your efforts on seeking answers).
- Use the questions to structure discussions about an approach with colleagues.
An example of a completed research reflection guide worksheet. Download the example PDF (392KB) or read the example below.
Robyn is the Centre Director at a community kindergarten and early childhood education and care (ECEC) service owned and managed by the local Aboriginal and Torres Strait Islander people. The service is in the outer suburbs of an Australian capital city. All children who attend come from an Aboriginal and/or Torres Strait Islander background. All speak English as their first language.
Robyn recently read about the Abecedarian Approach Australia in an online blog and has followed up by reading the original journal article to better understand whether the approach is evidence-based and would be relevant for her centre.
Research reflection guide: My notes
Now that Robyn has reflected on the research, she can decide what to do next. She can choose actions that apply to her context. She could:
- keep the completed worksheet as a record of decision-making about a particular approach
- revisit the completed worksheet as a reminder about what questions she may still have about an approach (and to focus her efforts on seeking answers)
- use the completed worksheets to discuss the education approach as team, for example as part of professional learning in a group community of practice
- use the questions to structure discussions about an approach with colleagues
- find out more about the approach by:
- searching academic search engines or Google Scholar
- checking the website of the authors’ institution
- contacting the authors directly to ask specific questions about the approach
- find out if professional learning is available to support the approach.
Robyn decides she wants to use the completed worksheet to discuss the approach with her team. But first, she decides to find out more about the Abecedarian approach.
She takes the following steps:
- She checks the authors’ institution (the University of Melbourne) and finds information relating to Abecedarian Approach Australia (3a).
- She conducts a search on Google Scholar, using key words associated with the approach (for example, ‘Abecedarian Approach Australia’).
- She searches the institution website and finds the Research in Effective Education in Early Childhood (REEaCh) website has research briefs reporting on the approach, as well as other related research.
- She finds out if there is professional learning available to support the approach by checking:
- the authors’ institution website and finds information about 3a Practitioner, Coach and Affiliate training programs
- government education websites to see whether there is funding available to access the training.