Creating the family engagement practice guides – detailed methodology

Detailed methodology (PDF, 699KB)

About the guides

The Australian Education Research Organisation (AERO) has released practice guides for practitioners about engaging with families to support children's early learning and development and students' learning outcomes. 

This detailed methodology describes the processes we followed to synthesise the available research evidence on family engagement for learning. A simplified description of this process is also available. 

What process did we use to create the guides?

The practice guides are based on findings from a rapid review process conducted by AERO. This process builds on work previously carried out by the Education Endowment Foundation ('the EEF review') (Axford et al., 2019).¹

Rapid reviews offer a methodologically rigorous way to synthesise research evidence in a timely and efficient manner (Garritty et al., 2021). While rapid review methodologies are used extensively in health to summarise bodies of research evidence, they are not yet a common practice within education research translation (White, 2021; Cirkony et al., 2021).

The specific steps involved in planning, collecting and reporting for this rapid review are:

1. Planning

Using the EEF search as a starting point: 

• refining the research question and defining key terms
• refining the eligibility criteria
• developing the search strategy

2. Collecting

• Carrying out the searches
• Screening studies (first by title/abstract and then by full text) 
• Carrying out quality assessments of included studies
• Carrying out data extraction from the included studies

3. Reporting

• Synthesising 'promising' and 'not promising' approaches from the included studies to provide clear and accessible practitioner guidance

What types of studies were eligible for inclusion?

We searched for research evidence to address the following research question:

What is the best current evidence on the practices and approaches early childhood education and care (ECEC) and school practitioners can use to promote and support family engagement in children’s learning?

Inclusion and exclusion criteria by population, activity, setting, study design, publication details and outcomes are listed in Table 1. These criteria were adapted from those used by the EEF (see Appendix B).

For this review:

“family engagement” is defined as the process of staff at early childhood services or schools working together with family members to support their child’s early learning and development or student learning outcomes.

“practitioners” are defined to include ECEC educators, teachers and leaders/directors, as well as school teachers, mid-level leaders and leaders.

Table 1: Eligibility criteria 

Where and how did we source the studies?

Our search strategy closely followed that of the EEF review, although fewer databases were searched given the scope and purpose of our review.

Table 2 lists the 5 databases used for this review. These were selected from the list of databases in the EEF review based on their relevance to the Australian context and availability. Search terms for each of the databases closely matched those used in the EEF review (with some adjustments to the limits on years and languages). Search terms are set out in Appendix C.

Database searches were carried out by AERO with assistance from The Australian Council for Educational Research (ACER) team in Melbourne. Search results were collated and converted into a standard Excel format. Screening was carried out by an AERO Senior Researcher, with queries about specific articles decided by the project team.

Table 2: Databases

How many studies did we include, and how did we synthesise the findings?

From an initial search locating 2,254 papers, 14 papers (5 meta-analyses and systematic reviews, and 9 primary studies) met the inclusion criteria. These were supplemented with 8 meta-analyses and systematic reviews from the original EEF review, the EEF review itself, and a meta-analysis on shared reading identified through a small additional targeted search (Noble et al, 2019). In total, 24 studies helped to inform the practice guides. The search process is illustrated in the PRISMA flow diagram³ in Appendix D. Some additional studies that did not meet the inclusion criteria, but may be of particular interest to Australian audiences, are listed in Appendix E.

The practice guides are designed to be concise documents presenting recommendations drawn from the research evidence, rather than a comprehensive summary of all of the included studies.⁴ For this reason, the project team developed a system for prioritising and synthesising key themes from the studies. This involved:

1. carrying out  quality assessments of the included meta-analyses and systematic reviews, by applying the same criteria used in the EEF review and considering AERO’s Standards of evidence⁵
2. extracting data from the included meta-analyses and systematic reviews
3. drawing out key themes of ‘promising’ and ‘not promising’ approaches identified in the reviews for ECEC, primary school and secondary school settings. To identify the key themes, greatest weight was placed on meta-analyses and systematic reviews that had higher quality assessments, were more recent, and described the strength of evidence of included primary studies as high. Potential themes were also considered for their relevance to the Australian context.
4. referring to primary studies when key themes required further illustration or explanation. These primary studies were identified either directly through the search or through the included meta-analyses or systematic reviews.

What other processes did we use to produce the guides?

To ensure that findings from the review were presented in a useful way, we formed a small project advisory group. The group consisted of accomplished practitioners from ECEC, primary school and secondary school nominated by the Australian Children’s Education and Care Quality Authority (ACECQA) and the Australian Institute for Teaching and School Leadership (AITSL). The group were consulted 3 times to provide input on:

• the relevance of the research evidence identified through the search process
• the structure, format and wording of the guides (to ensure they are clear, accessible, useful and relevant for practitioners)
• ideas for future resources to accompany the guides. 

We also sought insights and feedback from ACECQA, AITSL, the Australian Research Alliance for Children and Youth (ARACY), 3 peak bodies (the Australian Council of State School organisations, the Australian Parents Council, and Catholic School Parents Australia) and various state and territory jurisdictions.

AERO's practice guides are designed to be clear, concise, relevant to a range of ECEC and school contexts, and relevant to practitioners with different roles. As such, they are designed to be a starting point. We hope you find them useful in your work with families, and welcome any feedback.

References

Axford, N., Berry, V., Lloyd, J., Moore, D., Rogers, M., Hurst, A., Blockley, K., Durkin, H. and Minton, J. (2019). How can schools support parents’ engagement in their children’s learning? Evidence from research and practice. London: Education Endowment Foundation. (‘EEF review’)

Cirkony, C., Rickinson, M., Walsh, L., Gleeson, J., Salisbury, M. & Cutler, B. (2021). Reflecting on the conduct of rapid reviews for educational research. Educational Research. (Forthcoming).

Education Endowment Foundation (2019). Working with parents to support children’s learning (Guidance Report). London: Education Endowment Foundation. 

Evidence for Learning (2019). Working with parents to support children’s learning (Guidance Report). Sydney: Evidence for Learning. 

Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021). Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. Journal of Clinical Epidemiology, 130, 13–22. 

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, 71.
 

White, H. (2020). ‘The global evidence architecture in health and education: A comparative scorecard’. In Gorard, S. (Ed.), Getting Evidence into Education: Evaluating the Routes to Policy and Practice (1st ed.). Routledge, 20–33. 

Appendix A: Included studies

Note: Studies cited in the practice guides are indicated by an asterisk (*). For a description of studies cited in the practice guides (including additional primary studies identified through the reviews listed below), see the Annotated reference list.

Umbrella reviews meta-analyses and systematic reviews

*Grindal, T., Bowne, J. B., Yoshikawa, H., Schindler, H. S., Duncan, G. J., Magnuson, K., & Shonkoff, J. P. (2016). The added impact of parenting education in early childhood education programs: A meta-analysis. Children and Youth Services Review, 70, 238–249. https://doi.org/10.1016/j.childyouth.2016.09.018

*Higgins, S., & Katsipataki, M. (2015). Evidence from meta-analysis about parental involvement in education which supports their children’s learning. Journal of Children’s Services, 10(3), 280–290. https://doi.org/10.1108/JCS-02-2015-0009
 

Kim, J. S., & Quinn, D. M. (2013). The Effects of Summer Reading on Low-Income Children’s Literacy Achievement From Kindergarten to Grade 8: A Meta-Analysis of Classroom and Home Interventions. Review of Educational Research, 83(3), 386–431. https://doi.org/10.3102/0034654313483906

Lowe, K., Harrison, N., Tennent, C., Guenther, J., Vass, G., & Moodie, N. (2019). Factors affecting the development of school and Indigenous community engagement: A systematic review. The Australian Educational Researcher, 46(2), 253–271. https://doi.org/10.1007/s13384-019-00314-6

McDonald, S. (2019). Kindergarten Parent Engagement and Student Reading Literacy in Title I Schools: A Systematic Literature Review and Meta-Synthesis. Doctoral dissertation. Nova Southeastern University. https://nsuworks.nova.edu/cgi/viewcontent.cgi?article=1211&context=fse_…

*Noble, C., Sala, G., Peter, M., Lingwood, J., Rowland, C., Gobet, F., & Pine, J. (2019). The impact of shared book reading on children’s language skills: A meta-analysis. Educational Research Review, 28, 100290. https://doi.org/10.1016/j.edurev.2019.100290

*O’Connor, A., Nolan, A., Bergmeier, H., Hooley, M., Olsson, C., Cann, W., Williams-Smith, J., & Skouteris, H. (2017). Early childhood education and care educators supporting parent-child
relationships: A systematic literature review. Early Years, 37(4), 400–422. https://doi.org/10.1080/09575146.2016.1233169

See, B. H. (2015a). Identifying the most promising parental involvement interventions with impact on learning outcomes for primary school-aged children (1st report). School of Education, Durham University. www.dur.ac.uk/resources/education/SuttonTrustReportScopingreview1.pdf

*See, B. H. (2015b). Identifying the most promising parental involvement interventions with impact on learning outcomes for primary school-aged children (2nd report). School of Education, Durham University. www.dur.ac.uk/resources/education/SuttonTrustReportScopingreview2.pdf

See, B. H. & Gorard, S. (2013). What do rigorous evaluations tell us about the most promising parental involvement interventions? A critical review of what works for disadvantaged children in different age groups. London: Nuffield Foundation. www.nuffieldfoundation.org/sites/default/files/files/What_do_
rigorous_evaluations_tell_us_about_the_most_promising_parental_involvement_interventions.pdf

*See, B. H., & Gorard, S. (2015). Does intervening to enhance parental involvement in education lead to better academic results for children? An extended review. Journal of Children’s Services, 10(3), 252–264. https://doi.org/10.1108/JCS-02-2015-0008

*See, B. H., Gorard, S., El-Soufi, N., Lu, B., Siddiqui, N., & Dong, L. (2021). A systematic review of the impact of technology-mediated parental engagement on student outcomes. Educational Research and Evaluation, 1–32. https://doi.org/10.10 80/13803611.2021.1924791

*Sheridan, S. M., Smith, T. E., Moorman Kim, E., Beretvas, S. N., & Park, S. (2019). A Meta-Analysis of Family-School Interventions and Children’s Social-Emotional Functioning: Moderators and Components of Efficacy. Review of Educational Research, 89(2), 296–332. https://doi.org/10.3102/0034654318825437

*Smith, T. E., Sheridan, S. M., Kim, E. M., Park, S., & Beretvas, S. N. (2020). The Effects of Family-School Partnership Interventions on Academic and Social-Emotional Functioning: A Meta-Analysis Exploring What Works for Whom. Educational Psychology Review, 32(2), 511–544. https://doi.org/10.1007/s10648-019-09509-w

Primary studies (early childhood education and care) 

*Burgoyne, K., Gardner, R., Whiteley, H., Snowling, M. J., & Hulme, C. (2018). Evaluation of a parent-delivered early language enrichment programme: Evidence from a randomised controlled trial. Journal of Child Psychology and Psychiatry, 59(5), 545–555. https://doi.org/10.1111/jcpp.12819

*Cabell, S. Q., Zucker, T. A., DeCoster, J., Copp, S. B., & Landry, S. (2019). Impact of a Parent Text Messaging Program on Pre- Kindergarteners’ Literacy Development. AERA Open, 5(1), 233285841983333. https://doi.org/10.1177/2332858419833339

*Neumann, M. M. (2018). The effects of a parent–child environmental print program on emergent literacy. Journal of Early Childhood Research, 16(4), 337–348. https://doi.org/10.1177/1476718X18809120

*Sheridan, S. M., Knoche, L. L., Boise, C. E., Moen, A. L., Lester, H., Edwards, C. P., Schumacher, R., & Cheng, K. (2019). Supporting preschool children with developmental concerns: Effects of the Getting Ready intervention on school-based social competencies and relationships. Early Childhood
Research Quarterly, 48, 303–316. https://doi.org/10.1016/j.ecresq.2019.03.008

*Soto, X., Seven, Y., McKenna, M., Madsen, K., Peters-Sanders, L., Kelley, E. S., & Goldsteina, H. (2020). Iterative Development of a Home Review Program to Promote Preschoolers’ Vocabulary Skills: Social Validity and Learning Outcomes. Language, Speech & Hearing Services in Schools, 51(2), 371–
389. https://doi.org/10.1044/2019_LSHSS-19-00011

*Teepe, R. C., Molenaar, I., Oostdam, R., Fukkink, R., & Verhoeven, L. (2019). Helping parents enhance vocabulary development in preschool children: Effects of a family literacy program. Early Childhood Research Quarterly, 48, 226–236. https://doi.org/10.1016/j.ecresq.2019.03.001

*York, B. N., Loeb, S., & Doss, C. (2019). One Step at a Time: The Effects of an Early Literacy Text-Messaging Program for Parents of Preschoolers. Journal of Human Resources, 54(3), 537–566.
https://doi.org/10.3368/jhr.54.3.0517-8756R

Primary studies (primary school)

Nichols, S., & Hill, S. (2020). New Word Hunters: A family engagement strategy to extend Year 1 children’s vocabulary. Australian Journal of Language & Literacy, 43(2), 129–140.

Primary studies (secondary school) 

Santana, M., Nussbaum, M., Carmona, R., & Claro, S. (2019). Having Fun Doing Math: Text Messages Promoting Parent Involvement Increased Student Learning. Journal of Research on Educational Effectiveness, 12(2), 251–273. https://doi.org/10.1080/19345747.2018.1543374

Appendix B: How we built on the existing evidence base 

Existing EEF process

In 2019, the EEF published an evidence review and guidance paper on how schools can support parental engagement in their children’s learning (Axford et al., 2019). One component of the project involved synthesising international evidence on “activities delivered in or by schools and early years settings that promote and support [effective parenting] practices, particularly for children from disadvantaged backgrounds.”

The review identified 10 relevant reviews and 71 primary studies not already included in the systematic reviews. Of these, the 9 most relevant reviews and 47 most relevant primary studies were included in the synthesis.

AERO process

Rather than duplicate this work, AERO drew upon the search strategy and findings developed by the EEF when identifying research evidence to inform its practice guides for practitioners. This allowed AERO to include relevant research published since 2017.

Where possible, the search strategy remained closely aligned with the EEF search strategy. However, some important adjustments were made to:

• ensure the practice guides could be quickly produced and made available to practitioners
• focus particularly on research evidence relevant to Australian settings
• reflect the increasing recognition of working with “families” rather than “parents”.

These adjustments are set out in Table 3.

Table 3 Detailed adjustments in the review processes

Appendix C: Search terms

Table 5: Search terms for Education Research Complete (via EBSCOhost)

Table 6 Search terms for PsycINFO (via OvidSp)

Table 7: Search terms for Scopus

Appendix D: PRISMA flow diagram⁷

Appendix E: Additional studies of interest to Australian audiences

The studies below were excluded at the full-text screening stage as they did not meet the inclusion criteria for this rapid review. They are listed here as they may be of particular interest to Australian audiences as they illustrate detailed examples of family engagement or early literacy approaches.

Harwood, V., & Murray, N. (2019). Strategic discourse production and parent involvement: Including parent knowledge and practices in the Lead My Learning campaign. International Journal of Inclusive Education, 23(4), 353–368. https://doi.org/10.1080/13603116.2019.1571119

Jose, K., Taylor, C. L., Venn, A., Jones, R., Preen, D., Wyndow, P., Stubbs, M., & Hansen, E. (2020). How outreach facilitates family engagement with universal early childhood health and education services in Tasmania, Australia: An ethnographic study. Early Childhood Research Quarterly, 53, 391–402. https://doi.org/10.1016/j.ecresq.2020.05.006

Niklas, F., Cohrssen, C., & Tayler, C. (2018). Making a difference to children’s reasoning skills before school-entry: The contribution of the home learning environment. Contemporary Educational Psychology, 54, 79–88. https://doi.org/10.1016/j.cedpsych.2018.06.001

Renshaw, L., & Goodhue, R. (2020). National Early Language and Literacy Strategy: Discussion Paper. Canberra, Australia: ARACY for The National Early Language and Literacy Coalition. https://earlylanguageandliteracy.org.au/wp-content/uploads/2020/10/Nati…