One question, years of data: Students learning English as an additional language
How long does it take students learning English as an additional language (EAL) in New South Wales public schools to develop the English language skills to participate equitably in curriculum learning? This question sits at the heart of AERO's Learning English While Learning the Curriculum project, and answering it reliably required rigorous quality assurance at every stage of the work.
Given the methodological complexity of this research, AERO used a range of quality assurance and review processes throughout the project. Here is how that worked in practice.
Developing the research approach and methodology
An overall research approach was determined early in the project, with the methodology developed in consultation with key stakeholders and an expert advisory group.
AERO's project team worked closely with the NSW Department of Education (DoE) Multicultural Education Unit to understand how the data had been collected, including its strengths and limitations. An expert advisory group and additional subject matter experts were also engaged to support the work at key stages. Full details of these contributors are provided in the full report.
Input from the DoE and the advisory group informed the direction of the research and the development of the methodology, helping to ensure the work was appropriate to the data and context, methodologically sound, and relevant to policy and practice.
Ethics and approvals
Following development of the methodology, AERO signed data agreements with the NSW DoE and NSW Education Standards Authority (NESA) to enable the use of deidentified data about NSW school students. The Macquarie University Human Research Ethics Committee confirmed that the project met the requirements for exemption from formal ethical review, based on its use of existing data and negligible risk to participants.
Preparing and analysing the data
The project drew on 3 large datasets linked using shared school identifiers. Data preparation and analysis were subject to internal structured quality assurance processes, with each analytical task undertaken by a statistician and independently checked by at least 2 data analysts.
Interpreting results
Multiple sources of expertise shaped the interpretation of results, including the expert advisory group, NSW DoE and NESA. Their detailed feedback contributed to:
- strengthening explanations of methods and findings
- refining terminology and improving clarity
- identifying and addressing risks of deficit framing
- highlighting how findings may be interpreted by different audiences.
In particular, ongoing input from the NSW DoE Multicultural Education Unit helped interpret patterns in the data and ensured findings were understood in light of system context and the known characteristics of EAL student cohorts.
Reviewing outputs
Multiple external advisors and system stakeholders reviewed draft reports, including the NSW DoE, NESA and the expert advisory group. Their input contributed to appropriate communication of findings and their implications for education.
At this time, NSW DoE and NESA confirmed that data had been used appropriately and interpreted in line with system knowledge. External experts provided detailed feedback on the clarity of terminology, the analytical methods and technical interpretation of results, explanations of methods and findings, and the framing of results.
Sharing the findings
AERO shared the findings externally through briefings to AERO's Board and key national stakeholders, including the Australian Government. We then published a full report with detailed and transparent descriptions of methods, data sources and limitations, accompanied by a plain language research summary.
Findings from the first part of the research have already been published in The Australian Journal of Language and Literacy, after being subject to the journal's blind peer review process.