17/10/2023
The Cambridge English exams and learning materials help millions of people all over the world teach and learn English. But what difference does this make to their lives and more widely on society? The English Impact team at Cambridge University Press & Assessment works with students, teachers, schools, parents, universities and ministries of education to measure the difference Cambridge has on people learning and teaching English. Dr Brigita Seguis explains how they do this and why it matters.
Dr Brigita Seguis, Head of Impact Evaluation, Cambridge University Press & Assessment
Importance of Measuring Impact
“Any organisation that embraces corporate social responsibility has to monitor and report on its impact. We’re an educational organisation and part of the University of Cambridge, which makes this commitment even more important. Our University’s mission is to contribute to society through the pursuit of education, learning and research at the highest international levels of excellence. With this purpose in mind, we have to demonstrate how we actually do this. And that’s what we do through our impact evaluation work.
When we talk about impact, we think about all the outcomes; positive, negative, intended, and sometimes unintended that our Cambridge English products and services have on a range of different stakeholders. These stakeholders can be teachers, learners, parents, educational institutions, schools, and society as a whole. For example, we might set out to find out the impact of newly introduced Cambridge textbooks, courses or qualifications on a group of learners and teachers, or more widely on a school system. We might also monitor and measure impact over a longer period of time, focusing on how the introduction of learning materials or qualifications helped increase English language proficiency levels at the country level.
How We Approach Impact Evaluation
Our approach to impact evaluation focuses on looking back, looking forward and looking beyond. Firstly, measuring impact is about looking back and seeing what has worked, and what if anything didn’t work as well as intended, and understanding why it didn’t work. Secondly, it’s about looking ahead, so if a particular English language learning or teaching initiative has worked well, we ask: how can we do it even better? How can we replicate this success in other contexts? And lastly, if something didn’t work well, how can we address the underlying issues and support improvement?
A lot of our work focusses on carrying out impact evaluation studies. When gathering data for an impact study, we talk directly to learners, teachers and parents, we visit schools and observe classes, and we analyse learning and assessment data. I recently led a study looking at the impact of Evolve Digital,one of our online courses which helps adults speak English with confidence. Our research found that the course kept participants motivated and engaged, allowed them to make significant learning gains and helped increase their speaking confidence.
When we write an impact evaluation report, we start by giving an overview of the context, because this is crucial to understanding the changes and actions we are recommending. For our Evolve Digital study, the context was particularly important because the course was delivered as part of a wider teacher training initiative, initiated by the Ministry of Education in Japan. Once we had the findings from our report, we were able to provide recommendations to the Ministry on how they can better integrate the English language course and the teacher training initiative for next year. We also share the findings of the study with our product development team in Cambridge to help them improve the course in the future.
Measuring and Promoting Social Impact
Our work also includes measuring the wider social impact we can make as an organisation. We recently worked in partnership with the NHS on a study that looked at the barriers and challenges that refugees face when learning English as part of their attempts to return to practice in their new host country. The most common barriers included access to technology, childcare, erratic work schedules, as well as learning and exam anxiety. Interestingly, a lot of these barriers apply to other groups of adult learners too, which made our findings even more meaningful and significant.
One of the more practical recommendations that we were able to share with refugee charities and other stakeholders was the importance of offering adult learners a choice when it comes to learning English. While some adults are not used to the online learning environments and largely prefer the traditional classroom setting, others clearly prefer to take classes online due to the flexibility they offer. We also emphasised the importance of technology in facilitating refugees’ access not only to the online classes, but also additional learning materials that can help them prepare for the required English exams or further improve their English.
Ultimately our work is about making a positive change in people’s lives by focusing on something very practical. A few of the refugees who took part in our study told us: ‘thank you for listening to us and hearing our challenges’ because they often felt lost in the system. I guess it’s what you call the impact of an impact study, and it is a very rewarding feeling.
I’m really proud of the fact that we get direct contact with our stakeholders including teachers, learners and parents and school leaders. It’s being able to listen to them and to respond to their needs that really makes our jobs worthwhile".