Integrated skills testing in an academic English program: A potential to increase written proficiency (Poonam Anand, Carleton University)

To advance second language learners’ written proficiency, instead of a single proxy measure of students’ writing (e.g. impromptu writing), assessment in English for Academic Purposes (EAP) programs should be integrated, contextualized and formative (Fox, 2004). Recently, an EAP program at a Canadian university introduced such integrated skills assessment in the form of ‘Reading to Write’ and ‘Listening to Speak’ exams as the exit requirement. These integrated skills tests replaced the traditional discreet-point multiple-choice exams and 5-paragraph essays.
 
The purpose of my longitudinal case study (Yin, 2014) is to explore the phenomenon of washback in the context of assessment innovation in this EAP program. Washback means, “how assessment instruments affect educational practices and beliefs” (Cohen, 1994, p.41) and innovation is defined as “an idea, practice or object that is perceived as new by an individual or other unit of adoption” (Roger, 2003, p.60). The study of innovation practice increases the understanding of how and why washback comes about (Henrichsen, 1989; Wall, 2005).
 
My multi-phased study has made use of baseline data, before the innovation became operational. From this foundation, I compared the pre- and post- innovation washback by exploring educators’ and students’ accounts of assessment practices by answering the research question: “Can a high-stakes exam be used to leverage positive washback on teaching and learning? If so, how?” Positive washback is the alignment between curriculum, test and the construct to which they are directed (Green, 2007).
 
In this presentation, I will explain students’ accounts of assessment practices before- and after-change. A total of 254 students completed online surveys over a period of four semesters; nine of these students participated in three focus groups to further confirm the questionnaire results. The statistical analysis of the questionnaire and thematic analysis of the focus group data has revealed that the previous exams exerted negative washback in the form of decontextualized learning. There was a disjuncture between classroom activities and testing exercises. Additionally, the old exam promoted both a splitting and hierarchal skill setting, which doesn’t necessary prepare students for university studies. It also promoted an unintended curriculum that was focused on the preparation for the final discreet-point, multiple-choice exams.
 
However, the new integrated skills assessment seems to address some of these problems Firstly, constructed written responses, which respond to a reading or listening text, reflect a better interpretation of students’ language proficiency. Secondly, integrating skills promote problem/project-based learning by incorporating thematic learning through situated cognition (Rogoff, 1990; Lave & Wenger, 1991) as opposed to decontextualized learning. Finally, integrated skills tests enhance the face and content validity of the writing assessment by providing an evidence of washback in the form of “evidential link between the teaching or learning outcomes and the test properties thought to influence them” Messick (1996, p. 247).
 
These findings are pertinent to EAP administrators and teachers in addressing academic literacy concerns, and promoting academic readiness of university-bound second language learners.
 
References
Cohen, A. D. (1994). Assessing language ability in the classroom (2nd ed.). New York: Heinle and Heinle.
Fox, J. (2004). Test decisions over time: Tracking validity. Language Testing, 21, 437–465. doi:10.1191/0265532204lt292oa.
Green, A. B. (2007). IELTS washback in context: Preparation for academic writing in higher education. Studies in language testing, 25. Cambridge, England: Cambridge University Press/Cambridge ESOL.
Henrichsen, L. E. 1989. Diffusion of Innovations in English Language Teaching: The ELEC Effort in Japan, 1956-1968. New York: Greenwood Press.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge university press.
Messick, S. (1996). Validity and washback in language testing. ETS Research Report Series, 1996(1), i-18.
Rogers, E. M. (2010). Diffusion of innovations. Simon and Schuster.
Rogoff, B. (1990). Apprenticeship in thinking: Children’s guided participation in culture.
Wall, D. (2005). The impact of high-stakes examinations on classroom teaching: A case study using insights from testing and innovation theory. Studies in language testing, 22. Cambridge, England: Cambridge University Press.
Yin, R. K. (2014). Case study research: Design and methods. Sage publications.