Using grammar checkers to provide written corrective feedback (Paul John, Nina Woll, Mariane Gazaille, Université du Québec à Trois-Rivières; Walcir Cardoso, Concordia University)

This research project investigates the feasibility of using grammar-checking software to provide ESL learners with automatic feedback on writing errors. Overall, recent research confirms that feedback on error constitutes an important means of promoting accuracy within the communicative classroom (e.g., Bitchener, 2008; Bitchener et al., 2005; Chandler, 2003; Ellis, 2009; Ferris, 2006; Ferris et al., 2013; Guénette, 2007). Nonetheless, error correction is time-consuming for teachers and hence not always practical. One advantage of providing automatic feedback via grammar checkers, then, is that it frees up valuable teacher time. Another advantage is that students develop error correction skills that they can use independently (“strategy development” in Chapelle & Jamieson’s 2008 terms). All the same, important questions remain regarding the quality of the feedback that grammar checkers provide. Do certain grammar checkers deliver better feedback than others? The issue is particularly pertinent to second language writing since most grammar checkers are designed to address the writing errors produced by native speakers.
 
Our aim is to establish whether a given grammar checker i) correctly flags all errors (with or without providing appropriate correct forms and explanations) or ii) fails to flag actual errors (false negatives), and finally whether it iii) flags correct forms as errors (false alarms). Inappropriate suggestions for correction and false alarms are potentially problematic for ESL learners. While native speakers can rely on their intuition to override the grammar checker in such cases, ESL learners are essentially at the mercy of the software.
 
To investigate these issues, we examined the grammatical errors in 50 handwritten essays produced by 28 advanced francophone learners of English (nmale = 10, nfemale = 18, age 21-36), who were all enrolled in a university ESL teaching program in a predominantly French-speaking region of Quebec. The errors were partly classified according to whether they require contextual information to be identified (e.g., subject-verb agreement and anaphora) or apply to words in isolation (e.g., overgeneralizations of past tense –ed as in sitted, buyed). The former are more challenging for a grammar checker since they require either a rule-based syntactic parse or a corpus-based analysis. Previous research is comparatively limited in scope, tending to focus on how grammar checkers perform with regard to select grammar points such as article, preposition and collocation errors (e.g., De Felice & Pulman, 2008; Gamon et al., 2009; Tetrault & Chodorow, 2008; Yi et al., 2008).
 
Subsequently, we verified how well the learners’ errors are treated by three leading online resources (Grammarly, GrammarBase and Virtual Writing Tutor) and, for comparison, the grammar checker incorporated into Microsoft Word. Of the online grammar checkers, only Virtual Writing Tutor was developed specifically to target ESL writing errors.
Preliminary results indicate that all of the grammar checkers at times misfire in providing feedback. Nonetheless, and building on previous work (Buck, 2008; Hegelheimer, 2006; Potter & Fuller, 2008; Stapleton & Radia, 2010), we propose ways for ESL teachers to overcome the potential pitfalls of inaccurate feedback and thus to successfully incorporate grammar checkers into their ESL writing classes.
 
References
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17, 102-118.
Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective feedback on ESL student writing. Second Language Writing, 14, 191-205.
Buck, A. M. (2008). The Invisible Interface: MS Word in the Writing Center. Computers and Composition, 25, 396-415.
Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Second Language Writing, 12, 267-296.
Chapelle, C.A. & Jamieson, J. (2008). Tips for Teaching with CALL: Practical Approaches to Computer-Assisted Language Learning. Pearson-Longmann: White Plains, NY.
De Felice, R., & Pulman, S. G. (2008). A classifier-based approach to preposition and determiner error correction in L2 English. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), 169-176.
Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1, 3-18.
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 81-104). Cambridge: Cambridge University Press.
Ferris, D., Liu, H., Sinha, A., & Senna, M. (2013). Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22, 307-329.
Gamon, M., Leacok, C., Brockett, C., Dolan, W. B., Gao, J., Belenko, D., & Klementiev, A. (2009). Using Statistical Techniques and Web Search to Correct ESL Errors. CALICO Journal, 26(3), 491-511.
Guénette, D. (2007). Is feedback pedagogically correct? Research design issues in studies of feedback on writing. Second Language Writing, 16, 40-53.
Hegelheimer, V. (2006). Helping ESL writers through a multimodal, corpus-based, online grammar resource. CALICO Journal, 24 (1), 5-32.
Potter, R., & Fuller, D. (2008). My new teaching partner? Using the grammar checker in writing instruction. The English Journal, 98(1), 36-41.
Tetrault, J. R., & Chodorow, M. (2008). The ups and downs of preposition error detection in ESL writing. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), 865–872.
Yi, X., Gao, J., & Dolan, W. B. (2008). A web-based English proofing system for English as a second language users. In Proceedings of the Third International Joint Conference on Natural Language Processing, 619-624.