Using Natural Language Processing Technology to Analyze Teachers’ Written Feedback on Chinese Students’ English Essays

ming liu

Abstract


Writing an essay is a very important skill for students to master, but a difficult task for them to overcome. It is particularly true for English as Second Language (ESL) students in China. It would be very useful if students can receive timely and effective feedback about their writing. In order to build an automatic feedback system, we need to understand the relationship between textual features and human teacher feedback, and how well those features were used for predicting feedback rating. In this study, we analyzed 105 Chinese English majors’ essays with teachers’ feedback and used Coh-Metrix, a computational linguistic tool, to extract features from their writing. The study results showed some feedback was moderately correlated to some textual features (e.g. text easability cohesion and lexical diversity were related to coherence feedback) and those feedback are more predictable, such as spelling, grammar, supporting ideas and coherence. This finding has important implications for building automated writing feedback tool.


Keywords


Teacher Written Feedback; Advanced Educational Technology; Natural Language Processing

Full Text:

PDF

References


Anderson, J. (2005). Mechanically Inclined:Building Grammar, Usage, and Style into Writer's Workshop.

Brannon, L., & Knoblauch, C. H. (1982). On students' rights to their own texts: A model of teacher response. College Composition and Communication, 33, 157-166.

Britt, M. A., Wiemer-Hastings, P., Larson, A. A., & Perfetti, C. A. (2004). Using Intelligent Feedback to Improve Sourcing and Integration in Students' Essays. Int. J. Artif. Intell. Ed., 14, 359-374.

Bureau of Statistis of China, N. (2013). China Statistical YearBook.

Burstein, J., Chodorow, M., & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing service. AI Magazine, 25, 27. doi: 10.1002/rcm.5057

Crossley, S., & McNamara, D. (2010). Cohesion, coherence, and expert evaluations of writing proficiency. The 32nd Annual Conference of the Cognitive Science Society. Austin: TX.

Crossley, S. a., & McNamara, D. S. (2011a). Text Coherence and Judgments of Essay Quality: Models of Quality and Coherence. The 33rd Annual Conference of the Cognitive Science Society.

Crossley, S. a., & McNamara, D. S. (2011b). Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing. International Journal of Continuing Engineering Education and Life-Long Learning, 21, 170. doi: 10.1504/IJCEELL.2011.040197

Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency： The role of cohesion， readability， and lexical difficulty. Journal of Research in Reading, 35, 115-135.

Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-metrix: analysis of text on cohesion and language. Behavior research methods, instruments, & computers, 36, 193-202.

Haswell, R. (2006). The complexities of responding to student writing; or, looking for shortcuts via the road of excess. Across the Disciplines, 3.

Kintsch, W., & van Dijk, T. (1978). Towards a model of text comprehension and production. Psychological Review, 85, 363-394.

Lafferty, J., Sleator, D., & Temperley, D. (1992). Grammatical Trigrams: A Probabilistic Model of Link Grammar. Paper presented at the Proceedings of the AAAI Conference on Probabilistic Approaches to Natural Language.

Lee, I. (2004). Error correction in L2 secondary writing classrooms: The case of Hong Kong. Journal of Second Language Writing, 13, 285-312. doi: 10.1016/j.jslw.2004.08.001

Leki, I. (1991). The preferences of ESL students for error correction in college-level writing classes. Foreign Language Annals, 24, 203-218.

Liu, M., Calvo, R., & Rus, V. (2014). Automatic Generation and Ranking of Questions for Critical Review. Educational Technology & Society, 17, 333-346.

Liu, M., Calvo, R. A., & Rus, V. (2010). Automatic Question Generation for Literature Review Writing Support. Carnegie Mellon University, USA: Springer's Lecture Notes in Computer Science

Pennebaker, J. W., & Francis, M. E. (1999). Linguistic inquiry and word count (LIWC).

Rufenacht, R. M., McCarthy, P. M., & Lamkin, T. A. (2011). Fairy Tales and ESL Texts: An Analysis of Linguistic Features Using the Gramulator. Proceedings of the Twenty-Fourth International Florida Artificial Intelligence Research Society Conference.

Shermis, M. D., & Burstein, J. (2003). Automated essay scoring: A cross-disciplinary perspective. 16.

The student, the text, and the classroom context: A case study of teacher response, 7 23-55 (2000).

Thiesmeyer, E. C., & Theismeyer, J. E. (1990). Editor:A System for Checking Usage, Mechanics, Vocabulary, and Structure.

Villalon, J., Kearney, P., Calvo, R. A., & Reimann, P. (2008). Glosser: Enhanced Feedback for Student Writing Tasks.

Williams, R., & Dreher, H. (2004). Automatically Grading Essays with Markit©. Issues in Informing Science and Information Technology, 1, 693-700.


Refbacks

  • There are currently no refbacks.


e-ISSN: 1694-2116

p-ISSN: 1694-2493