Asia Pacific Journal of Corpus Research (아시아태평양코퍼스연구)

Institute for Corpus Research (국립인천대학교 코퍼스연구소)
권호 표지
DOI QR코드

DOI QR Code

Predicting CEFR Levels in L2 Oral Speech, Based on Lexical and Syntactic Complexity

  • Received : 2020.10.29
  • Accepted : 2021.07.28
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the wide spread of the Common European Framework of Reference (CEFR) scales, many studies attempt to apply them in routine teaching and rater training, while more evidence regarding criterial features at different CEFR levels are still urgently needed. The current study aims to explore complexity features that distinguish and predict CEFR proficiency levels in oral performance. Using a quantitative/corpus-based approach, this research analyzed lexical and syntactic complexity features over 80 transcriptions (includes A1, A2, B1 CEFR levels, and native speakers), based on an interview test, Standard Speaking Test (SST). ANOVA and correlation analysis were conducted to exclude insignificant complexity indices before the discriminant analysis. In the result, distinctive differences in complexity between CEFR speaking levels were observed, and with a combination of six major complexity features as predictors, 78.8% of the oral transcriptions were classified into the appropriate CEFR proficiency levels. It further confirms the possibility of predicting CEFR level of L2 learners based on their objective linguistic features. This study can be helpful as an empirical reference in language pedagogy, especially for L2 learners' self-assessment and teachers' prediction of students' proficiency levels. Also, it offers implications for the validation of the rating criteria, and improvement of rating system.

Keywords

References

  1. Bailey, K. (2005). Practical English Language Teaching: Speaking. New York: McGraw-Hill.
  2. Bhat, S., & Yoon, S. (2015). Automatic assessment of syntactic complexity for spontaneous speech scoring. Speech Communication, 67, 42-57. https://doi.org/10.1016/j.specom.2014.09.005
  3. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press.
  4. Council of Europe (2018). Common European Framework of Reference for Languages: Learning, Teaching, Assessment Companion Volume with New Descriptors. Strasbourg: Council of Europe.
  5. Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238. https://doi.org/10.2307/3587951
  6. Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: the roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35(2), 115-135. https://doi.org/10.1111/j.1467-9817.2010.01449.x
  7. Crossley, S. A., Salsbury, T., & McNamara, D. S. (2012). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29(2), 243-263. https://doi.org/10.1177/0265532211419331
  8. Ferris, D. R. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414-420. https://doi.org/10.2307/3587446
  9. Hawkins, J., & Buttery, P. (2010). Criterial features in learner corpora: theory and illustrations. English Profile Journal, 1(1).1-23.
  10. Hawkins, J., & Filipovic, L. (2012). Criterial Features in L2 English: Specifying the Reference Levels of the Common European Framework. Cambridge: Cambridge University Press.
  11. Izumi, E., Uchimoto, K., & Isahara, H. (2004). The NICT JLE corpus: Exploiting the language learners' speech database for research and education. International Journal of the Computer, the Internet and Management ,12(2), 119-125.
  12. Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12(4), 377-403. https://doi.org/10.1016/j.jslw.2003.09.001
  13. Kang, O., & Yan, X. (2018). Linguistic features distinguishing examinees' speaking performances at different proficiency levels. Journal of Language Testing & Assessment, 1, 24-39. https://doi.org/10.23977/langta.2018.11003
  14. Kim, J. (2014). Predicting L2 Writing Proficiency using linguistic complexity measures: A corpus-based study. English Teaching, 69(4), 27-51. https://doi.org/10.15858/engtea.69.4.201412.27
  15. Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4),474-496. https://doi.org/10.1075/ijcl.15.4.02lu
  16. Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers' language development. TESOL Quarterly, 45(1), 36-62. https://doi.org/10.5054/tq.2011.240859
  17. Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners' oral narratives. The Modern Language Journal, 96(2), 190-208. https://doi.org/10.1111/j.1540-4781.2011.01232_1.x
  18. Milanovic, M. (2009). Cambridge ESOL and the CEFR. Research Notes, 37, 2-5.
  19. Tono, Y. (Ed.). (2013). The CEFR-J Handbook: A Resource Book for Using CAN-DO Descriptors for English Language Teaching. Tokyo: Taishukan Publishing.
  20. Wisniewski, K. (2018). The empirical validity of the Common European Framework of Reference Scales. An exemplary study for the vocabulary and fluency scales in a language testing context. Applied Linguistics, 39(6), 933-959. https://doi.org/10.1093/applin/amw057
  21. Wolfe-Quintero, K., Inagaki, S., & Kim, H. Y. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy, & Complexity. Honolulu: University of Hawaii Press.
  22. Yoon, H. (2017). Linguistic complexity in L2 writing revisited: Issues of topic, proficiency, and construct multidimensionality, System, 66, 130-141. https://doi.org/10.1016/j.system.2017.03.007