Automated writing evaluation tools in Business English Writing: An experimental study among Indonesian undergraduates
Keywords:
Automated writing evaluation, Business English writing, Indonesian EFL students, Self-regulated learning, Writing self-efficacyAbstract
Business English writing competence is increasingly critical for Indonesian university graduates navigating global professional environments, yet many students continue to experience persistent difficulties with grammar, coherence, and professional register. Automated Writing Evaluation (AWE) tools such as Grammarly and Grammark provide immediate, data-driven feedback that may support writing development beyond surface-level error correction (Barrot, 2021; Calma et al., 2022). This quasi-experimental study examined whether AWE integration in a Business English writing course improved Indonesian undergraduates' perceived usefulness, ease of use, motivation, self-regulated learning (SRL), and writing self-efficacy (WSE), relative to conventional instructor feedback. Sixty undergraduate students were randomly assigned to an AWE group (n = 30) or a control group (n = 30). A validated 38-item Likert questionnaire and open-ended reflective prompts were administered post-intervention. Three key findings emerged: (1) AWE significantly enhanced perceived usefulness (d = 2.92) and ease of use (d = 2.74); (2) motivation, self-regulated learning, and writing self-efficacy increased substantially in the AWE group; and (3) students unanimously recommended a hybrid AWE–teacher feedback model to address AWE's limitations in higher-order writing development (Thi & Nikolov, 2022). These findings suggest that AWE integration holds considerable promise for Indonesian EFL Business English instruction, particularly when systematically complemented by expert human feedback.
References
Adi, I. S. M. A., PUTRI, A. M. J., & Sardi, A. (2024). Examining university students' Business English Writing performance: Frequent errors and pitfalls. Research and Innovation in Applied Linguistics [RIAL], 2(2), 79-94.
Al-Inbari, F. A. Y., & Al-Wasy, B. Q. M. (2023). The impact of automated writing evaluation (AWE) on EFL learners’ peer and self-editing. Education and Information Technologies, 28(6), 6645-6665. https://doi.org/10.1007/s10639-022-11432-7
Ariyanti, A., Fitriana, R., & Pane, W. S. (2018). Self-regulated learning in writing of EFL learners. IJELTAL (Indonesian Journal of English Language Teaching and Applied Linguistics), 3(1), 155-166.
Artino, A. R. (2012). Academic self-efficacy: from educational theory to instructional practice. Perspectives on medical education, 1(2), 76-85. https://doi.org/10.1007/s40037-012-0012-5
Asiyah, S., Budiyanto, C., & Tamrin, A. (2018). Technology acceptance model in the analysis of the influence of e-learning implementation to students' motivation. Indonesian Journal of Informatics Education, 2(1), 51-56.
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological review, 84(2), 191. https://doi.org/10.1037/0033-295X.84.2.191
Barrot, J. S. (2023). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning, 36(4), 584-607. https://doi.org/10.1080/09588221.2021.1936071
Bitchener, J., & Knoch, U. (2010). The contribution of written corrective feedback to language development: A ten month investigation. Applied linguistics, 31(2), 193-214. https://doi.org/10.1093/applin/amp016
Bozdoğan, D., & Kasap, B. (2019). Writing skills and communication competence of undergraduate students in an engineering English course. Journal of Language Teaching and Learning, 9(2), 49-65. https://doi.org/10.33335/jltl.171
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Bryant, C., Yuan, Z., Qorib, M. R., Cao, H., Ng, H. T., & Briscoe, T. (2023). Grammatical error correction: A survey of the state of the art. Computational Linguistics, 643-701. https://doi.org/10.1162/coli_a_00478
Calma, A., Cotronei-Baird, V., & Chia, A. (2022). Grammarly: An instructional intervention for writing enhancement in management education. The International Journal of Management Education, 20(3), 100704. https://doi.org/10.1016/j.ijme.2022.100704
Chen, C. F. E., & Cheng, W. Y. E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2), 94–112.
Cheng, G. (2017). The impact of online automated feedback on students' reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18-27. https://doi.org/10.1016/j.iheduc.2017.04.002
Damayanti, S., & Azizah, S. (2024). Grammarly’s effectiveness in enhancing English writing: A case study of vocational high school students. PANYONARA: Journal of English Education, 6(2), 15-24.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 13(3), 319-340. https://doi.org/10.2307/249008
Dewanti, R. (2018). The influence of self-efficacy on students' writing performance. Journal of English Education, 7(2), 145–162.
Dewi, U. (2022). Grammarly as automated writing evaluation: Its effectiveness from EFL students' perceptions. Lingua Cultura, 16(2), 155–161. https://doi.org/10.21512/lc.v16i2.8315
Dikli, S., & Bleyle, S. (2014). Automated essay scoring feedback for second language writers: How does it compare to instructor feedback?. Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006
Ellis, R. (2009). A typology of written corrective feedback types. ELT Journal, 63(2), 97–107. https://doi.org/10.1093/elt/ccn023
Ferris, D. R. (2004). The “grammar correction” debate in L2 writing: Where are we, and where do we go from here?. Journal of Second Language Writing, 13(1), 49–62. https://doi.org/10.1016/j.jslw.2004.04.005
Geng, J., & Razali, A. B. (2022). Effectiveness of the automated writing evaluation program on improving undergraduates' writing performance. English Language Teaching, 15(7), 49–58. https://doi.org/10.5539/elt.v15n7p49
Guo, Q., Feng, R., & Hua, Y. (2022). How effectively can EFL students use automated written corrective feedback (AWCF) in research writing?. Computer Assisted Language Learning, 35(9), 2312–2331. https://doi.org/10.1080/09588221.2021.1879161
Hamdani, M. H. (2025). Indonesian EFL students' writing challenges: A meta-analysis of affective, linguistic, and cognitive dimensions. Global Education Journal, 8(1), 45–73.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Hyland, K. (2003). Second language writing. Cambridge University Press.
Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational psychologist, 44(4), 250-266. https://doi.org/10.1080/00461520903213600
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44, 100450. https://doi.org/10.1016/j.asw.2020.100450
Lee, S.-M. (2020). The impact of using machine translation on EFL students' writing. Computer Assisted Language Learning, 33(3), 157–175. https://doi.org/10.1080/09588221.2018.1553186
Li, W. Y., Kau, K., & Shiung, Y. J. (2023). Pedagogic exploration into adapting automated writing evaluation and peer review integrated feedback into large-sized university writing classes. SAGE Open, 13(4). https://doi.org/10.1177/21582440231209087
Lo, N., Chan, S., & Wong, A. (2025). Evaluating teacher, AI, and hybrid feedback in English language learning: Impact on student motivation, quality, and performance in Hong Kong. SAGE Open, 15(3). https://doi.org/10.1177/21582440251352907
Ngo, T. T. N., Chen, H. H. J., & Lai, K. K. W. (2022). The effectiveness of automated writing evaluation in EFL/ESL writing: A three-level meta-analysis. Interactive Learning Environments. https://doi.org/10.1080/10494820.2022.2096642
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
Nickerson, C. (2005). English as a lingua franca in international business contexts. English for Specific Purposes, 24(4), 367–380. https://doi.org/10.1016/j.esp.2005.02.001
Ningrum, R. W. (2023). Academic writing difficulties for Indonesian postgraduate students: A phenomenological study. Journal of English Education, 8(4), 365–384.
Nugraeni, R. (2025). The correlation between self-regulated learning strategies and EFL students' writing performance. Indonesian EFL Journal, 11(1), 1–15.
Palermo, C. (2020). Implementing automated writing evaluation in different instructional contexts. Journal of Writing Research, 12(1), 1–32. https://doi.org/10.17239/jowr-2020.12.01.01
Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385–407. https://doi.org/10.1007/s10648-004-0006-x
Rahimi, M., & Fathi, J. (2022). Exploring the impact of wiki-mediated collaborative writing on EFL students' writing performance, writing self-regulation, and writing self-efficacy: A mixed methods study. Computer Assisted Language Learning, 35(1–2), 120–163. https://doi.org/10.1080/09588221.2019.1683700
Rahimi, M., Fathi, J., & Zou, D. (2025). Exploring the impact of automated written corrective feedback on the academic writing skills of EFL learners: An activity theory perspective. Education and Information Technologies, 30, 2691–2735. https://doi.org/10.1007/s10639-024-12896-5
Ranalli, J., & Link, S. (2020). Student perceptions of automated writing feedback and the role of teacher mediation. Journal of Writing Research, 12(1), 141–171. https://doi.org/10.17239/jowr-2020.12.01.05
Saricaoglu, A. (2019). The relationship between automated writing evaluation feedback uptake and text quality. System, 82, 1–10. https://doi.org/10.1016/j.system.2019.02.009
Sari, T. (2024). The impact of automated writing evaluation on EFL learners' writing self-efficacy, self-regulation, anxiety, and performance. Journal of Computer Assisted Learning, 40(5), 2065–2080. https://doi.org/10.1111/jcal.13004
Shang, H. F. (2019). Exploring online peer feedback and automated corrective feedback on EFL writing performance. Interactive Learning Environments. https://doi.org/10.1080/10494820.2019.1629601
Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65. https://doi.org/10.1016/j.asw.2013.11.007
Thi, N. K., & Nikolov, M. (2022). How teacher and Grammarly feedback complement one another in Myanmar EFL students' writing. Asia-Pacific Education Researcher, 31(6), 767–779. https://doi.org/10.1007/s40299-021-00625-2
Toba, R., Noor, W., & Sanu, L. (2019). The current issues of Indonesian EFL students' writing skills: Ability, problem, and reason in writing comparison and contrast essay. Dinamika Ilmu, 19(1), 57–73. https://doi.org/10.21093/di.v19i1.1506
Tsao, J. J. (2021). Effects of EFL learners' L2 writing self-efficacy on engagement with written corrective feedback. Asia-Pacific Education Researcher, 30(6), 575–584. https://doi.org/10.1007/s40299-021-00591-9
Ware, P. D. (2011). Computer-generated feedback on student writing. TESOL Quarterly, 45(4), 769–774. https://doi.org/10.5054/tq.2011.272525
Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10(2), 157–180. https://doi.org/10.1191/1362168806lr190oa
Wilson, J., & Roscoe, R. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764
Zhai, N., & Ma, X. (2023). The effectiveness of automated writing evaluation on writing quality: A meta-analysis. Journal of Educational Computing Research, 61(4), 875–900. https://doi.org/10.1177/07356331221127300
Zhang, Z. V., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.02.004
Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational Psychology, 25(1), 82–91. https://doi.org/10.1006/ceps.1999.1016
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Binar Winantaka, Anwar Efendi, Nur Hidayanto Pancoro Setyo Putro
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish in ELTEJ agree to the following terms: Authors retain copyright and grant the ELTEJ right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share (copy and redistribute the material in any medium or format) and adapt (remix, transform, and build upon the material) the work for any purpose, even commercially with an acknowledgement of the work's authorship and initial publication in ELTEJ. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in ELTEJ. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
