Development and Calibration of a Moodle-Based Computerized Adaptive Test Using Rasch Model for Electrical Competencies in Vocational Schools
DOI:
https://doi.org/10.12928/joves.v9i2.14405Keywords:
Development, Computerized Adaptive Test, Evaluation, 4D Model, Rasch Model, Moodle, Vocational SchoolAbstract
This study developed a Moodle-based Computerized Adaptive Test (CAT) to assess vocational students’ competencies in electrical installation and troubleshooting. Conventional assessments often lack efficiency, security, and personalized feedback, motivating the development of an adaptive evaluation system. Using the 4D research and development model, 150 test items were created and calibrated with the Rasch Model. After removing misfit and overfit items, 106 items were confirmed valid and reliable (reliability = 0.96). The instrument provides a balanced difficulty distribution, enabling accurate measurement across different ability levels. Expert validation rated the content as “Highly Feasible” (94.75%) and the media as “Feasible” (89.75%), while student responses showed positive acceptance (76%–80%). Implementation demonstrated that the CAT effectively differentiates student abilities and provides immediate diagnostic feedback for formative assessment. The study offers an adaptive, Moodle-integrated assessment tool and recommends expanding the item bank to better measure higher-order thinking skills.
References
Daftar pustaka di atas memiliki 31 referensi.
Berikut adalah daftar pustaka tersebut yang telah dirapikan dan ditambahkan spasi (enter) di antara setiap referensinya agar langsung bisa Anda salin:
Akhtar, H., & Kovacs, K. (2024). Measurement precision and user experience with adaptive versus non-adaptive psychometric tests. Personality and Individual Differences, 225, 112675. https://doi.org/10.1016/j.paid.2024.112675
Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences (4th ed.). Routledge.
Badan Pusat Statistik. (2024, November 5). Tingkat pengangguran terbuka (TPT) sebesar 4,91 persen dan rata-rata upah buruh sebesar 3,27 juta rupiah per bulan. https://www.bps.go.id/id/pressrelease/2024/11/05/2373/unemployment-rate-was-4-91-percent-.html
Corral, G., et al. (2021). Immediate feedback in a flipped classroom improves learning. Journal of Medical Education and Curricular Development, 8. https://doi.org/10.1177/23821205211029280
de Ayala, R. J. (2022). The theory and practice of item response theory (2nd ed.). Guilford Press.
Ebenbeck, N., Bastian, M., Mühling, A., & Gebhardt, M. (2024). Duration versus accuracy: What matters for computerised adaptive testing in schools? Journal of Computer Assisted Learning, 40(6), 3443–3453. https://doi.org/10.1111/jcal.13074
Fahrurrizal, F., Marwan, M., & Bahri, S. (2024). Efektivitas pengelolaan pemberian tunjangan profesi dalam optimalisasi kinerja profesional guru. Research and Development Journal of Education, 10(2), 725–735. https://doi.org/10.30998/rdje.v10i2.25030
Falani, I. (2023). Desain dan validasi aplikasi tes literasi matematika berbasis komputer dengan pendekatan item response theory. Jurnal Pendidikan Matematika, 12(2).
Hadi, S. (2013). Pengembangan computerized adaptive test berbasis web. Aswaja Pressindo.
Hadi, S., Haryanto, H., AM, M. A., Marlina, M., & Rahim, A. (2022). Developing classroom assessment tool using learning management system-based computerized adaptive test in vocational high schools. Journal of Education Research and Evaluation, 6(1), 143–155. https://doi.org/10.23887/jere.v6i1.35630
Hizqiyah, I. Y. N., Widodo, A., Sriyati, S., & Ahmad, A. (2023). Development of a digital problem solving skills test instrument: Rasch model analysis. Jurnal Penelitian Pendidikan IPA, 9(4), 1658–1663. https://doi.org/10.29303/jppipa.v9i4.2671
Kementerian Energi dan Sumber Daya Mineral Republik Indonesia. (2019). Tenaga teknik ketenagalistrikan bersertifikat beri rasa aman menggunakan listrik. https://www.esdm.go.id/id/media-center/arsip-berita/tenaga-teknik-ketenagalistrikan-bersertifikat-beri-rasa-aman-menggunakan-listrik
Luo, H., & Yang, X. (2024). Efficiency of computerized adaptive testing with a cognitively designed item bank. Frontiers in Psychology, 15. https://doi.org/10.3389/fpsyg.2024.1353419
Mao, X., Zhang, J., & Xin, T. (2022). The optimal design of bifactor multidimensional computerized adaptive testing with mixed-format items. Applied Psychological Measurement, 46(7), 605–621. https://doi.org/10.1177/01466216221108382
Nadia, Kamila, V. Z., Islamiyah, Wardhana, R., & Setyadi, H. J. (2022). Penerapan computerized adaptive test (CAT) pada ujian semester menggunakan algoritma teori respon butir model 2PL berbasis web (Studi kasus: Kelas XI SMAN 1 Muara Jawa Kota Samarinda). Adopsi Teknologi dan Sistem Informasi (ATASI), 1(1), 36–51. https://doi.org/10.30872/atasi.v1i1.43
Nurdiansyah, A. I., Sriyono, & Eko, S. K. (2020). Pengembangan program computerized adaptive test hasil belajar fisika SMK. Jurnal Inovasi Pendidikan Sains, 1(1).
Oladele, J. I. (2025). Computerised adaptive method for assessing university undergraduates' mental well-being within an African context: An open-source set-up with Concerto. MethodsX, 14, 103140. https://doi.org/10.1016/j.mex.2024.103140
Pan, X. (2022). Exploring the multidimensional relationships between educational situation perception, teacher support, online learning engagement, and academic self-efficacy in technology-based language learning. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1000069
Rahim, A., Hadi, S., Marlina, M., Susilowati, D., Muti'ah, M., & Irhas, I. (2024). Transfigurations of the computerized mathematics final exam: How was the results of computerized adaptive test-based assessment? Jurnal Pendidikan Matematika dan IPA, 15(2), 257–274. https://doi.org/10.26418/jpmipa.v15i2.69485
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Muñoz-Rujas, N., Rodríguez-Arribas, S., Escolar-Llamazares, M. C., Alonso-Santander, N., Martínez-Martín, M. Á., & Mercado-Val, E. I. (2021). Teaching and learning styles on Moodle: An analysis of the effectiveness of using STEM and non-STEM qualifications from a gender perspective. Sustainability, 13(3). https://doi.org/10.3390/su13031166
Tananda, O., Nasir, M., Milana, M., & Muslim, M. (2023). Perbandingan antara sistem computer based test dan paper based test pada hasil pembelajaran mata pelajaran PMKR di SMKN 1 Sumatera Barat. JTPVI: Jurnal Teknologi dan Pendidikan Vokasi Indonesia, 1(1), 67–74. https://doi.org/10.24036/jtpvi.v1i1.10
Thompson, N. A., & Weiss, D. J. (2021). A history of computerized adaptive testing. In B. P. Veldkamp & C. Sluijter (Eds.), Theoretical and practical advances in computer-based educational measurement (pp. 3–33). Springer. https://doi.org/10.1007/978-3-030-43485-4_1
Veldkamp, B. P., & Sluijter, C. (Eds.). (2021). Theoretical and practical advances in computer-based educational measurement. Springer.
Wang, Z., Wang, C., & Weiss, D. J. (2022). Termination criteria for grid multiclassification adaptive testing with multidimensional polytomous items. Applied Psychological Measurement, 46(7), 551–570. https://doi.org/10.1177/01466216221108383
Wijaya, W. M., Syarifah, L. S., & David, T. (2024). Enhancing e-learning in vocational schools: Key characteristics of instructional design. Jurnal Akuntabilitas Manajemen Pendidikan, 12(1), 68–78. https://doi.org/10.21831/jamp.v12i1.75984
Wulandari, F., Hadi, S., & Haryanto, H. (2020). Computer-based adaptive test development using fuzzy item response theory to estimate student ability. Computer Science and Information Technology, 8(3), 66–73. https://doi.org/10.13189/csit.2020.080302
Wyse, A. E. (2021). How days between tests impacts alternate forms reliability in computerized adaptive tests. Educational and Psychological Measurement, 81(4), 644–667. https://doi.org/10.1177/0013164420979656
Xi, C., Tu, D., & Cai, Y. (2022). Dual-objective item selection methods in computerized adaptive test using the higher-order cognitive diagnostic models. Applied Psychological Measurement, 46(5), 422–438. https://doi.org/10.1177/01466216221089342
Xiao, J., & Bulut, O. (2022). Item selection with collaborative filtering in on-the-fly multistage adaptive testing. Applied Psychological Measurement, 46(8), 690–704. https://doi.org/10.1177/01466216221124089
Yang, A. C. M., Flanagan, B., & Ogata, H. (2022). Adaptive formative assessment system based on computerized adaptive testing and the learning memory cycle for personalized learning. Computers and Education: Artificial Intelligence, 3. https://doi.org/10.1016/j.caeai.2022.100104
Yuniarti, N., Hariyanto, & Zamtinah. (2024). Media pembelajaran vokasional. UNY Press.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Universitas Ahmad Dahlan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with Journal of Vocational Education Studies (JOVES) agree to the following terms: Authors retain the copyright and grant the Universitas Ahmad Dahlan right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share (copy and redistribute the material in any medium or format) and adapt (remix, transform, and build upon the material) the work for any purpose, even commercially with an acknowledgement of the work's authorship and initial publication in Universitas Ahmad Dahlan. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in Universitas Ahmad Dahlan. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
.png)
