Students are Not Sure about Their Conceptual Understanding: A Comparative Study of the Level of Conceptual Understanding and the Level of Confidence Using Rasch Modeling

Toni Kus Indratno; Moh. Irma Sukarelawan; Ariati Dina  Puspitasari; Soeharto Soeharto

doi:10.12928/irip.v6i2.6901

Authors

Toni Kus Indratno Universitas Ahmad Dahlan https://orcid.org/0000-0001-8852-6231
Moh. Irma Sukarelawan Universitas Ahmad Dahlan http://orcid.org/0000-0002-3823-3964
Ariati Dina Puspitasari Universitas Ahmad Dahlan https://orcid.org/0000-0002-3127-5942
Soeharto Soeharto University of Szeged https://orcid.org/0000-0003-4332-7401

DOI:

https://doi.org/10.12928/irip.v6i2.6901

Keywords:

Conceptual Understanding, Four-tier, Misconception, Rasch Model

Abstract

Misconceiving fundamental physics concepts is a persistent challenge in education, hindering students' ability to grasp more advanced topics. This study aims to compare students' conceptual understanding and confidence levels using a 4-tier diagnostic instrument. Conducted as a survey involving 56 high school students in Yogyakarta, the research utilized a 4-tier diagnostic test covering topics of temperature and heat. Data were collected through Google Forms and analyzed using Excel and Winsteps 4.6.1 based on the Rasch Model. Data analysis involved coding, preparing raw data, and utilizing Wright maps combined with Logit Value of Person (LVP) for assessing students' conceptual understanding and confidence levels. Results indicated a significant inconsistency between students' conceptual understanding and their confidence, with many students displaying high confidence despite low understanding, and vice versa. This mismatch highlights the necessity for educational strategies that not only clarify concepts but also align students' confidence with their actual understanding. The study's implications suggest the need for more comprehensive diagnostic tools and targeted pedagogical interventions to enhance learning outcomes in physics education.

Author Biographies

Toni Kus Indratno, Universitas Ahmad Dahlan

Department of Physics Education, Faculty of Teacher Training and Education, Universitas Ahmad Dahlan, Indonesia

Moh. Irma Sukarelawan, Universitas Ahmad Dahlan

Department of Physics Education, Faculty of Teacher Training and Education, Universitas Ahmad Dahlan, Indonesia

Ariati Dina Puspitasari, Universitas Ahmad Dahlan

Department of Physics Education, Faculty of Teacher Training and Education, Universitas Ahmad Dahlan, Indonesia

Soeharto Soeharto, University of Szeged

National Research and Innovation Agency (BRIN), Jakarta, Indonesia

Research Center of Educational Technologies, Azerbaijan State University of Economics, Baku, Azerbaijan

Doctoral School of Education, University of Szeged, 32-34, Petofi S. Sgt., Szeged, H-6722, Hungary

References

S. A. Kiray and S. Simsek, “Determination and Evaluation of the Science Teacher Candidates’ Misconceptions About Density by Using Four-Tier Diagnostic Test,” Int. J. Sci. Math. Educ., vol. 19, no. 5, pp. 935–955, Jun. 2021, https://doi.org/10.1007/s10763-020-10087-5.

M. I. Sukarelawan, S. Sriyanto, A. D. Puspitasari, D. Sulisworo, and U. N. Hikmah, “Four-Tier Heat and Temperature Diagnostic Test (4T-HTDT) to Identify Student Misconceptions,” JIPFRI (Jurnal Inov. Pendidik. Fis. dan Ris. Ilmiah), vol. 5, no. 1, pp. 1–8, Jul. 2021, https://doi.org/10.30599/jipfri.v5i1.856.

M. R. Luce and M. A. Callanan, “Family Conversations About Heat and Temperature: Implications for Children’s Learning,” Front. Psychol., vol. 11, pp. 1–18, Aug. 2020, https://doi.org/10.3389/fpsyg.2020.01718.

Z. N. Fadhilatullathifi, B. Ardiyanto, D. D. Rahayu, T. Almukholani, I. Rinayah, and F. Rahmawati, “Four-Tier Diagnostic Test Method to Identify Conceptual Understanding in Calculus,” J. Phys. Conf. Ser., vol. 1613, no. 1, p. 012075, Aug. 2020, https://doi.org/10.1088/1742-6596/1613/1/012075.

S. Mubarak and Y. Yahdi, “Identifying Undergraduate Students’ Misconceptions in Understanding Acid Base Materials,” J. Pendidik. IPA Indones., vol. 9, no. 2, pp. 276–286, Jun. 2020, https://doi.org/10.15294/jpii.v9i2.23193.

M. I. Sukarelawan, J. Jumadi, and N. A. Rahman, “An Analysis of Graduate Students’ Conceptual Understanding in Heat and Temperature (H&T) Using Three-Tier Diagnostic Test,” Indones. Rev. Phys., vol. 2, no. 1, pp. 9–14, Jun. 2019, https://doi.org/10.12928/irip.v2i1.910.

M. I. Sukarelawan, A. D. Puspitasari, D. Sulisworo, H. Kuswanto, and J. Jumadi, “A Shift in the Conceptual Understanding of Physics Students Through the Wright Map,” J. Pendidik. Dan Pengajaran, vol. 55, no. 1, pp. 127–141, 2022, https://doi.org/10.23887/jpp.v55i1.38342127A.

Y. J. Jafer, “Assessing Kuwaiti Pre-service Science Teachers’ Greenhouse Effect Perceptions and Misconceptions,” Int. J. Sci. Math. Educ., vol. 18, no. 4, pp. 657–667, Apr. 2020, https://doi.org/10.1007/s10763-019-09992-1.

G. Stylos, A. Sargioti, D. Mavridis, and K. T. Kotsis, “Validation of the thermal concept evaluation test for Greek university students’ misconceptions of thermal concepts,” Int. J. Sci. Educ., vol. 43, no. 2, pp. 247–273, Jan. 2021, https://doi.org/10.1080/09500693.2020.1865587.

E. V. Slater, J. E. Morris, and D. McKinnon, “Astronomy alternative conceptions in pre-adolescent students in Western Australia,” Int. J. Sci. Educ., vol. 40, no. 17, pp. 2158–2180, Nov. 2018, https://doi.org/10.1080/09500693.2018.1522014.

K. A. Bentaleb, S. Dachraoui, T. Hassouni, E. mehdi Alibrahmi, E. Chakir, and A. Belboukhari, “Development of a Survey to Assess Conceptual Understanding of Quantum Mechanics among Moroccan Undergraduates,” Eur. J. Educ. Res., vol. 11, no. 4, pp. 2219–2243, Oct. 2022, https://doi.org/10.12973/eu-jer.11.4.2219.

A. H. Aminudin, İ. Kaniawati, E. Suhendi, A. Samsudin, B. Coştu, and R. Adimayuda, “Rasch Analysis of Multitier Open-ended Light-Wave Instrument (MOLWI): Developing and Assessing Second-Years Sundanese-Scholars Alternative Conceptions,” J. Educ. Gift. Young Sci., vol. 7, no. 3, pp. 557–579, Sep. 2019, https://doi.org/10.17478/jegys.574524.

I. Aykutlu, Ö. Ensari, and C. Bayrak, “Prospective teachers’ conceptual understanding of the polarization of light,” Eur. J. Phys., vol. 44, no. 1, p. 015701, Jan. 2023, https://doi.org/10.1088/1361-6404/ac93cc.

A. Hrnjičić, A. Alihodžić, F. Čunjalo, and D. Kamber Hamzić, “Development of an Item Bank for Measuring Students’ Conceptual Understanding of Real Functions,” Eur. J. Sci. Math. Educ., vol. 10, no. 4, pp. 455–470, Jul. 2022, https://doi.org/10.30935/scimath/12222.

X.-M. Wang, S.-M. Wang, J.-N. Wang, G.-J. Hwang, and S. Xu, “Effects of a Two-Tier Test Strategy on Students’ Digital Game-Based Learning Performances and Flow Experience in Environmental Education,” J. Educ. Comput. Res., vol. 60, no. 8, pp. 1942–1968, Jan. 2023, https://doi.org/10.1177/07356331221095162.

M. Nasir, C. Cari, W. Sunarno, and F. Rahmawati, “The Development of a Two-Tier Diagnostic Test for Student Understanding of Light,” J. Turkish Sci. Educ., vol. 20, no. 1, pp. 150–172, Mar. 2023, https://doi.org/10.36681/tused.2023.009.

W. Widodo, M. A. Mahdiannur, S. Suryanti, and N. L. Choirunnisa, “Mobile Interactive Multimedia to Assist Prospective Science Teachers Holding Conceptual Understanding in Problem-Solving Electrical Circuits,” TEM J., vol. 12, no. 4, pp. 2251–2263, Nov. 2023, https://doi.org/10.18421/TEM124-36.

S. Wahyuni and T. P. Saputra, “Identification and remediation of misconceptions using three-tier diagnostic test based on CRI and literacy worksheet,” 2023, p. 050061. https://doi.org/10.1063/5.0125856.

L. A. R. Laliyo, B. Sumintono, and C. Panigoro, “Measuring changes in hydrolysis concept of students taught by inquiry model: stacking and racking analysis techniques in Rasch model,” Heliyon, vol. 8, no. 3, p. e09126, Mar. 2022, https://doi.org/10.1016/j.heliyon.2022.e09126.

J. Jumadi, M. I. Sukarelawan, and H. Kuswanto, “An investigation of item bias in the four-tier diagnostic test using Rasch model,” Int. J. Eval. Res. Educ., vol. 12, no. 2, p. 622, Jun. 2023, https://doi.org/10.11591/ijere.v12i2.22845.

N. Onder-Celikkanli and M. Tan, “Determining Turkish high school students’ misconceptions about electric charge imbalance by using a four-tier misconception test,” Phys. Educ., vol. 57, no. 5, p. 055010, Sep. 2022, https://doi.org/10.1088/1361-6552/ac68c1.

D. Kaltakci-Gurel, A. Eryilmaz, and L. C. McDermott, “Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics,” Res. Sci. Technol. Educ., vol. 35, no. 2, pp. 238–260, Apr. 2017, https://doi.org/10.1080/02635143.2017.1310094.

D. Kaltakci Gurel, A. Eryilmaz, and L. C. McDermott, “A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science,” EURASIA J. Math. Sci. Technol. Educ., vol. 11, no. 5, pp. 989–1008, Oct. 2015, https://doi.org/10.12973/eurasia.2015.1369a.

Soeharto, B. Csapó, E. Sarimanah, F. I. Dewi, and T. Sabri, “A Review of Students’ Common Misconceptions in Science and Their Diagnostic Assessment Tools,” J. Pendidik. IPA Indones., vol. 8, no. 2, pp. 247–266, Jun. 2019, https://doi.org/10.15294/jpii.v8i2.18649.

M. I. Sukarelawa, J. Jumadi, and H. Kuswanto, “Pengembangan Pembelajaran Mobile Remedial System (MRS) Untuk Meningkatkan Kemampuan Metakognisi dan Mereduksi Miskonsepsi Fisika pada Materi Suhu dan Kalor,” Universitas Negeri Yogyakarta, 2023.

D. Kaltakci-Gurel, A. Eryilmaz, and L. C. McDermott, “Identifying pre-service physics teachers’ misconceptions and conceptual difficulties about geometrical optics,” Eur. J. Phys., vol. 37, no. 4, p. 045705, Jul. 2016, https://doi.org/10.1088/0143-0807/37/4/045705.

R. W. Emerson, “Likert Scales,” J. Vis. Impair. Blind., vol. 111, no. 5, pp. 488–488, Sep. 2017, https://doi.org/10.1177/0145482X1711100511.

S. Jamieson, “Likert scales: how to (ab)use them,” Med. Educ., vol. 38, no. 12, pp. 1217–1218, Dec. 2004, https://doi.org/10.1111/j.1365-2929.2004.02012.x.

S. K. Johal and M. Rhemtulla, “Comparing estimation methods for psychometric networks with ordinal data.,” Psychol. Methods, vol. 28, no. 6, pp. 1251–1272, Dec. 2023, https://doi.org/10.1037/met0000449.

P. A. Bishop and R. L. Herron, “Use and Misuse of the Likert Item Responses and Other Ordinal Measures.,” Int. J. Exerc. Sci., vol. 8, no. 3, pp. 297–302, 2015, [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/27182418

A. R. Suparman, E. Rohaeti, and S. Wening, “Development of Computer-Based Chemical Five-Tier Diagnostic Test Instruments: A Generalized Partial Credit Model,” J. Effic. Responsib. Educ. Sci., vol. 17, no. 1, pp. 92–106, Mar. 2024, https://doi.org/10.7160/eriesj.2024.170108.

E. Istiyono, W. S. B. Dwandaru, K. Fenditasari, M. R. S. S. N. Ayub, and D. Saepuzaman, “The Development of a Four-Tier Diagnostic Test Based on Modern Test Theory in Physics Education,” Eur. J. Educ. Res., vol. 12, no. 1, pp. 371–385, Jan. 2023, https://doi.org/10.12973/eu-jer.12.1.371.

M. Irmak, H. İnaltun, J. Ercan-Dursun, H. Yaniş-Kelleci, and N. Yürük, “Development and Application of a Three-Tier Diagnostic Test to Assess Pre-service Science Teachers’ Understanding on Work-Power and Energy Concepts,” Int. J. Sci. Math. Educ., vol. 21, no. 1, pp. 159–185, Jan. 2023, https://doi.org/10.1007/s10763-021-10242-6.

A. C. Atmaca Aksoy and S. Erten, “A Four-Tier Diagnostic Test to Determine Pre-Service Science Teachers’ Misconception about Global Warming,” J. Balt. Sci. Educ., vol. 21, no. 5, pp. 747–761, Oct. 2022, https://doi.org/10.33225/jbse/22.21.747.

H. Habiddin and E. M. Page, “Uncovering Students’ Genuine Misconceptions: Evidence to Inform the Teaching of Chemical Kinetics,” Acta Chim. Slov., vol. 70, no. 2, pp. 184–195, Jun. 2023, https://doi.org/10.17344/acsi.2022.7880.

S. Mi, J. Ye, L. Yan, and H. Bi, “Development and validation of a conceptual survey instrument to evaluate senior high school students’ understanding of electrostatics,” Phys. Rev. Phys. Educ. Res., vol. 19, no. 1, p. 010114, Feb. 2023, https://doi.org/10.1103/PhysRevPhysEducRes.19.010114.

K. B. Putica, “Development and Validation of a Four-Tier Test for the Assessment of Secondary School Students’ Conceptual Understanding of Amino Acids, Proteins, and Enzymes,” Res. Sci. Educ., vol. 53, no. 3, pp. 651–668, Jun. 2023, https://doi.org/10.1007/s11165-022-10075-5.

Y. K. Yan and R. Subramaniam, “Using a multi-tier diagnostic test to explore the nature of students’ alternative conceptions on reaction kinetics,” Chem. Educ. Res. Pract., vol. 19, no. 1, pp. 213–226, 2018, https://doi.org/10.1039/C7RP00143F.

B. Sumintono and W. Widhiarso, Aplikasi model rasch untuk penelitian ilmu-ilmu sosial. Cimahi: Trim Komunikata Publishing House, 2014.

J. M. Linacre, “Winsteps® (Version 4.6.1) [Computer Software].” 2021. [Online]. Available: http://www.winsteps.com

M. Ramdhan, M. I. Sukarelawan, M. A. Thohir, and F. Arifiyanti, “Junior high school student perception of online learning in pandemic Covid-19: Gender, social media ownership, and internet access duration perspective,” Int. J. Educ. Learn., vol. 4, no. 1, pp. 28–40, Apr. 2022, https://doi.org/10.31763/ijele.v4i1.517.

M. I. Sukarelawan, E. Gustina, S. M. Ayu, L. Sofiana, and S. Sriyanto, “Workshop penelitian alternatif di masa pandemik covid-19 bagi guru-guru SMA/SMK,” in Peran Perguruan Tinggi (PT) dalam Meningkatkan Kapastias Masyarakat di Era Pandemi, 2021, pp. 1538–1544. [Online]. Available: http://seminar.uad.ac.id/index.php/senimas/article/view/7542

D. Adams, K. M. Chuah, B. Sumintono, and A. Mohamed, “Students’ readiness for e-learning during the COVID-19 pandemic in a South-East Asian university: a Rasch analysis,” Asian Educ. Dev. Stud., vol. 11, no. 2, pp. 324–339, Mar. 2022, https://doi.org/10.1108/AEDS-05-2020-0100.

D. Adams, M. H. J. Tan, B. Sumintono, and S. P. Oh, “Blended Learning Engagement in Public and Private Higher Education Institutions: A Differential Item Functioning Analysis of Students’ Backgrounds,” Malaysian J. Learn. Instr., vol. 17, no. 1, pp. 133–158, 2020, https://doi.org/10.32890/mjli2020.17.1.6.