Rasch Analysis of the Force and Motion Conceptual Evaluation Test: Validity and Reliability in Measuring Force and Motion Understanding of Students
Abstract
Understanding Force and Motion is fundamental in physics education as it forms the basis for more complex concepts and has practical applications in various fields, from engineering to everyday problem-solving. This paper aims at assessing the quality of test questions using the Rasch model to gauge students' understanding of Force and Motion within physics education. The significance of accurately assessing these concepts cannot be overstated, as it ensures that students have a solid foundation for future learning. Adopting a descriptive qualitative approach, the research employed the FMCE (Force and Motion Conceptual Evaluation) test instrument alongside Rasch modelling. The study involved 35 high school students who had covered the Force and Motion curriculum. Analysis with Winstep software (Version 3.65.0) revealed that items 1 and 7 were invalid. The instrument demonstrated commendable reliability, with an item reliability of 0.73. Difficulty level analysis identified five questions as outliers, categorised as either very difficult or very easy. The discrimination analysis confirmed that the instrument effectively differentiated between students who answered correctly and those who did not. Overall, the FMCE exhibited solid validity and high reliability, although some items necessitate revision. The study's limitations, particularly the small sample size, may affect the generalisability of the findings. Despite these limitations, the study provides valuable insights into the assessment of Force and Motion concepts in high school students, though caution is advised when interpreting the results. Future research should consider a larger sample size and diverse educational contexts to enhance the robustness and applicability of the findings.
Keywords: Concept understanding, force and motion, high school students, instrument test, Rasch model
Full Text:
PDFReferences
Auna Hidayati, R., & Taqwa, M. R. A. (2023). Computer based recitation program development with feed back in Newton’s Law topic. International Journal of Education and Teaching Zone, 2(1), 113–129. https://doi.org/10.57092/ijetz.v2i1.49
Austvoll-Dahlgren, A., Semakula, D., Nsangi, A., Oxman, A. D., Chalmers, I., Rosenbaum, S., & Guttersrud, Ø. (2017). Measuring ability to assess claims about treatment effects: the development of the ‘Claim Evaluation Tools.’ BMJ Open, 7(5), e013184. https://doi.org/10.1136/bmjopen-2016-013184
Bond, T. G., & Fox, C. M. (2019). Applying the Rasch model: Fundamental measurement in the human sciences (4th ed.). Routledge.
Börkan, B. (2017). Exploring variability sources in student evaluation of teaching via many-facet Rasch Model. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 15–15. https://doi.org/10.21031/epod.298462
Black, P., & Wiliam, D. (2020). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 32(3), 171-195. https://doi.org/10.1007/s11092-020-09367-x
Browne, M. W., & Schreiber, J. B. (2020). The impact of conceptual understanding on problem-solving in physics education: A review. International Journal of Science Education, 42(9), 1427-1445. https://doi:10.1080/09500693.2020.1754557
Creswell, J. W. (2018). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th Ed.). University of Nebraska–Lincoln.
Czerniak, C. M., & Johnson, J. A. (2020). Addressing misconceptions in physics: A focus on motion and force. Journal of Research in Science Teaching, 57(9), 1425-1443. https://doi:10.1002/tea.21624
Didik, L. A., Wahyudi, Muh., & Kafrawi, M. (2020). Identifikasi miskonsepsi dan tingkat pemahaman mahasiswa tadris fisika pada materi listrik dinamis menggunakan 3-tier diagnostic test. Journal of Natural Science and Integration, 3(2), 128. https://doi.org/10.24014/jnsi.v3i2.9911
Ding, L., & Beichner, R. J. (2021). Student understanding of force and motion: The role of integrating knowledge. Journal of Research in Science Teaching, 58(7), 1047-1069. https://doi:10.1002/tea.21612
Eerman A., Karakaya, I., & Sata, M. (2020). Evaluation of university students’ rating behaviors in self and peer rating process via many facet Rasch Model. Eurasian Journal of Educational Research, 20(89), 1–22. https://doi.org/10.14689/ejer.2020.89.2
Finkelstein, N. D., & Leong, C. (2021). Conceptual difficulties and misconceptions in high school physics: Evidence from assessment and instruction. Physics Education Research Conference Proceedings, 2021, 207-212. https://doi:10.1119/perc.2021.pr.027
Hake, R. R., & Gibbons, A. S. (2020). Interactive engagement vs. traditional methods: How physics education research informs practice. American Journal of Physics, 88(1), 30-38. https://doi:10.1119/1.5123476
Hattie, J., & Timperley, H. (2020). The power of feedback for learning: Implications for assessment. Review of Educational Research, 90(4), 567-594. https://doi.org/10.3102/0034654319872322
Ismunandar, D., Rosyadi, R., Nurafifah, L., & Jofre, A. (2022). Improving students’ problem solving abilities through the application of auditory intellectually repetition model. Journal of Instructional Mathematics, 3(2), 53–60. https://doi.org/10.37640/jim.v3i2.1385
Isnani, I., Utami, W. B., Susongko, P., & Lestiani, H. T. (2019). Estimation of college students’ ability on real analysis course using Rasch model. REID (Research and Evaluation in Education), 5(2), 95–102. https://doi.org/10.21831/reid.v5i2.20924
Kali, Y., & Ronen, M. (2019). Developing and evaluating assessments for conceptual understanding in science education. Research in Science Education, 49(3), 991-1010. https://doi.org/10.1007/s11165-018-9733-2
Kadir Masalesi, A. (2022). Resource identification and level of understanding of particle dynamics concepts. International Journal of Education and Teaching Zone, 1(2), 204–221. https://doi.org/10.57092/ijetz.v1i2.48
McDermott, L. C., & Redish, E. F. (2021). Resource letter: PER-1: Physics education research. American Journal of Physics, 89(6), 589-600. https://doi:10.1119/10.0002981
Mestre, J. P., Docktor, J. L., Strand, N. E., & Ross, B. H. (2011). Conceptual problem solving in physics (pp. 269–298). https://doi.org/10.1016/B978-0-12-387691-1.00009-0
Meyer, J. P., & Timm, W. R. (2021). Rasch analysis in education: Theory and practice. Educational Measurement: Issues and Practice, 40(1), 15-29. https://doi.org/10.1111/emip.12324
Miller, K. L., & Kopp, M. D. (2020). Evaluating conceptual understanding in STEM education: The role of formative assessment. Journal of Educational Measurement, 57(2), 159-177. https://doi.org/10.1111/jedm.12236
Misbah, M., Trisnowati, E., Rahim, A., & Zb, A. (2022). Investigating problem solving and mathematical connections in solving the fermi-dirac equation. International Journal of Education and Teaching Zone, 1(2), 106–115. https://doi.org/10.57092/ijetz.v1i2.36
Mursidi, A., & Soeharto, S. (2017). An introduction: Evaluation of quality assurance for higher educational institutions using rasch model. JETL (Journal of Education, Teaching and Learning), 1(1), 1. https://doi.org/10.26737/jetl.v1i1.25
Nehru, N., Ananda, R., Zb, A., & Navalyyan, D. (2022). Analysis of mathematical critical thinking ability and mathematical creativity: Judging from the process of deriving the Fermi-Dirac Formula. International Journal of Education and Teaching Zone, 1(2), 87–96. https://doi.org/10.57092/ijetz.v1i2.33
Palimbong, J., Mujasam, M., & Allo, A. Y. T. (2019). Item analysis using Rasch model in semester final exam evaluation study subject in physics Class X TKJ SMK Negeri 2 Manokwari. Kasuari: Physics Education Journal (KPEJ), 1(1), 43–51. https://doi.org/10.37891/kpej.v1i1.40
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 20111. https://doi.org/10.1103/PhysRevPhysEducRes.15.020111
Rasmuin, R., & Luddin, S. (2022). Tingkat kesulitan soal buatan guru bidang studi matematika menurut teori tes klasik pada tingkat SMP di Kota Baubau. Jurnal Akademik Pendidikan Matematika, 33–40. https://doi.org/10.55340/japm.v8i1.699
Ryan, Q. X., Frodermann, E., Heller, K., Hsu, L., & Mason, A. (2016). Computer problem-solving coaches for introductory physics: Design and usability studies. Physical Review Physics Education Research, 12(1), 010105. https://doi.org/10.1103/PhysRevPhysEducRes.12.010105
Singh, C., & Chabay, R. (2022). Understanding and overcoming misconceptions in physics: A focus on motion and force. Physical Review Physics Education Research, 18(2), 020102. https://doi:10.1103/PhysRevPhysEducRes.18.020102
Sulman, F., Sutopo, S., & Kusairi, S. (2021). FMCE-PHQ-9 assessment with Rasch model in detecting concept understanding, cheating, and depression amid the covid-19 pandemic. Tadris: Jurnal Keguruan Dan Ilmu Tarbiyah, 6(2), 297–309. https://doi.org/10.24042/tadris.v6i2.9273
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan (1st ed.). Trim Komunikata.
Suranata, K., Dharsana, I. K., Paramartha, W. E., Dwiarwati, K. A., & Ifdil, I. (2021). Pengaruh model strength-based counseling dalam LMS Schoology untuk meningkatkan kesehatan mental dan kesejahteraan psikologis siswa. JPPI (Jurnal Penelitian Pendidikan Indonesia), 7(1), 137–146. https://doi.org/10.29210/020211142
Susongko, P. (2016). Validation of science achievement test with the Rasch model. Jurnal Pendidikan IPA Indonesia, 5(2), 268–277. https://doi.org/10.15294/jpii.v5i2.7690
Tornabene, R. E., Lavington, E., & Nehm, R. H. (2018). Testing validity inferences for Genetic Drift Inventory scores using Rasch modeling and item order analyses. Evolution: Education and Outreach, 11(1), 6. https://doi.org/10.1186/s12052-018-0082-x
Wang, J., & Wang, Y. (2021). Improving conceptual understanding through diagnostic assessment and feedback. Assessment & Evaluation in Higher Education, 46(4), 533-547. https://doi.org/10.1080/02602938.2020.1798754
Yusuf, I., Zb, A., & Rozal, E. (2022). Understanding mathematical communication concepts and skills: analysis of the ability of prospective physics teachers? International Journal of Education and Teaching Zone, 1(2), 97–105. https://doi.org/10.57092/ijetz.v1i2.34
Zhu, Y., & Dong, Y. (2019). Misconceptions in elementary and high school students' understanding of motion and force: A review. International Journal of Science Education, 41(6), 732-750.
Refbacks
- There are currently no refbacks.
Copyright (c) 2024
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright of the article is reserved by the author(s). Published by the Physics Education Study Program, Faculty of Teacher Training and Education, Universitas Lampung in Collaboration with the Physical Society of Indonesia (since 2019). This article is an open-access article under the Creative Commons non-commercial-share-alike 4.0 International License (CC BY-NC-SA 4.0) license.
