Development and Validation of the Relativity Concept Inventory Test Using Item Response Theory Generalized Partial Credit Model 3PL
Abstract
Abstract: This research aims to develop a valid and reliable Relativity Concept Inventory Test instrument. This instrument is based on relativity material, which includes Einstein's first and second postulates, time dilation, velocity addition, and length contraction. Methods for preparing instruments include 1) test design, 2) test validation, 3) test trials, and 4) test data analysis. The design of the test grid is based on Bloom's taxonomy C2 to C5 and produces 13 questions. The instrument is made in the form of multiple-choice questions and is equipped with a level of confidence. Instrument validation was carried out by 6 physics education lecturers and 1 high school teacher, with analysis using the V Aiken formula. The validated instrument was then tested on 130 students from 2 high schools in Madiun. Trial data was analyzed using the Generalized Partial Credit Model 3PL (GPCM-3PL). The development results show that: 1) 13 multiple choice Relativity Concept Inventory Test questions with a level of confidence were successfully developed, 2) expert validation showed that all question items got a score of 0.93, which is included in the valid, with instrument reliability of 0, 42 (very low category), 3) the results of the trial test showed that the relativity concept inventory was proven to be fit with the GPCM, with different power of the items of 0.407 and the level of difficulty showed that 11 items were valid with a range of -1.05 to 1.64, while 2 questions (numbers 8 and 9) are invalid with a value of more than -2. Apart from that, the question items have no potential to be guessed, as evidenced by the guessing value of 0 (zero). This Relativity Concept Inventory Test instrument meets the requirements for use in measuring students' conceptual understanding
Keywords: assessment, inventory concept, relativity.
DOI: http://dx.doi.org/10.23960/jpmipa/v25i1.pp142-154
Full Text:
PDFReferences
Adams, R. J., & Khoo, S. T. (1993). Quest: the interactive test analysis system. australian council for educational research.
Adedoyin, O. O., & Mokobi, T. (2013). Using irt psychometric analysis in examining the quality of junior certificate mathematics multiple. International Journal of Asian Social Science, 3(4), 992–1011.
Agnew, S., Kerr, J., & Watt, R. (2021). The effect on student behaviour and achievement of removing incentives to complete online formative assessments. Australasian Journal of Educational Technology, 2021(4), 173–185.
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142.
Allen, K., Reed, T., & Terry, R. (2006). Work in progress: assessing student confidence of introductory statistics concepts. Proceedings - Frontiers in Education Conference, FIE, 13–14.
Aslanides, J., & Savage, C. (2013). Relativity concept inventory: Development, analysis, and results. Physical Review Special Topics-Physics Education Research, 9(1), 10118–10128.
Baker, F. (2021). The basics of item response theory. (C. Boston & L. Rudner, Eds.). ERIC Clearinghouse on Assessment and Evaluation.
Cormier, S., & Steinberg, R. (2010). The twin twin paradox: exploring student approaches to understanding relativistic concepts. The Physics Teacher, 48(9), 598–601.
D Mardapi. (2008). Teknik penyusunan instrumen tes dan nontes [Techniques for preparing test and non-test instruments]. Mitra Cendikia Press.
Dai, S., Vo, T., Kehinde, O., He, H., Xue, Y., Demir, C., & Wang, X. (2021). Performance of polytomous irt models with rating scale data: an investigation over sample size, instrument length, and missing data. Frontiers in Education, 6, 1–18.
Elisa, A., Sotos, C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2009). How confident are students in their misconceptions about hypothesis tests? Journal of Statistics Education, 17(2), 1–21.
Embretson, S., Yang, X., Green, J. L., Camilli, G., & Elmore, P. B. (2006). Item response theory. In Handbook of complementary methods in education research.
Ene, E., & Ackerson, B. J. (2018). Assessing learning in small sized physics courses. Physical Review Physics Education Research, 14(1).
Farmer, S. (2021). Introduction of einsteinian physics to the upper secondary school physics curriculum in scotland. In Teaching Einsteinian Physics in Schools (1st ed.). Routledge.
Farrell, G., & Leung, Y. (2008). Convergence of validity for the results of a summative assessment with confidence measurement and traditional assessment.
Gero, A., Tsybulsky, D., & Levin, I. (2019). Research and design triads in the digital epoch: Implications for science and technology education. Global Journal of Engineering Education, 21(1), 80–83.
Goncher, A., Boles, W., & Jayalath, D. (2015). Using textual analysis with concept inventories to identify root causes of misconceptions. Proceedings - Frontiers in Education Conference, FIE, 2015, 1–4.
Guilford, J. P. (1956). Fundamental statistics in psychology and education, 3rd ed. In Fundamental statistics in psychology and education, 3rd ed. McGraw-Hill.
Hartle, J. B. (2005). General relativity in the undergraduate physics curriculum. American Journal of Physics, 74(1), 14–21.
Hazari, Z., Tai, R. H., & Sadler, P. M. (2007). Gender differences in introductory university physics performance: The influence of high school physics preparation and affective factors. Science Education, 91(6), 847–876.
Istyono, E. (2020). Pengembangan instrumen penilaia dan analisis belajar fisika dengan teori klasik dan modern [development of instruments for assessment and analysis of physics learning with classical and modern theories]. UNY Press.
Kulgemeyer, C., & Wittwer, J. (2021). When learners prefer the wrong explanation: misconceptions in physics explainer videos and the illusion of understanding. 1-29.
Listianingrum, S. A., Jumadi, J., & Zakwandi, R. (2022). Physics student misconception: relative velocity, time dilatation, and length contraction. Jurnal Ilmiah Pendidikan Fisika, 6(2), 1–7.
McGrath, K. (2019). Investigating the impact of parameter instability on item response theory proficiency estimation. Proceedings of the 2019 AERA Annual Meeting.
Muraki, E. (1992). A Generalized partial credit model: application of an em algorithm. Applied Psychological Measurement, 16(2), 159–176.
Pasquali, L., & Primi, R. (2003). Fundamentos da teoria da resposta ao item: TRI. Avaliaçao Psicologica: Interamerican Journal of Psychological Assessment, 2(2), 99–110.
Piacsek, A. A. (2018). A new pre/post test to assess student mastery of introductory level acoustics and wave mechanics. The Journal of the Acoustical Society of America, 144(3), 1785–1786.
Sachan, A., Bhadri, G. N., & Kittur, J. (2019). Design and development of concept assessment tool (cat):a concept inventory. Journal of Electrical Engineering & Technology, 33(1), 16–21. https://api.semanticscholar.org/CorpusID:214242537
Samejima, F. (1997). Graded Response Model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of Modern Item Response Theory (pp. 85–100). Springer New York.
Scherr, R. E., Shaffer, P. S., & Vokos, S. (2001). Student understanding of time in special relativity: Simultaneity and reference frames. American Journal of Physics, 69(S1), 24–35.
Sievertsen, H., Gino, F., & Piovesan, M. (2016). Cognitive fatigue influences students’ performance on standardized tests. Proceedings of the National Academy of Sciences of the United States of America, 113.
Singh Rana, S. (2014). Test item analysis and relationship between difficulty level and discrimination index of test items in an achievement test in biology. Indian Journal Of Research, 3(6), 56–58.
Siong, L. chin, Tyug, O. Y., Phang, F. A., & Pusppanathan, J. (2023). The use of concept cartoons in overcoming the misconception in electricity concepts. Participatory Educational Research, 10(1), 310–329.
Thacker, B., Kim, E., Trefz, K., & Lea, S. M. (1994). Comparing problem solving performance of physics students in inquiry‐based and traditional introductory physics courses. American Journal of Physics, 62(7), 627–633.
Vicovaro, M. (2023). Grounding intuitive physics in perceptual experience. Journal of Intelligence, 11(10), 1–20.
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 Jurnal Pendidikan MIPA
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The copyright is reserved to The Jurnal Pendidikan MIPA that is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.