Validity and Reliability of the Global Warming Instrument: A Pilot Study Using Rasch Model Analysis
Abstract
Abstract: This research aims to determine the profile of students regarding literacy skills for global warming material. This research uses a quantitative descriptive method to reveal the results of the research obtained based on students' responses to the 10 questions developed. This research was conducted at a school in Surabaya with a total of 15 respondents consisting of 4 men and 11 women. The results obtained were then analyzed using the Rasch Model with Winstep software to make it easier to map the level of instrument validity and the level of students' literacy skills. The results of this research reveal that the relationship between the instrument developed and students has very poor validity and reliability values with a value obtained of 0.22. However, the results of further analysis can map item difficulty levels and students' ability levels based on the Wright Map where students who have very high literacy skills are coded 01LK and 08PK while the most difficult item is Q2. Apart from that, students who have very poor literacy skills are good at coding 07LK while the easiest item is with code Q1. In this way, there is a need to improve problem instruments and further research to continue to develop the potential of scientific literature, not only on global warming material.
Keywords: instruments, literacy scientific, measurements, rasch model.
Abstrak: Penelitian ini bertujuan untuk mengetahui profil siswa mengenai kemampuan literasi untuk materi pemanasan global. Penelitian ini menggunakan metode deskriptif kuantitatif untuk mengungkap hasil penelitian yang diperoleh berdasarkan respon siswa terhadap 10 pertanyaan yang dikembangkan. Penelitian ini dilakukan di salah satu sekolah di Surabaya dengan jumlah responden sebanyak 15 orang yang terdiri dari 4 laki-laki dan 11 perempuan. Hasil yang diperoleh kemudian dianalisis menggunakan Rasch Model dengan software Winstep untuk memudahkan dalam memetakan tingkat validitas instrumen dan tingkat kemampuan literasi siswa. Hasil dari penelitian ini mengungkapkan bahwa hubungan antara instrumen yang dikembangkan dengan siswa memiliki nilai validitas dan reliabilitas yang sangat buruk dengan nilai yang diperoleh sebesar 0,22. Namun, hasil analisis lebih lanjut dapat memetakan tingkat kesulitan butir soal dan tingkat kemampuan siswa berdasarkan Wright Map dimana siswa yang memiliki kemampuan literasi sangat tinggi diberi kode 01LK dan 08PK sedangkan butir soal yang paling sulit adalah Q2. Selain itu, siswa yang memiliki kemampuan literasi yang sangat kurang baik diberi kode 07LK sedangkan butir soal yang paling mudah diberi kode Q1. Dengan demikian, perlu adanya perbaikan instrumen soal dan penelitian lebih lanjut untuk terus mengembangkan potensi literatur ilmiah, tidak hanya pada materi pemanasan global.
Kata kunci: instrumen, literasi sains, pengukuran, model rasch.
DOI: http://dx.doi.org/10.23960/jpmipa/v24i4.pp912-922
Full Text:
PDFReferences
Afandi, A., Sajidan, S., Akhyar, M., & Suryani, N. (2019). Development frameworks of the Indonesian partnership 21st-century skills standards for prospective science teachers: A Delphi Study. Jurnal Pendidikan IPA Indonesia, 8(1), 89–100.
Arrafi, W. Q. L., Wasis, W., Ningsetyo, M., Amiruddin, M. Z. Bin, Desysetyowati, N., & Sunarti, T. (2023). Implementation of Probolinggo Local Wisdom-Based Problem-Based Learning Model to Improve the Science Literacy Skills of High School Students. Physics Education Research Journal, 5(1), 35–42.
Chan, S.-W., Looi, C.-K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: a Rasch model measurement analysis. Journal of Computers in Education, 8, 213–236.
Čipková, E., Karolčík, Š., & Scholzová, L. (2020). Are secondary school graduates prepared for the studies of natural sciences?–evaluation and analysis of the result of scientific literacy levels achieved by secondary school graduates. Research in Science & Technological Education, 38(2), 146–167.
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412.
Dalaiden, Q., Schurer, A. P., Kirchmeier‐Young, M. C., Goosse, H., & Hegerl, G. C. (2022). West Antarctic Surface Climate Changes Since the Mid‐20th Century Driven by Anthropogenic Forcing. Geophysical Research Letters, 49(16), e2022GL099543.
Driga, A. M., & Drigas, A. S. (2019). Climate Change 101: How Everyday Activities Contribute to the Ever-Growing Issue. Int. J. Recent Contributions Eng. Sci. IT, 7(1), 22–31.
Drigas, A., & Mitsea, E. (2021). 8 Pillars X 8 Layers Model of Metacognition: Educational Strategies, Exercises &Trainings. International Journal of Online & Biomedical Engineering, 17(8).
Duckett, L. J. (2021). Quantitative research excellence: Study design and reliable and valid measurement of variables. Journal of Human Lactation, 37(3), 456–463.
économiques, O. de coopération et de développement. (2019). PISA 2018 Assessment and Analytical Framework. OECD publishing.
Hagell, P. (2014). Testing rating scale unidimensionality using the principal component analysis (PCA)/t-test protocol with the Rasch model: the primacy of theory over statistics. Open Journal of Statistics, 4(6), 456–465.
Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But…. Communication Methods and Measures, 14(1), 1–24.
Jufrida, J., Basuki, F. R., Kurniawan, W., Pangestu, M. D., & Fitaloka, O. (2019). Scientific Literacy and Science Learning Achievement at Junior High School. International Journal of Evaluation and Research in Education, 8(4), 630–636.
Kamal, S. A. (2022). The Effects of Global Warming: The Case Study of Karachi’s Heat Waves & Its Implication. International Journal of Policy Studies, 2(1).
Kienhues, D., Jucks, R., & Bromme, R. (2020). Sealing the gateways for post-truthism: Reestablishing the epistemic authority of science. Educational Psychologist, 55(3), 144–154.
Kumar, U. (2023). Global Warming, Climate Change and Environment. Guarav book center.
Ludlow, L. H., & Haley, S. M. (1995). Rasch model logits: Interpretation, use, and transformation. Educational and Psychological Measurement, 55(6), 967–975.
OECD. (2021). PISA 2025 Science Framework (First Draft). May, 1–93.
Pittman, R. T., Zhang, S., Binks‐Cantrell, E., Hudson, A., & Joshi, R. M. (2020). Teachers’ knowledge about language constructs related to literacy skills and student achievement in low socio‐economic status schools. Dyslexia, 26(2), 200–219.
Raof, S. A., Musta’amal, A. H., Zamzuri, F. K., & Salleh, M. H. (2021). Validity and reliability of students perceptions on OBE approach in Malaysian VC using Rasch model. Journal of Innovation in Educational and Cultural Research, 2(2), 44–50.
Sharon, A. J., & Baram‐Tsabari, A. (2020). Can science literacy help individuals identify misinformation in everyday life? Science Education, 104(5), 873–894.
Spiro, R. J., Vispoel, W. P., Schmitz, J. G., Samarapungavan, A., & Boerger, A. E. (2013). Knowledge acquisition for application: Cognitive flexibility and transfer in complex content domains. In Executive control processes in reading (pp. 177–199). Routledge.
Sumintono, B. (2018). Rasch model measurements as tools in assesment for learning. 1st International Conference on Education Innovation (ICEI 2017), 38–42.
Sumintono, B., & Widhiarso, W. (2014a). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial (edisi revisi). Trim Komunikata Publishing House.
Sumintono, B., & Widhiarso, W. (2014b). Aplikasi Model Rasch untuk Penelitian Ilmu-Ilmu Sosial [Application of the Rasch Model for Social Sciences Research]. Bandung, Indonesia: Trimkom Publishing House.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 1273–1296.
Tennant, A., & Conaghan, P. G. (2007). The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care & Research, 57(8), 1358–1362.
Upadhyay, R. K. (2020). Markers for global climate change and its impact on social, biological and ecological systems: A review. American Journal of Climate Change, 9(03), 159.
Valladares, L. (2021). Scientific literacy and social transformation: Critical perspectives about science participation and emancipation. Science & Education, 30(3), 557–587.
Widodo, W., Sudibyo, E., Suryanti, S., Sari, D. A. P., Inzanah, I., & Setiawan, B. (2020). The Effectiveness of Gadget-Based Interactive Multimedia in Improving Generation Zâ€TM s Scientific Literacy. Jurnal Pendidikan IPA Indonesia, 9(2), 248–256.
Zhao, Y. (2020). Two decades of havoc: A synthesis of criticism against PISA. Journal of Educational Change, 21(2), 245–266.
Refbacks
- There are currently no refbacks.
Copyright (c) 2023 Jurnal Pendidikan MIPA
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The copyright is reserved to The Jurnal Pendidikan MIPA that is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.