The Effect of Flipped Classroom Model on Mathematical Ability: A Meta Analysis Study

: The Effect Of Flipped Classroom Model on Mathematical Ability: A Meta Analysis Study. Objective : This study aims to determine the effect of the flipped classroom model on mathematical ability. Methods : The study design was a meta-analysis by analyzing 70 effect sizes from 44 primary studies that met the inclusion criteria. Findings : The combined effect size was (d = 0.73; p < 0.01). Measure the effect based on education level (Qb = 21.04; p < 0.05), use of LMS (Qb = 5.91; p < 0.05), sample size (Qb = 9.83). ; p < 0.05), year of study (Qb = 34.85; p < 0.05), type of publication (Qb = 7.08; p < 0.05), trial period (Qb = 46.60; p < 0.05), and region (Qb = 49.59; p < 0.05). Conclusion : The flipped classroom model has an effect on mathematical ability. The effect of the flipped classroom model compared to traditional teaching on math skills differs according to the educational level group, use of LMS, sample size, year of study, type of publication, time of experiment, and region.


 INTRODUCTION
Mathematical skills have long been recognized as essential, not only for academic success but also for efficient functioning in everyday life (Carey et al., 2017). By studying mathematics, we practice accuracy, consistency and mental discipline, which are important skills needed for effective and responsible problem solving and decision making in everyday life (Brezavšcek et al., 2020). However, mathematics teaching currently places students as passive subjects (Rajghatta, 2014), students passively receive information from the teacher. This causes students difficulty in understanding mathematics subjects (Offer & Bos., 2009). Nowadays, mathematics educators face one of the main challenges to improve students' performance in mathematics (Tan & Tan., 2015).
With the rapid advances in educational technology today, the teaching and learning environment has begun to change and develop (Karagol & Esen, 2020). Therefore, teaching in schools is expected to be able to find new approaches to develop and update the teaching process. These approaches should focus on the role of the learner and make it central to the learning process. Every student can learn and achieve a level of proficiency if the teaching and learning environment and teaching methods are in accordance with their abilities and needs (Elian & Hamaidi, 2018). In line with technology and science, changing needs of learners, differentiation in instructional design and evolving opportunities form the basis for new teaching approaches to be put into practice. Reverse learning is a new alternative to traditional learning environments (Limayanta et al., 2021;Abeysekera & Dawson, 2015;Altakhaeyneh, 2022;Kvashnina & Martynk, 2016;Arnold-Garza, 2014;Bergmans & Sams, 2012;Bishop & Vergler, 2013 ;Enfield, 2013). The flipped classroom model utilizes problem-based and active learning techniques and new technologies to engage students (Arnold-Gaza, 2014). Flipped classrooms offer students the opportunity to be more independent in managing their learning, they can explore materials such as videos, readings or exercises at their own pace (Holton et al., 2016). Teaching with the flipped classroom approach requires students to make pre-class preparations by watching videos, while class time is used for discussion and problem solving activities related to the topic (Pierce & Fox, 2012;Tune et al., 2013). Pre-class activities are expected to allow students to use their study time independently to acquire fundamental knowledge and skills. While learning is in class (face to face), students are expected to participate individually and collaboratively, and receive individualized support from the teacher (Brewer & Movahedazarhouligh, 2018;Umam et al., 2019;DePietro et al., 2020). Flipped classroom requires class activities to be student-centered, active learning, teachers not only provide information, but make them independent learners (Bergmann & Sams, 2012;Bergmann & Sams, 2014;Abeysekera & Dawson, 2015). Activities used in the flipped classroom model related to active learning are considered to be derived from constructivism (Bishop & Vergleher, 2013;Arnold-Gaza, 2014;Abeysekera & Dawson, 2015).
A meta-analysis study that focuses on the effect of using the flipped classroom on mathematics learning has so far only been conducted by Yakar (2021) in Turkey. However, the meta-analysis studies carried out only focused on the elementary school level. The study is also tentative due to the limited inclusion criteria and scope of the search. The studies analyzed are more dominant in thesis research and studies conducted in Turkey. This study extends and complements previous research that focused on determining the overall effect of the flipped classroom on students' mathematical abilities. Therefore, the purpose of this study was to determine the effect of the flipped classroom approach on students' mathematical abilities compared to the traditional learning approach using the meta-analysis method. , this study seeks answers to the following questions:

 METHODS Participants
The population in this meta-analysis study were all students who were involved in research related to the effect of using flipped classroom on students' mathematical abilities published in an online database. The sample in this metaanalysis was 3645 participants who were involved in studies that met the inclusion criteria.

Design and Procedures
In this study, the meta-analysis method was used to review the results of research examining the effect of the flipped classroom model on students' mathematical abilities. In general, the procedures in the meta-analysis are; determining inclusion criteria, study tracing, data collection and variable coding, statistical analysis (Borenstein et al., 2009;Retnawati et al., 2018;Juandi & Tamur., 2020).

Inclusion criteria
In this meta-analysis, the determination of inclusion criteria aims to facilitate the search for studies at a later stage. All studies collected in the initial search were then examined and assessed using the inclusion criteria defined for inclusion in the metaanalysis and further evaluation. The inclusion criteria established in this meta-analysis included: 1. The year of publication ranges from 2015 to 2022; 2. Studies can be in the form of doctoral theses, master's theses, undergraduate theses, and articles published in national or international journals; 3. Studies using experimental or quasiexperimental research methods; 4. There is at least 1 experimental group with the flipped classroom model and the comparison group as the control group with the traditional model; 5. The study must report the mean, standard deviation and sample size of each experimental group and control group; or sample size and t-value; or sample size and p-value; or sample size with F-value.

Data Collection and Coding
The stage of collecting relevant studies using online databases such as Google Scholar, ERIC, Elsevier, and others. The keywords used in the literature search were "Effectiveness of the Flipped Classroom" and "Mathematics" in both Indonesian and English. Based on the specified inclusion criteria, 70 effect sizes were obtained from 44 primary studies. After getting an article that is eligible (meets the inclusion criteria), then identify the characteristics of the literature by coding. The coding in this study was carried out by two people (raters) so that subjective errors could be avoided. The coding content includes information; 1) Education Level; 2) Sample Size; 3 years; 4) use of LMS; 5) Type of Publication; 6) Time of experiment; 7) regions; 8) Frequency; and 9) Percentage. Table 1 presents a summary of the coding results.
. The heterogeneity test in this study was carried out using the Q parameter approach. If the p-value < 0.05, the estimation model that is suitable for calculating the summary effect is the random effects model. If the p value > 0.05, then a fixed effect model estimate is used (Borenstein et al., 2009;Retnawati et al., 2018;Juandi & Tamur., 2020). Studies containing the statistics required in the meta-analysis require a publication bias test (Retnawati et al., 2018;Juandi & Tamur, 2020;Yunita et al., 2020;Martaputri et al., 2021;Setiawan et al., 2022). The publication bias test used the File-Safe N (FSN) approach. If the File-Safe N value > (5K+10), where k is the number of studies included in the meta-analysis, then this study has no publication bias problem and can be scientifically justified (Mulen et al., 2001).

 RESULTS AND DISCUSSION Effect Size of Each Study
The first step in this meta-analysis was to calculate the effect size of each study. The study effect size was calculated with the help of JASP 0.16.1.0 software. Effect size values range from -0.201 to 1.965. Table 3 provides a summary of the effect size values, for each study.  Based on table 3 above, out of a total of 70 effect sizes, there are eleven effect sizes (n = 11 or 15.71%) classified as negligible effects, eleven effect sizes (n = 11 or 15.71%) classified as small effects, fifteen effect sizes (n = 15 or 21.43%) were classified as moderate effects, eighteen effect sizes (n = 18 or 25.71%) were classified as large effects, and fifteen effect sizes (n = 15 or 21.43%) were classified as very large effects. Figure 1 presents the number of effect size classifications.

Heterogeneity Test
The second stage is to test for heterogeneity and select the appropriate estimation model. The heterogeneity test was conducted to determine the model to be used in calculating the effect size of the 70 studies to be analyzed. The heterogeneity test in this study was carried out using the Q parameter approach with degrees of freedom (df = 70-1 = 69). Table 4 presents the results of the heterogeneity test. The results of the heterogeneity test (see table 2) showed that (Q = 363.99, p < 0.001). It can therefore be concluded that the variance between the effect sizes used in this study is heterogeneous. The value of I² being 81.04% also indicates high heterogeneity. According to these statistics, because the study was very heterogeneous, a random effects model was used in calculating the combined effect size. It is also potential for analysis of moderator variables to determine the contribution of each moderator variable to the difference in variance between effect sizes included in this meta-analysis.

Overall Effect Size and Analysis of Moderator Variables
The third step is to calculate the combined effect size and analyze the moderator variables. The moderator variables identified in this study are (level of education, sample size, skills measured, years, use of LMS, type of publication, time of experiment and region. Table 5 presents a summary of the combined effect sizes using random effects model estimation and analysis of moderator variables. The results of the analysis showed that the overall effect size of the study was (d= 0.73; k = 70). This effect size is in the medium category. These results indicate that the overall use of the flipped classroom model has a moderate effect on mathematical ability when compared to traditional learning. This finding is in line with the results of a previous metaanalysis conducted by Yakar (2021) which revealed that the use of the flipped classroom had a moderate effect on mathematics achievement compared to traditional learning (d = 0.51; k = 46). However, different results were found by Algarni (2018) who conducted a meta-analysis of studies in 2010-2017. The results of his research revealed that learning mathematics using a flipped classroom had little effect when compared to traditional learning (0.27; k = 34). This difference in results becomes the basic idea for further research involving more primary studies and more recent years of study.
Based on the moderator variable of education level, the results of the analysis reveal that the effect of the flipped classroom model compared to traditional teaching on mathematics learning differs according to education level. Of the four groups, the use of the flipped classroom model was effective in the junior high school group, high school group and university group. While the elementary school group was not proven significant. This reveals that the flipped classroom in mathematics learning is not proven to be effective when compared to traditional learning at the elementary school level. This result is different from the metaanalysis conducted by (Yakar, 2021;Karagol & Esen, 2019) which revealed that the use of the flipped classroom was effective on the mathematics achievement of elementary school students. However, the problem in this study is that primary school groups are grouped in grades 1 to 6, while the studies above do not group elementary school groups in the same way, this may result in different results. To obtain consistent results, it is recommended to conduct more primary studies and stricter inclusion criteria, especially focusing on the elementary school level.
Based on the moderating variable of LMS use, the results of the analysis reveal that the effect of the flipped classroom model compared to traditional teaching on mathematics learning differs according to the group using the LMS. The use of the flipped classroom model in learning mathematics is more effective in the group that uses the LMS than the group that does not use the LMS. These results are in accordance with the findings (Bradley, 2016;Chaney, 2016;Crowley, 2018;Comfort, 2016;Day, 2017;Osborne, 2020;Ozeda et al., 2017) which revealed that the use of LMS resulted in greater mathematics academic achievement than conventional. Thus it can be said that in order to achieve a higher level of effectiveness, flipped classroom learning is recommended to use a small sample size. Despite the differences, the two groups confirmed that the use of the flipped classroom model was effective compared to traditional teaching.
Based on the sample size moderator variable, the results of the analysis show that the use of the flipped classroom model is more effective in groups with small sample sizes (d<30) than groups with large sample sizes (d>30). In this study, the sample size was grouped based on the experimental group. These results indicate that the use of the flipped classroom is more effective if it involves the number of students below or equal to 30 compared to the number of students above 30. This result is in line with the findings (Yakar, 2021;Karagon & Esen, 2019;Juandi et al., 2021). Their findings show that small sample sizes produce larger effect sizes. Thus it can be said that in order to achieve a higher level of effectiveness, flipped classroom learning is recommended to use a small sample size. Although this study reported that there were significant differences based on the sample size group, both groups proved to be effective using the flipped classroom model compared to traditional teaching.
Based on the moderator variable in the year of research, it was found that the effectiveness of the flipped classroom model compared to traditional teaching in mathematics learning differs according to the year of the study. The use of the flipped classroom model was most effective in the 2019-2022 group compared to 2015-2018. We speculate that the use of the flipped classroom was more effective in the current research year perhaps because the flipped classroom model has gone through a development process from previous years, so the results obtained will be better than the previous year. Although there are significant differences in this study, the two groups proved effective in using the flipped classroom model compared to traditional teaching.
Based on the moderator variable of publication type, the results of the analysis reveal that the effect of the flipped classroom model compared to traditional teaching on mathematics learning differs according to the type of publication. The use of the flipped classrom model in learning mathematics is more effectively reported by the journal group than that reported by the thesis group. We speculate that results reported in journals tend to report only significant research results. This result is also in line with the findings of Yakar (2021) which revealed that the reported effect sizes in the article group were significantly different from the thesis group. However, different results reported by Orhan (2019) and Karagol & Esen (2019) in their meta-analysis study showed that the effect sizes of the studies grouped into articles and theses were not significantly different. Although these findings report different effect sizes by publication type group, both groups confirm that the flipped classroom model is more effective than traditional teaching.
Based on the moderating variable for the duration of the experiment, the results of the analysis reveal that the effect of the flipped classroom model compared to traditional teaching on mathematics learning differs according to the experimental duration group. The use of the flipped classroom model was most effective in the group with an experimental duration of less than one month compared to the group with an experimental duration of more than one month. These results are in line with the findings of the meta-analysis of Juandi et al. (2021) who found that the duration of the experiment affected the effect size. However, this result is different from the findings (Yakar, 2021;Karagol & Esen;2019;Saygili & Cetin;. Their findings show that the effect of the flipped classroom model compared to traditional teaching on mathematics learning does not differ according to the experimental duration group. Although this study found significant differences, the two groups proved effective in using the flipped classroom model compared to traditional teaching. Based on the measured region moderator variable, it was found that the effectiveness of the flipped classroom model compared to traditional teaching in mathematics learning differs by region group. Of the two groups, the use of the flipped classroom model was most effective in the national region group compared to the international region group. This finding is in line with the results of a meta-analysis conducted by Karagol & Esen (2019) which also revealed that the effect of using flipped classrooms compared to traditional teaching differs between the national (Turkey) and international levels. Saygili & Cetin (2021) also found that the effect size of using LMS versus traditional learning differed between countries. However, in contrast to the results of Yakar's (2021) analysis who also conducted a metaanalysis study in Turkey, the results of the analysis showed that there were no significant differences between national and international groups. Although there were significant differences in this study, both groups were proven to report that the use of the flipped classroom model was more effective than traditional teaching.

Evaluation of Publication Bias
The final step in the meta-analysis is to detect publication bias. The evaluation of publication bias was carried out to show that the meta-analysis carried out was truly objective, in the sense that the articles that were the material for the meta-analysis were correct and showed results that were in accordance with the reality in the field. There are many methods that can be used to analyze publication bias. In this study, publication bias was evaluated using the File-Safe N method. Table 6 presents the results of the diagnosis of Fail-Safe N values.
The results of the FSN test are shown in table 6. Because the value of k = 70 then 5k + 10 = 360. The Fail-Safe N value obtained is (FSN = 16114) with target significance  + 10), this indicates that the metaanalysis carried out has no problems of publication bias and is scientifically justified (Mullen et al., 2001;Borenstein et al., 2009;Retnawati et al., 2018;Juandi & Tamur., 2020)  CONCLUSIONS The results of the analysis show that the application of the flipped classroom model has an effect on students' mathematical abilities compared to the application of the traditional approach. Based on the analysis of moderator variables, it is known that the effect of the flipped classroom model on mathematical abilities differs according to education level, use of LMS, sample size, year of research, duration of experiment, type of publication and national/international.
The findings of this meta-analysis show the consistency of the publication of research results on the effect of using the flipped classroom model on students' mathematical abilities. Apart from the reported validation results, this study also has limitations. This study only analyzed 70 effect sizes. This study also only analyzes mathematical abilities in general. Further research needs to expand the research sample and analyze mathematical abilities more specifically, for example: critical thinking skills, mathematical communication, and others. In addition, it is also recommended to be more specific in reviewing the analysis of moderator variables in this study by involving more research so that research findings become more accurate.