Effect of Blended Learning on Students’ Learning Outcomes: A Meta-Analysis

: Effect of Blended Learning on Students’ Learning Outcomes: A Meta-analysis. Objective : This research aims to determine the effect of blended learning implementation in Indonesia between 2016-2020. Methods : This is a meta-analysis study and 27 articles met the criteria analyzed by Revman 5.4. This study reported effect size of blended learning on learning outcomes consisting of subgroups of education levels (elementary school, junior high school, senior high school, vocational education, and college) and subgroups of learning outcomes (knowledge, thinking ability, and attitude). Findings : The theoretical effect size score of Standard Mean Difference or SMD = 1.01 (95% CI [0.802– 1.20). P < 0.00001) for blended learning compared to conventional. Subgroup analysis obtained a high category effect size for Blended Learning’s application according to the level of education and learning achievement. Except for the effect size for attitudes belongs to the moderate category. Conclusion : This study concluded that Blended Learning positively affects knowledge mastery, thinking ability, motivation, and learning independence at all levels of education.


 INTRODUCTION
Education plays an important role in developing knowledge. skills. attitudes and values that enable people to contribute and benefit from an inclusive and sustainable future. Education needs to do more than just prepare young people for the world of work; equip students with the skills they need to become active, responsible citizens. Many research in education conducted to enhance education quality including teacher professional development, teaching methods, and learning tools.
Nowadays, learning paradigm change that student can learn at the same time but different place, and provide asynchronous learning where space and time are not obstacles (Nieuwoudt. 2020). This means students can determine when and where they will study. This is very reasonable because Information and Communication Technology provide facilities that are easy and accessible to students from various levels of education. The use of technology as a learning facilitator is very possible because the current generation of school age is a digital generation that is accustomed to living in a digital device environment.
There is no doubt that technology is changing the purpose and direction of education (van de Oudeweetering &Voogt. 2018). This happens because learning resources provide support to knowledge development through facilitation of knowledge formation with various digital media (Chai et al.. 2015). New technologies provide different ways of interpreting communication. collaboration and social intersession (Olsson et al., 2020).
One of the learning methods that utilize Information and Communication Technology is Blended Learning. Previously Blended Learning defined an effective combination of presentation of materials, models and different learning types (Murray et al.. 2009). Blended Learning defined as a combination of different learning methods (expository, discovery methods, cooperative models) and different methods of material delivery (personal communication, broadcast, publication) (Klentien &Wannasawade. 2016). Then Blended Learning is defined based on the place of learning as a combination of presentation of materials can be in the form of students studying some of the teaching materials outside the classroom and some of them in the classroom with the direction of the teacher (Brown, 2016). Chew et al. (2008) explains that Blended Learning involves two things. namely education and educational technology. Also, Wong et al. (2014) stated Blended Learning is the integration of the paradigm of face-to-face teaching and e-learning. Friesen (2012) redefined the term "faceto-face" with "teacher presence" which means teacher attendance can be in online form. So according to Friesen, Blended Learning is designed to provide learning opportunities facilitated by the internet and digital media that are able to present teachers and students. In this sense the presence of teachers and students can be accommodated by digital media such as zoom, Google Meet, etc. Thus, on distance learning it is also possible to conduct Blended Learning. This opinion is supported by Krasnova (2015) that Blended Learning is a teaching method that combines face-to-face and online in the form of interactive collaborations that serve to complement each other.
Blended Learning as one of the environments of digital learning has advantages in terms of personalization, flexibility and efficiency (Laurillard. 2015). First, personalization means that the digital environment can match the individual's abilities i.e. learning speed and learning needs (Patrick &Sturgis, 2015). Second, flexibility means online learning allows students to learn anytime and anywhere but face-to-face learning remains important to anticipate problems facing students or students. Third, efficiency is online learning is free and can't be access easily.
Many studies have been conducted by researchers previously related to the effectiveness of Blended Learning in education. Atmacasoy &Aksu (2018) conducted a systematic literature review on Blended Learning in pre-service teachers in Turkey. They concluded Blended Learning develops positive attitudes, and faceto-face learning activity increases due to social interactions between peers and teachers. Bernard et al. (2014) conducted a meta-analysis of Blended Learning and the use of technology in universities. The findings are technology-assisted Blended Learning can improve learning outcomes and learning quality. Boelens et al. (2017) conducted a systematic literature review of four important concepts in designing Blended Learning. The results show social interaction occurs through face-to-face, while personalization and monitoring of student learning progress occurs when online activities.
Differ to earlier research, this meta-analysis studies aimed to investigate the influence of Blended Learning on learning outcomes at all levels of education in Indonesia. The research is limited to national research from 2016-2021. This research focuses on aspects of education, mastery of knowledge. thinking skills and attitudes of learners.

 METHODS
This research includes descriptive research using meta-analysis method that is to make studies and conclusions of topics taken from research reports. This study uses the results of similar research as basic data in conducting studies and conclusions. This study aims to find out the significance of Blended Learning to the learning outcomes of learners.
This study investigates the application of Blended Learning at the education level from elementary school, junior high school, senior high school, vocational education and college. The application of Blended Learning is not limited to certain disciplines including Social Sciences and sciences. In this study the influence of Blended Learning in compared to traditional strategy on learning outcomes.
In accordance with the purpose of the study is to find the influence of Blended Learning in Indonesia. then this research is limited to national research from 2016-2021. Systematic searches conducted in the electronic databases of Google Scholar and Garuda. Article search strategies using keywords: 1) "Influence of Blended Learning"; 2) "Blended Learning" AND "Learning outcomes"; 3) "Blended Learning" AND experiments;4) "Blended Learning" AND "Learning outcomes" AND students.
Articles to be involved in research are articles derived from research that has the following criteria: 1) The article published in the last 5 years; 2) The article publications indexed journals Sinta and Garuda; 3) Experimental Research; 4) Research variables are cognitive. affective and psychomotor domains; 5) The research population is elementary school, junior high school, senior high school, Vocational Education, and college students. There were 1.450 articles found initially related to Blended Learning. The articles filtered and finally we got 27 articles that eligible for analysis ( Figure 1).
After the screening process, 27 articles were obtained that were used in the study. Data analysis refers to Al-Samarraie & Hurmuzan (2018) the process of data analysis is carried out beginning with data extraction. The data is concise and summarizes independently the data from the included studies. The data organized in Table 1, contain: author ID, subject, methods, education level, research purposes, instruments and effect size.  Table 1, shows 27 studies involved in metaanalysis and Blended Learning conducted in elementary school (2 studies). junior high school (5 studies). Vocational Education (2 studies), high school (15 studies) and college (7 studies). Research is also spread across many fields of study covering the fields of study of Social Sciences. Science and Language. The treatment given to experimental class varies including Blended Learning and the combination of Blended Learning with other methods (Discovery Learning, Problem Prompting) and with various platforms available on the internet (Edmodo, Quiper School, Microsoft Mathematics, Schoology, Google Classroom, Animated videos).
Next, the data analyzed by the Review Manager application (RevMan 5.4) to explore  ) was applied to estimate the influence of heterogeneity on the results of this meta-analysis. Heterogeneity category (I 2 ) is low (25% -50%), moderate (50% -75%) and height (I2 > 75%) (Cohen. 1992). According to Cochrane (2011) random effect model is applied if heterogeneity I 2 > 50%; otherwise, fixed effect model is used. In addition, sensitivity analysis is performed by removing each study individually to assess the consistency and quality of the results.
Finally, the data is presented in the form of forest plot, green box shows Effect Size (ES) of each study and horizontal lines show confident interval (CI) of each estimate. The diamond in each subgroup shows the overall effect size of the research involved in meta-analysis.

 RESULT AND DISCUSSIONS Subgroup Analysis by Education Level.
As mentioned before, the research was conducted on different levels of education (elementary school, junior high school, Vocational Education, senior high school, college) using different measurement scales, so the data analyzed using Standardized Mean Difference. Forest plot (Figure 2) shows the overall Blended Learning influence score on a different scale. This study analyzed 27 articles (one article examined two variables) using experimental design involving 1616 participants (833 participants in the experimental group and 783 participants in the control group). Five studies (Wahyuni, Riantika, Septiyan, Fitriasari & Tsaniah) concluded that there were no significant differences in the score of study results between the experimental and control groups, while other studies showed statistically significant differences. In this study, random effect was used to predict the overall influence and compare subgroup differences based on education level and study result category because the research used different instruments (tests and questionnaires).
Effect Sized pooled has a significant difference in theoretical ES score with Standard Mean Difference (SMD) for overall is 1.01 (95% CI [0.802-1.20], P < 0.00001) for Blended Learning comparison with conventional. It is mean Blended Learning has a high influence on learning outcomes (Cohen. 1992). There is a significant relationship between Blended Learning and the improvement of learning outcomes as showed by P < 0.05 is P = 0.00001. The results of this analysis show Blended Learning has ES 1.01 against the learning results. The funnel plot (Figure 3) of 27 symmetrical studies shows that there is no publication bias. This means that researchers. editors. or publishers continue to publish the results of primary studies. even if they do not show significant effects.
Two studies in elementary school involved 35 students in the experimental group and 35 students in the control class. Effect size Blended Learning in high category SMD = 1.32 (95% CI [0.36 -2.28], P = 0.007). The implementation of Blended Learning in junior high school conducted by five researchers, involving 154 students in experimental group and 151 students in control group. Only one study, Septiyan et al. (2020) reported no significant effects of Blended Learning, but 4 other researchers stated that there was a significant Blended Learning influence. The application of Blended Learning in Junior High School has a high effect size with SMD = 0.98 (95% CI [0.58 -1.37], P < 0.00001).
The implementation of Blended Learning in senior high school was conducted by 12 researchers and involved 373 students in the experiment group and 360 students in the control group. Nine researchers reported that Blended Learning have positive effect learning outcomes, except for three researchers namely Wahyuni (2018), Tsaniah (2019) and Riantika (2019). The overall effect size for SMA remains high i.e., SMD = 1.00 (95%CI [0.68 -1.31], P < 0.00001).
Blended Learning implementation in Vocational Education conducted by two researchers. The study involved 68 students in experimental group and 66 students in control group. The findings of the study at the high school level showed that Blended Learning has a High Effect Size including SMD = 1.36 (95%CI [0.98 -1.74], P < 0.00001). This means that blended learning has a positive effect on learning outcomes and is significant compared to conventional learning.  Based on the analysis of the data above it can be concluded that the influence of Blended learning on learning outcomes is very high (SMD = 1.01) compared to traditional learning strategies. Among all levels of education, the highest impact of Blended Learning was found in vocational school (SMD = 1.35), then consecutively in elementary school (SMD = 1.32), junior high school (SMD = 0.98), senior high school (SMD = 1.00) and college (SMD = 0.88). The findings are in accordance with the findings of Krismadinata et al. (2020) that Blended Learning increases the effectiveness of learning in vocational education as well as improve HOTS and skills. Moreover, Vo et al. (2017) concluded that Blended learning significantly associated students STEM performance for college students than traditional classroom practice. Blended Learning impact on learning outcomes because it facilitated: 1) ease and speed in information; 2) bring up various learning resources (Setiawan, 2017). Moreover, ccording to Stein and Graham (2014).
Blended learning has the advantages to access anytime and everywhere. Online social interaction leads to positive and productive communication in the form of information exchange (Mustapa et al., 2015). Online discussions and face-to-face discussions contribute to the overall learning experience. Blended Learning is changing the paradigm from teaching to learning, thus enabling students to become more engaged in the learning process and more enthusiastic and, consequently increase their perseverance and commitment (Ismail et al. 2018).
There were seven studies at college and involved 374 students (203 experimental group students and 171 control groups). All researchers reported Blended Learning has a positive effect except Fitriasari which suggests there is no Blended Learning effect on learning outcomes (-0.43 -0.65). Effect size of Blended Learning in college also in high cathegory SMD = 0.88 (95%CI [0.49 -1.28], P < 0.0001).
Subgroup difference test using random effect model obtained Chi2 = 3.76 (P = 0.44) and I 2 = 0%. meaning Effect Size Blended Learning is not depending on by differences in education levels.

Subgroup analysis based on learning outcomes
The results of learning that studied in this research are mastery of knowledge, thinking ability, and attitude including motivation and selfregulated learning. The influence of Blended Learning on learning outcomes is found in 28 studies (one study examined knowledge and thinking). The meta-analysis conducted based on random effect model and using Standard Mean Difference because the measurement scale conducted in this study varies. Heterogeneity test in Table 3, shows I2 = 69% so that the data analyzed based on random effects.
Forest plot (Figure 4) shows that Effect Sized pooled has a significant difference in theoretical test scores with the acquisition of SMD = 1.01 (95% CI [0.802-1.20], P < 0.00001) for comparison of Blended Learning with conventional. With SMD > 0.8 it means Blended Learning impact learning outcomes   (Cohen,1992). There is a significant relationship between Blended Learning and the improvement of learning outcomes as showed by P < 0.05, P = 0.00001. The results of this analysis show Blended Learning has ES=1.01 against the learning results. The funnel plots of 28 symmetrical studies ( Figure.  5) show that there is no publication bias. This means that researchers, editors, or publishers continue to publish the results of primary studies even if they do not show significant effects.
Heterogeneity tests on each subgroup fall into the moderate category, knowledge mastery (I 2 = 67%), thinking ability (I 2 = 71%), and attitude (motivation/self-regulated) (I 2 = 77%). According to Cohen (1992), the use of Random effect analysis is correct. While heterogeneity between subgroups is I 2 = 0% means homogeneous research data or no differences between subgroups so the comparisons can be done between groups. Furthermore, compare the influence of Blended Learning on learning outcomes (knowledge, thinking ability and attitude (motivation and SRL)). The influence of Blended Learning on learning outcomes that fall into a high category is on the aspect of knowledge SMD = 1.02 (95%CI [0.77 -1.26], P < 0.00001) and thinking ability SMD = 1.11 (95%CI [0.76 -1.47], P < 0.00001). While the influence of Blended Learning is happening in the aspect of attitude (motivation/SRL) SMD = 0.66 (95%CI [0.02 -1.30], P = 0.04).
The application of Blended Learning influence student's knowledge was carried out by 17 researchers and involved 918 people (482 in experimental classes and 446 in control classes). Three researchers reported that there is no influence of Blended Learning on knowledge mastery, namely Riantika (2019), Tsaniah (2019) and Wahyuni (2018). But overall effect size Blended Learning on knowledge mastery remains high with SMD = 1.02 (95%CI [0.77 -1.26], P < 0.00001).
Furthermore. the influence of Blended Learning on thinking ability was implemented by 8 researchers and involved 500 people (259 people in the experimental class and 251 people in the control class). The findings showed that there is an influence of Blended Learning on thinking ability i.e., SMD = 1.11 (95%CI [0.76 -1.74], P < 0.00001). This means that blended learning has a positive and significant influence on thinking ability.
The application of Blended Learning to improve attitudes (motivation and SRL) was investigated by three researchers. The study involved 178 students (92 people on experimental group and 86 people in control group). Two researchers found a positive influence of Blended Learning on attitudes unless Fitriasari (2018) found Blended Learning had no effect on selfregulated learning. But the overall Effect size of Blended Learning towards attitudes belongs to the moderate category of SMD = 0.66 (95%CI [0.02 -1.30], P < 0.0001).
Subgroup difference test using random effect model obtained Chi2 = 3.76 (P = 0.44) and I 2 = 0%. It means there is no overall difference between the learning results. This means that the influence of Blended Learning on subgroups of learning outcomes is not influenced by categories according to learning outcomes.
Based on analysis according to the results of the study concluded that the influence of Blended Learning on the mastery of knowledge and thinking skills is significant and included in the high category compared to conventional learning. The findings are in accordance with Baragash &Al-Samarraie (2018) that Blended Learning improves the quality of learning and improves critical thinking. In addition, Blended Learning has the advantage of being able to grow the learning community in a wide range of classrooms through potential learning support facilities (Al-Shami et al.. 2018). Learning facilities can be accessed online by students from various sources and can be selected by users according to their needs and abilities. For students who need more time and more explanations, Blended Learning can be the proper solution for them.
Moreover, the influence of Blended Learning on motivation and independence of learning is higher than conventional learning. This study is in line with Tsai et al. (2018)

that Blended
Learning can improve self-efficacy, and have an impact on learning independence. This finding corresponds to Halverson (2016) that Blended Learning enhances learning pleasure and confidence. Blended Learning changes the paradigm from teaching to teaching, then makes students more active in learning so as to increase perseverance and commitment (Kashefi et al.. 2018).
Blended Learning provides diverse learning resources such as videos, animations and images, so it is very different from the learning environment in the classroom with limited learning media and learning activities. Therefore, Blended Learning makes students more motivated and their self-regulated increases. Moreover, students prefer to have discussion outside the classroom, because is less pressure and more comfortable so students who have low self believe will be sophisticated.

 CONCLUSIONS
Based on the results and discussions can be concluded that Blended Learning improves learning outcomes at all levels of education from elementary school to college. In addition, Blended Learning has a high influence on material mastery and thinking ability. Except for motivation and self-regulated learning Blended Learning has a moderate level of influence.
Implementation of Blended Learning in the learning process is tailored to the needs and context at each level of education. Blended Learning in elementary school students is more about exploration to construct knowledge and engage contextual learning environments. While the implementation at the junior high school level is more related to collaboration for the development and application of knowledge. At the high school and college level more on that demand independence and collaboration in creating ideas or ideas.
Therefore, it is recommended to all educators to apply Blended Learning as one of the learning models. Teacher need to prepare students to be able to use information and communication technology that will be involved in the learning process. Widely used facilities are learning facilities available in various websites on the internet. In this case teachers need to introduce students on a website that supports learning such as those available in Kemdikbud or from other public or private educational institutions. In addition. the strategy of finding information using a secure and accountable browser.
Next is a well-thought-out learning design containing student activities. supporting learning resources in the form of text, images or videos. These supporting resources can be provided by the teacher or use other people's products available on the internet. Lastly, teachers also consider the media or means of students to communicate with teachers or collaborate with other students such as using WhatsApp applications, email, Edmodo. etc.
This research is limited to investigating the influence of Blended Learning on some learning outcomes in the cognitive and affective domain. Therefore, there are still opportunities for further researchers to investigate the influence of Blended Learning on cognitive and other affective domains such as high levels of thinking, collaboration, communication, self-confidence, confidence, etc.