STEM Meta-Analytic Project Introduction


This meta-analytic project is titled “A Meta-Analysis of the Effectiveness of Small-Group Instruction Compared to Lecture-Based Instruction in Science, Technology, Engineering, and Mathematics (STEM) College Courses.” The project was funded by the Research and Evaluation in Science and Engineering (REESE) Program of the National Science Foundation (NSF) for the duration of two years (January 2009 to December 2010 with an additional one year no-cost extension).

The significance of the project stems from the numerous and consistent calls from federal agencies, employers, and national societies for an urgent need for STEM instructional reform and innovation in K-12 through graduate school. These organizations have emphasized the need to examine current teaching practices and student-learning processes with an emphasis on various forms of active small-group learning pedagogies such as cooperative, collaborative, problem-based learning, or team-based learning methods. These methods of learning/teaching emphasize effective communication and interactive teamwork skills among the students in the STEM college classrooms, as opposed to the traditional lecture-based instruction. These educational goals can be accomplished by using active small-group learning/teaching methods that provide students with self-directed and real-life learning experiences that help STEM graduates to be competitive in the global workplace. By replacing or supplementing the traditional lecture-based instruction with the various active small-group learning pedagogies in STEM college classrooms, the learning power shifts from the instructor to the learner, and is deemed to be far more effective in promoting higher achievement, better attitudes towards subject matters, and persistence in STEM classrooms as compared to the traditional lecture-based instruction.

A major purpose of this meta-analytic study was to produce rigorous scientific evidence and an answer to the question of whether the use of various forms of small-group learning methods in STEM college classrooms is more effective than the traditional lecture-based instruction in promoting higher achievement and attitudes toward STEM subjects as well as persistence in STEM college classrooms. This is an important question and is particularly challenging if it is to be answered in a way that can provide useful information for educators and policy makers on the most effective pedagogies of teaching/learning STEM college courses. The answer will help them to design future instructional environments that can be implemented successfully in real STEM college classrooms.

The design and implementation of this project produced three major meta-analytic data sets, one data set for each of the three outcome measures (achievement, attitude, and persistence). Each data set presented formidable logistical challenges with respect to (a) searching and identifying the STEM primary studies based on preset rigorous inclusion/exclusion criteria, (b) reducing the publication bias by searching the library databases for dissertations and unpublished reports, (c) coding of the study features and characteristics and the necessary summary statistics for calculating the effect sizes of the 193 primary studies included in the review, (d) ensuring the accuracy and quality of the coding process by training the coders and designing a comprehensive coding instrument, (e) using the multilevel approach for meta-analysis to increase the generalizability of the findings of this study, and (f) using sensitivity and publication bias analyses to assess the meta-analytic decisions that were made and the conclusions that were reached. The process of confronting these challenges over the period of this project has produced valuable lessons for future meta-analysts who conduct large scale meta-analytic project.

A total of 193 studies met the established criteria and were included in the meta-analysis. One-hundred and twenty two of the 193 primary studies focused on students’ achievement scores in STEM college courses, another 33 of the 193 primary studies focused on students’ attitudes toward STEM subject matter in the college classrooms, and the remaining 46 studies focused on students’ persistence (retention) in STEM college classrooms.. For the achievement and attitude primary studies, 158 and 33 standardized mean-difference.For the persistence primary studies, 70 proportion-difference effect sizes were extracted.

The results of the present meta-analytic review show that during the last four decades many different forms of small-group learning methods (cooperative learning, collaborative learning, problem-based learning, team-based learning, peer learning, and inquiry-based learning) have been developed, used, and evaluated in STEM college classrooms. The results also show that when compared with lecture-based instruction, all forms of small-group learning methods had positive impacts, in varied degrees, on student achievement, attitude, and persistence in various STEM college courses. For example, the results show that the weighted average effect-size for achievement was 0.37 using various forms of small-group learning methods across all STEM disciplines. An average effect size of 0.37 means that students’ STEM achievement is changed from the 50th percentile for the students who had been taught by the traditional lecture-based instruction to the 65th percentile for the students who had been instructed using various forms of small-group learning methods. In addition, the results show that there are differential effects of various forms of small-group learning methods across the different STEM disciplines. For example, the technology primary studies had a weighted average effect-size of 0.55, while the mathematics primary studies had a weighted average effect-size of 0.33.

Furthermore, the results show that various forms of small-group learning methods were effective in (a) promoting students’ attitudes towards STEM subject matters with a weighted average effect-size of 0.31, and (b) reducing students’ withdrawal level and failure in STEM college classrooms by 7%.

The findings of this research are consistent with and confirm previously reported and published meta-analytic findings and conclusions about the effectiveness of small-group learning methods in increasing students’ achievement in STEM college classrooms (e.g. Springer, Stanne, and Danovan, 1999; Johnson, Johnson, and Stanne, 2000).

Many primary studies were excluded from our meta-analysis because they were missing important summary statistics (e.g., means, standard deviations, sample sizes) needed to calculate the effect sizes. Consequently, the investigators and researchers of the effectiveness of various forms of small-group learning methods need to use better practices in reporting the results of the STEM primary studies. For example, reporting the basic descriptive statistics for both groups (mean, standard deviations, and sample sizes) and the pedagogical implementation practices in the STEM classrooms. Furthermore, our results show that a small number of primary studies in engineering and technology disciplines were implemented to assess and disseminate the results of the effectiveness of various forms of small-group learning methods compared to lecture-based instruction. Therefore, there is an urgent need to conduct more primary studies, especially in engineering and technology disciplines, to examine the effectiveness of small-group learning in STEM college classrooms.

In conclusion, the findings of the present meta-analytic study have shed some light on the accumulated literature of the effectiveness of various methods of small-group learning in STEM college classes., This study provided a positive answer to the effectiveness of various forms of small-group learning methods in comparison to the traditional lecture-based instruction in promoting higher STEM achievement, more positive attitude toward STEM subject matters, and increased persistence (retention) in STEM college classrooms. We learned that if students who are taking STEM classes in colleges are placed in an environment in which they can actively connect the STEM instruction to their previously learned materials and have an opportunity to experience collaborative and cooperative scientific inquiry, the academic achievement of these students in STEM courses will be accelerated. Based on the results of this study, it is important to note that STEM teachers and educators are recommended to use and implement any of the various small-group pedagogies and methods (e.g., cooperative, collaborative, team-based, problem-based, and inquiry-based) that have been shown to be effective in improving student achievement in STEM courses. Therefore, the current findings provide an evidence-based knowledge to future STEM researchers and educators, which will have significant educational policy implications in undergraduate STEM education.