Lessons Learned from Evaluating Afterschool Sports Programming: Outcomes and Key Elements to Maximize Outcomes

Written By: Haley Umans

Project Team: Dr. Tiffany Berry (PI) and Dr. Michelle Sloper (PM), Haley Umans, Yaqiong Wang, Grace Skiles

Evaluation Project: Three-Year Impact Evaluation of LAUSD Beyond the Bell Sports Programs: Linking Quality with Outcomes

Many would agree that youth participation in organized sports is a positive thing – there is a reason that organized sports are the most popular out-of-school (OST) time youth program activity.[1] Scholars and practitioners are aware that sports are naturally engaging, structured, enjoyable, and have the potential to benefit youth. However, as OST sports programming has grown and gained popularity, the conversation about how youth actually benefit from sports and how programs can curate high-quality sports experiences for youth has been limited.[2]

Recently, YDEval completed a three-year evaluation (2016-2019)  in collaboration with the LA84 Foundation and Los Angeles Unified School District (LAUSD)’s Beyond the Bell (BTB) afterschool program. For this project, we evaluated the impact of afterschool sports and investigated the key program elements required to make an impact. To understand the benefits for youth, we conducted a quasi-experimental evaluation (i.e., comparing different groups of students, but students were not randomized into the groups) of the afterschool sports programming delivered by BTB to over 100 low-resourced middle schools (6th – 8th grade) in Los Angeles, CA. The evaluation used multiple comparison groups and tracked the students over the three years to understand how youth benefitted from sports and why they benefited. This blog post will summarize the evaluation findings around the impact of sports and discuss the key elements of an impactful sports program. The design and methods of the evaluation are discussed in another blog post.

Impact of Sports

This evaluation sought to understand the impact of OST sports programming on physical health, social-emotional, and academic outcomes. This data was sourced from the district, including district academic data (i.e., grades and test scores) or a district-wide surveys. Because we had multiple comparison groups and multiple years of data, this blog will highlight findings that represent consistent trends observed.

Physical Health Outcomes: Benefits for Sports Participants

We examined physical health outcomes through student pass rates on fitness tests (FITNESSGRAM) administered by the school district. Sports participants consistently did better on the district physical fitness tests compared to non-participants. Overall fitness test pass rates from 7th graders in 2018-2019 are highlighted.

Social-Emotional Learning Outcomes: Some Benefits for Sports Participants

Social-emotional learning (SEL) outcomes were included on the  district-wide survey, including connectedness, social awareness, self-management, growth mindset, and self-efficacy. Sports participants reported higher levels of self-efficacy and school connectedness, but lower social awareness and self-management compared to non-participants. Average levels of self-efficacy are highlighted in the figure.

Academic Outcomes: Limited Benefits for Sports Participants  

We examined academic outcomes by looking at grades, state test results, and school-day attendance. Test scores were typically lower for sports participants and there were no differences in grades for sports participants compared to non-participants. However, sports participants consistently attended more days of school, ranging from 3.1 to 13.5 more days compared to non-participants. This is beneficial for schools who lose money when students miss school and beneficial for students who will have more learning opportunities. Average days of attendance in 2017-2018 are highlighted in the figure.

Although many academic indicators did not favor the sports participants, it is worth noting that academic outcome are least aligned with the program content; therefore, we would expect fewer benefits for sports participants in this domain.

The evaluation highlighted that sports participants compared to non-participants demonstrated a number of physical health, SEL, and school attendance benefits; however, this evaluation also sought to understand how and why these benefits were achieved.

Key Elements to Achieve Impact of Sports

What key elements should be in place in a sports program for youth to promote positive youth development for youth participants? Often, programs assume that access to afterschool sports alone will lead to some sort of benefit. This assumption often results in programs missing the key ingredients and processes that need to be in place to positively impact youth through sports.

Moving beyond access, there are three key elements associated with an impactful afterschool sports program as demonstrated by our evaluation: (1) well-trained and supported coaches who can implement a high-quality program, (2) youth having sufficient levels of participation in the program, and (3) youth having an optimal experience. This section explains what these key elements entail and how they influenced youth outcomes in our evaluation study.

Well-Trained & Supported Coaches Implement a High-Quality Program

Committed and qualified coaches must be trained and supported to implement a high-quality sports program. In our evaluation, youth from high-quality program sites had higher academic, physical, and social-emotional outcomes compared to non-participants. This begs the questions – what does high-quality training for coaches look like? And what does it mean for a coach to implement a high-quality sports program? We must consider what it means to offer quality programming afterschool, and in the context of sports, specifically.

In the afterschool setting, coaches’ training should align to high-quality practices for expanded learning opportunities. According to the California Department of Education, high-quality practices for expanded learning programs include: (1) creating a safe and supportive environment, (2) promoting active and engaged learning, (3) creating opportunities for skill building, (4) fostering opportunities for genuine youth voice and leadership, (5) promoting well-being with healthy choices and behaviors, and (6) diversity, access and equity. These features represent point-of-service quality standards, meaning that these are the critical components of the direct interactions between youth and any outside of school time program.

In addition to these features of program quality, sports programs must also consider best practices in sports-specific programming. Notably, positive coaching techniques, intentional sports skill development, and opportunities for authentic leadership roles are key for youth to make the most of their time in sports programming. In the research and evaluation literature, these overarching components were identified as important influences on youth development in sports.

Positive Coaching: Youth will benefit more when coaches use techniques aimed at fostering psychological safety in the program environment (i.e., trust, respect, engagement) and promote the mastery of important skills (i.e., focus on effort and self-improvement rather than performance).[3] Youth have better developmental experiences when there is a focus on performance and self-improvement, rather than on winning and social comparison.[4] Positive coaching practices are also related to better intrinsic motivation, persistence, and development of positive life skills for youth.[4], [5], [6]

Sports Specific Skill Development Opportunities: While opportunities for skill development is important for any afterschool program activity, there are specific practices that can be employed to build sports specific skills. For example, youth are more likely to develop skills through sports when coaches are explicit about the purpose of the activities, allow opportunities to practice skills (particularly in competition), model and demonstrate skills themselves, provide individualized feedback, believe youth can improve, and encourage youth to be better (i.e., S.A.F.E. features).[7] These practices effectively promote the acquisition of skills in youth.6  Beyond practices to build technical skills (e.g., sports rules, techniques, strategies), coaches should also create opportunities to build social and emotional skills during sports, such as mental preparation for competition and rapport with coaches.

To ensure coaches are effectively trained in these concepts and approaches, training opportunities must occur frequently in an ongoing manner. Many programs only conduct training when a new employee is hired or on a yearly basis at the start of a new school year. However, to offer the most effective support and ensure coaches have the necessary knowledge and competence, professional development for coaches should be an on-going and interconnected process. Training for sports should go beyond a review of sports rules and procedures to include training on how to enact elements of program quality and promote broad skill development. Ongoing activities may include formal training sessions, on-site coaching for coaches, data collection to track coaches’ progress and a focus on areas for improvement.

Youth Attend the Program Consistently

For sports programs to maximize the impact on youth, youth need to be recruited into the program and attend the program consistently over time. In our evaluation, we collected data on the number of days youth participated in the program, the number of years youth participated in the program, and the number sports youth participated in. Specifically, we looked if outcomes varied by the following participation patterns:

  • Days of Attendance: Low (20 days or less) vs. Medium (21 – 50 days) vs. High (51 days or more)
  • Number of Sports: Participation in one sport vs. participation in multiple sports in a year
  • Number of Years: Participation in one year of sports vs. participation multiple years of sports

The evaluation found that youth who had consistent and high levels sports attendance over multiple years had greater outcomes. For example, sports participants who are high attendees did consistently better than low attendees on physical fitness pass rates, days of school attendance, and social-emotional outcomes (self-efficacy, growth mindset, and school connectedness).

Research on the recruitment and retention into OST programs suggests that high-quality programming (e.g., safe and supportive environment, well-trained staff) that incorporates the voicse of youth in the programming decisions will bring students into the program and encourage them to stay.[7] Incorporating attendance incentives (e.g., competitions with other schools, leadership roles), family involvement in the program, and opportunities to connect to the community can also help support retention.9

Youth Receive Quality Programming with Optimal Experiences

Even if coaches are trained to implement a high-quality program, youth must also have opportunities to engage in high-quality experiences in sports. In the evaluation, sports participants completed a survey regarding their program experiences including experiences with positive coaching, skill development, youth voice and leadership, and satisfaction with the program. Higher ratings for positive youth experiences were consistently related to and predictive of  greater outcomes for youth. Incorporating opportunities for youth voice and leadership was particularly predictive of many of the positive youth development outcomes examined in this study. This further supports the notion that ongoing, high-quality training for coaches, consistent and sustained program quality, and high levels of program attendance can bolster the ways youth benefit from sports programming.

Conclusion

In short, OST sports programming has the potential to make a positive impact on physical health, social-emotional, and academic outcomes for youth. This is particularly true if sports programs consider the key elements of a high-quality sports programming and intentionally create programming that is consistent with these key elements. While this may seem like a tall order, please consider the following takeaways that you may apply to your own program.

  1. Be Intentional: Intentionally consider and align coaches training opportunities, high-quality program implementation, youth attendance, and optimal youth experiences to maximize how youth benefit from sports.
  2. Invest in Coaches: Coaches are the driver of youth development. Training for coaches should go beyond sports-specific skills to include how to engage with youth around positive coaching, developing whole-child outcomes, and encouraging positive youth development. Continuous feedback for coaches should be used by the program leadership to support opportunities to reinforce positive practices and support the implementation of lessons learned from training.
  3. Engage in Evaluation: Evaluation can help sports programs gain a more nuanced understanding of program impact by collecting evidence that can be used to share impact, inform decision making, and support program improvements. These efforts build opportunities to better promote positive development for youth over time.

References:

[1] Larson, R. W., & Verma, S. (1999). How children and adolescents spend time across the world: work, play, and developmental opportunities. Psychological Bulletin, 125(6), 701.

[2] Lee, O., Park, M., Jang, K., & Park, Y. (2017). Life lessons after classes: investigating the influence of an afterschool sport program on adolescents’ life skills development. International Journal of Qualitative Studies on Health and Well-being, 12(1), 1307060.

[3] Fraser-Thomas, J. L., Côté, J., & Deakin, J. (2005). Youth sport programs: An avenue to foster positive youth development. Physical Education & Sport Pedagogy, 10(1), 19-40.

[4] Gould, D., Flett, R., & Lauer, L. (2012). The relationship between psychosocial developmental and the sports climate experienced by underserved youth. Psychology of Sport and Exercise, 13(1), 80-87.

[5] Coatsworth, J. D., & Conroy, D. E. (2007). Youth sport as a component of organized afterschool programs. New Directions for Youth Development, 115, 57-74. DOI: 10.1002/yd

[6] Larson, R. W. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170–183. doi:10.1037//0003-066X

[7] Petitpas, A. J., Cornelius, A. E., Van Raalte, J. L., & Jones, T. (2005). A framework for planning youth sport programs that foster psychosocial development. The sport psychologist, 19(1), 63-80.

[8] Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010). A meta‐analysis of after‐school programs that seek to promote personal and social skills in children and adolescents. American journal of community psychology, 45(3-4), 294-309.

[9] Gillard, A., & Witt, P. (2008). Recruitment and Retention in Youth Programs. Journal of Park & Recreation Administration, 26(2).