Lessons Learned from Evaluating 21st Century Afterschool Programming During COVID-19: Key Elements for an Impactful Program

Written By: Brittany Hite

Project Team: Dr. Tiffany Berry (PI), Brittany Hite (PM), Emma Cohen (RA), and Evangelia Theodorou (RA)

Evaluation Project: Montebello Unified School District’s Extended Learning Opportunities

As 2020 unfolded, people around the world were impacted by the coronavirus pandemic, also termed COVID-19. This virus swept the nation, forcing everything but essential businesses to shut down. Montebello Unified School District’s Extended Learning Opportunity (MUSD’s ELO) was among the many programs affected, as youth were no longer allowed to attend in-person programming. While three-quarters of programs across the country underwent lay-offs, re-assignments, and furloughs, MUSD’s ELO realized youth needed staff and programming more than ever during this tumultuous pandemic. Thus, MUSD’s ELO leadership and program staff came together to adapt programming into an online format to continue providing emotional support, enrichment opportunities, and homework help for the thousands of youth at their sites. This blog aims to contribute to the growing body of knowledge and strategies for virtual programming, to support the essential role of after-school education and enrichment during the COVID-19 pandemic.

YDEval has been collaborating with MUSD’s ELO for the past nine years to conduct multi-site evaluations across 25 sites in Montebello, CA. Due to COVID-19, MUSD’s ELO moved to an online programming format in a two-pronged manner: (1) phone call check-ins with youth and families, (2) using Google Classrooms and/or other platforms like Zoom to interact with youth and answer questions, and (3) sharing district resources around COVID-19, sharing videos of activities they could do at home, and even making videos of themselves discussing activities youth could do at home. Although all sites had a Google Classroom and engaged in phone call check-ins, sites varied in their implementation of online programming (see picture to the right).

The following key take-aways emerged from this evaluation:

(1) Youth and family needs will remain hidden unless you seek them out.

During this time, connection is difficult. Before COVID, family needs came into the spotlight naturally through interactions at pick up, talking with youth, or touching base at events. There were many opportunities for touch points with youth and families. Now, during COVID, youth and parents are a lot less easily accessible. With this drop in connection, there has also been a decrease in understanding of the new and emerging needs youth and families are facing.

It is difficult to support families without a strong understanding of their current needs. Thus, it is imperative that staff and districts continue to find innovative and intentional ways to connect with youth and parents.

(2) After-school staff provided essential support to school districts during this time.

The shifting context brought up much uncertainty and a number of new responsibilities for everyone. Teachers and school districts were no exception. From trying to figure out how to transfer everything to an online context to trying to figure out what things would look like if things shifted back to an in-person context, school district personnel and teachers had their hands full.

While after-school program staff also were trying to figure out how to shift their programming to an online format, they realized some of the biggest needs from youth and families regarded classwork, access to district resources and information, and emotional support. Although none of these things were within staff’s roles or responsibilities, they realized they were best positioned to help. Thus, staff stepped up and started to meet youth and families’ remaining and emerging needs.

Some after-school programs in MUSD’s ELO even reached out to school districts and teachers to see how else they could support them in this continuous transition. After-school program staff are perfectly positioned to fill in these gaps and provide support to school districts and teachers given their established relationships with youth and families.

To help staff be most impactful at supporting the district, it is necessary for staff to have up-to-date district information and class information. Staff will not be as effective if they do not know what resources or information the district wants families to know. Additionally, teachers usually expect certain steps to be done when completing classwork problems or may have certain deadlines for youth. If staff do not know about these steps or deadlines, then they may not be as effective as they can be when providing support to youth.

(3) Even as you put out old fires, new ones may emerge.

All of this is new territory. That means solutions are constantly being created from scratch. It is important to remember that even as you create solutions, new challenges may arise as you implement these solutions. Even if programs and staff are anticipating potential pitfalls, it is likely that unforeseen challenges will still arise. Thus, it is important to consistently be checking in with youth, families, and program staff to see what new challenges are arising.

Programs can do this either formally through evaluation services and/or informally through phone call check-ins or brief surveys. The mindset with which you meet these challenges will also greatly impact your ability to effectively persevere in this time. Growth mindset notes that people who look at arising challenges as puzzles to be solved – rather than impeding obstacles – have more success in finding innovative solutions and perseverance.

(4) Best Practices for online programming.

A few key best practices also emerged from MUSD’s ELO’s own trial and error. We will touch on some of these best practices here but be sure to look into the other resources we have provided at the end of this blog to learn more.

Maintain structure: Youth’s whole world is in flux right now. Nothing seems to be set in stone. Maintaining some semblance of structure within your virtual programming can help to establish a sense of normalcy and consistency for youth. Staff at MUSD’s ELO did this by incorporating rules and guidelines for the Google Classroom and video calls, providing themed days (i.e., Motivation Mondays, Tasty Tuesdays, Wellness Wednesdays, Thoughtful Thursdays, and Fun Fridays), and setting up recurring meeting times with youth.

Emotional support: Youth are human beings first and students second. Just as COVID-19 has been mentally and emotionally tough on most people in America, it has been tough for youth as well. Youth have lost a lot of connection by not being able to be at school, seeing all their friends. They also lost a lot of ways they used to have fun and relax since most places of business are shut down. Additionally, most youth are stuck looking at their computer all day as they try to keep up with ongoing classwork tasks and deadlines. After-school programming is a key place to provide connection and emotional support. There are a few resources below about how to provide emotional support to youth during this time.

Re-imagine activities for an online setting: After-school program staff have done an amazing job figuring out how to provide activities in an online setting. While pivoting activities, it is helpful to think about:

  • Encouraging youth to do the activities themselves at their house
  • Discussing alternatives for resources youth may not have access to in their house
  • Providing activities that allow youth to leave the screen and do something outside
  • Providing activities that allow youth to express themselves through art and play

There are a few resources below that provide a lot of amazing ideas for virtual activities staff could provide to youth.

Preparation AND adaptability: Staff were most effective at providing fun and engaging activities when they prepared beforehand. Additionally, they were best able to troubleshoot on the fly when they pre-emptively brainstormed issues that may occur and how to combat them. At the same time, they noted the importance of being adaptable. No matter the amount of preparation, unforeseen issues will still arise. Thus, staff shared how important it is to have patience when issues arise, be prepared to think on your feet, and keep a light attitude when challenges occur.

Ensure activities are engaging: Youth have to sit at their computer all day these days. Shake it up and make things fun for them by creating engaging activities. Activities will be most engaging when they:

  • Are rooted in youth’s interests
  • Are fun and silly and/or allow youth to be fun and silly
  • Include hands-on activities that youth can try themselves
  • Apply to the real-world
  • Are working toward a goal
  • Include a competition or contest of some sort
  • Are short, simple, and clear
  • Include questions and prompts that make youth think
  • Include polls or forms that youth can directly respond to

Do not recreate the wheel: Program staff across America have been so innovative and creative in figuring out how to effectively provide programming and support for youth in these new times. A few organizations have started to pull together that information and bring in experts to talk about how best to continue to provide high-quality services to our youth and families.

Additionally, in a future blog post we will share a resource we compiled that contains key resources we have found to be most helpful around the following categories:

  • SEL tips and resources for COVID-19
  • Repository of activity ideas for virtual programming
  • Public forums where after-school program staff can find resources, crowdsource solutions, etc.
  • Guides on how to use specific platforms like Google Classroom and Zoom
  • Pre-made educational activities from organizations like NASA, National Geographic, etc.
  • Tips for talking with kids about COVID-19
  • Tips on serving justice-involved youth, English language learners, etc.