The mid-point of any new initiative is a pivotal time, presenting us with the opportunity to look back and reflect on our early successes and challenges. Evaluations can play an important role in this process; the right kind can reveal valuable insights about what’s working and what’s not. More critically, evaluation can help us identify ways to address program challenges, refine our approach, and ultimately deepen our impact over time.
Recently, The Improve Group team completed a mid-term evaluation for the Rural Information Technology Alliance (RITA) initiative, an innovative collaborative consortium of four colleges working to meet the information technology needs of rural areas. Funded by the U.S. Department of Labor Trade Adjustment Assistance Community College and Career Training (TAACCCT) Grants Program, RITA is part of a major federal effort to help prepare adults for high-wage, high-skill employment while reducing workforce shortages.
To conduct this mid-term evaluation, we focused on several key questions exploring how successfully RITA is being implemented, to what extent RITA is reaching program goals, and whether the program can be sustained after the initial grant period. We collected extensive qualitative and some quantitative insights to understand the perspectives of involved faculty and program leaders on how RITA is operating. We also have been employing a quasi-experimental design, recruiting a comparison group to better assess the outcomes of RITA students over time. Preliminary data have revealed some important lessons about RITA’s work so far:
Early Successes: The evaluation identified many ways where RITA is being implemented successfully and already making a difference, on campuses and in communities. Strong program leadership, effective program communications, and a commitment to supporting students and faculty have helped RITA reach its initial benchmarks for enrollment. In addition, program leaders are building strong relationships in the technology industry, priming potential employers for hiring RITA students.
Ongoing Challenges: It was also revealed that some challenges and roadblocks are preventing RITA from achieving all of its first and second year milestones. While students are expressing interest in RITA programs of study, they are not yet completing these programs’ information technology certifications. Additionally, recruiting faculty to be a part of the RITA program has proved to be challenging on some campuses, and bureaucratic processes at both the college and federal level slowed program implementation and advancements to existing curricula.
Looking Ahead: To help program leaders address these challenges and leverage their successes, our team, in partnership with RITA leadership, identified a set of strategies to increase program impact in the second half of the grant period. One recommendation is a renewed focus on recruiting students from target populations, like incumbent workers. In response, RITA staff are developing new strategies to attract incumbent workers by combining customized training with traditional degree programs and by focusing marketing and recruitment efforts to recruit from those populations. Though there is a robust data management system that is now in place to track program implementation and student success, improvements to data entry strategies should be adjusted to help refine the program’s picture of success.
According to Michael Olesen, RITA Consortium Director, the roll that The Improve Group, as third-party evaluator, has played in the success to date of the RITA consortium cannot be underestimated. “In a grant where we are to begin to have impact almost immediately, having someone there to help measure our performance through a complete and non-biased process has provided us with validation regarding what we are doing right and identified where we need to make adjustments. The mid-term evaluation was a useful tool to update our stakeholders, both inside our colleges and at the federal level, on our activities. At the college-level, we were able to use this report to bring important issues to the attention of senior leadership.”
Regardless of the size and scope of an initiative, a mid-term evaluation can reveal innovative strategies and identify the best levers to boost efficiency and impact. Evaluation isn’t just something you do at the end to see how things went—it can highlight ways to transform challenges and ensure program success all along the way.
Click here for additional information about the project.