News

CHRT’s program evaluation team works with stakeholders to design, analyze, synthesize, and report on evaluations. They aim to improve programs by assessing progress, demonstrating impact, and developing lessons learned.

We sat down with Melissa Riba, the program evaluation team lead, to learn more about the team’s projects, purpose, and intended impact.

What are some of the challenges CHRT’s program evaluation team addresses?

The program evaluation team works with organizations that have identified a problem and created an intervention or program to address it. They’re doing great work, built on their evidence and experience, and they know they’re changing people’s lives. Their challenge is demonstrating that.

Evaluation can be a challenge for organizations because it’s a very time intensive, deliberate, systematic process. It’s a lot more extensive than simply identifying data or metrics, and it needs to be incorporated into early planning of a program to do it well. Often, organizations are trying to get a program up and running quickly and they don’t have the time or resources to plan evaluation. 

That’s where we can come in as consultants. We work hand in glove with the program to demonstrate impact and make the case for the program’s value – or to identify adjustments required to make the desired impact. This is often necessary to get continued funding to sustain or expand the program, because funders want to know the program is having the desired impact. As third party evaluators, we also provide the essential objectivity of an outside perspective.

As well as demonstrating value to funders and other stakeholders, we work on continuous quality improvement. By collaborating closely with the implementation organization, we identify areas where they should make mid-course adjustments. We provide quarterly data and information so they can see if they’re on track towards their targets, and discuss solutions if they’re not.

People working in health care and combating health disparities have a great deal of expertise and passion. Everyone undertakes these programs to improve people’s lives. We partner with them to demonstrate that they are indeed having an impact or consult on program adjustments to improve impact.

What projects are in the team’s portfolio now, and what do you anticipate their impact will be?

One project that we’ve been working on for five years is an evaluation of the behavioral health primary care integration process with the state of Michigan. We’ve been looking at how to integrate care and improve information sharing to improve care for the patient. As this project is winding down, we’re seeing a lot of impact.

We’re also working on a smaller pilot program in Detroit with an early childhood education center. Providers there observed that a lot of caregivers experience health disparities or behavioral health needs like postpartum depression. They’re piloting an evidence-based peer-to-peer program with a small cohort of moms to improve wellness, reduce depression and anxiety, and support parent-child relationships. On the program evaluation team, we often work with a small pilot to show outcomes and gather lessons learned, and then help our client make the case to expand it. This project is exciting because if this program does well the organization can expand the model to other parts of the state.

The program evaluation team has a robust portfolio, and we’re working on several other projects now as well, including evaluating the Certified Community Behavioral Health Clinics (CCBHC) in Michigan.

What processes does the team use to advance projects?

We use a culturally relevant and equitable evaluation framework for our projects. I’ve been doing evaluations for more than 25 years, and one common issue is that the exercise of evaluation is not always inclusive of a vulnerable community in a culturally relevant, equitable, appropriate way. Over the last few years, we’ve been incorporating this framework into all that we do to address this issue. We look through the perspective of the people being impacted by the program and seek input and guidance from people with lived experience. 

To achieve this, we focus on methods and how we collect data. We take into account health literacy and aim to understand where people are coming from. For example, when we were working on a project to integrate care in a pediatric practice, we initially wanted parents to take a brief survey. We realized that wouldn’t be appropriate because there may be multiple children present at an appointment, or children fussing, and the parents didn’t have time for a survey. Based on collaboration and feedback, we adjusted our method and had much more success. 

It comes down to being flexible. We work very collaboratively with our partners to make sure our evaluation is grounded in the reality of the work they’re doing and the community they’re working in.

We’re also really passionate about lessons learned. People get a little scared about evaluation, like you’re being judged. But that’s not our approach. We’re focused on the successes and the challenges, and how we can learn from them.

Everyone on our team is passionate about taking the hard work that our clients do on a daily basis and demonstrating its impact. Our team is very multidisciplinary, and our work is very applied. We take on the really messy projects. Our interns often say that our work is very different from what they learned in their evaluation classes, because we’re grounded in the real world and focused on adding value without getting in the way of our clients’ important work. It’s so exciting when we can see the impact immediately.

Health and Social Equity team members include: 

• Marissa Rurka, who works on projects related to Law Enforcement Assisted Diversion and Deflection (LEADD), healthy aging, family caregiving, and health-related social needs.
• Jonathan Tsao, who works on projects including LEADD and Certified Community Behavioral Health Clinics (CCBHC) evaluation.
• Poonam Yadav, who is involved in LEADD and the Michigan Physicians Survey project.
• Sheritha Rayford, who is evaluating Michigan’s CCBHCs.
• Jadrienne Horton, an intern working on the LEADD program.
• Jennie Scheerer, the 2023 Rebecca Copeland Memorial Intern.