Evaluation and Performance Measurement Plans can be an intimidating aspect of publicly funded HIV prevention work. Let’s look at some of the reasons why evaluation can be your friend, and then break down some terminology to focus in on a few key processes that are relevant for typical community based HIV prevention work.
CDC Framework for Program Evaluation
CDC’s framework for evaluation in public health emphasizes practicality and the inclusion of stakeholders in program development, evaluation, measurement and improvement. In other words, as illustrated in the figure below, the process starts and continues with direction from those served, staff, funders, community partners, etc.
In the context of HIV prevention programs stakeholder priorities might be adequately addressed through common required performance measures, or with a few added measures that are complementary to those mandated by a government funder. There is an expectation of consistency and accuracy in gathering data and reporting findings, while recognizing that agencies are rarely funded or staffed to do research or extensive evaluation. Most agency-level EPMP work is technically defined more as monitoring rather than evaluation. Funder provided data management systems such as CareWare and EvaluationWeb are typically used to facilitate consistent data collection across agencies.
Some Basic Terminology for the Field
CDC’s Capacity Building Assistance Provider Network offers a Fundamentals of Grant Writing course which includes key terminology that is helpful in conceptualizing some important elements of EPMP.
Formative assessment usually is completed before an organization commits to a new program to inform its design and related planning.
Process monitoring occurs during a program and looks at what is being done, with who, and how much.
Process evaluation occurs during a program and compares what is being done, with who, and how much with agreed upon objectives. Asks: did we meet our process objectives (i.e. did we carry out the specific activities we said we would)?
Outcome monitoring occurs during and after a program and examines if there was any change in participants, based on program goals. Asks: did we meet our outcome objectives (i.e. did participants change behavior or knowledge in the expected ways)?
Outcome evaluation is usually not applicable in the EPMP work of many organizations funded for HIV prevention activities. Outcome evaluation occurs during and after a program and compares participant outcomes with the same measures in a group of people that did not participate in the intervention or prevention activity. This is more in the realm of experimental, quasi-experimental or other research methodology. Most HIV prevention organizations are not funded to perform research or this type of program evaluation.
Impact evaluation occurs after the programs have been completed and analyzes the long-term impact on the key health outcomes, HIV incidence for example. Usually health departments conduct this type of evaluation.
Relax, It’s Easy Peasy
EPMPs do require an investment of time, attention to detail and some technical capacity building. When a community and agency “own” the evaluation cycle, it can be a welcome and demystified process, serving as a powerful tool for ongoing stakeholder engagement and improved program outcomes.
https://www.cdc.gov/hiv/programresources/evaluation.html Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention), Updated May 31, 2018.
Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR 1999;48(No. RR-11):[p.4].
https://www.cdc.gov/hiv/pdf/funding/announcements/ps18-1802/CDC-HIV-PS18-1802-AttachmentC-EPMP-1Pager.pdf Overview of PS18-1802 Evaluation and Performance Measurement Plans (EPMPs)
Fundamentals of Grant Writing Course (CDC with CAI, Latino Commission on AIDS, Hands United, JSI, PROCEED, Inc., National Minority AIDS Council, National Community Health Partners, Capacity For Health), Revised June 2019.