Program Performance Measures: A Basic Primer

The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.

By Amy Johnson
October 9, 2015

This month’s column provides a basic primer on program performance measures. It will discuss some of the frequently used terms and highlight resources designed specifically for public sector and nonprofit managers to use in developing sound performance measurement strategies.

There are varied perspectives on what should be considered when evaluating a program or activity. At the most basic level, the strategy should contain measures that reflect inputs, outputs and outcomes. When taken together, these measures support an understanding of the performance of different elements that comprise the system that is to be evaluated. Managers frequently use logic models to aid in the identification of relevant measures from each of these categories and outline the relationships among them.

Input measures capture resources applied to an activity or program. Examples might include dollars invested, level of effort or parts supplied. Output measures quantify what is produced by an activity or program. Output is represented in units and can be counted. The number of parts produced or people affected are examples of output measures. Outcome measures describe the degree of progress made toward achieving the goals of the activity or program. Outcomes are sometimes represented as a percentage of the objective that has been achieved.

Logic models identify the linkages among inputs, activities or programs, outputs and outcomes associated with the activity or program. In creating a logic model, a manager first defines the objectives to be achieved by undertaking the program or activity that is to be measured. Next, the manager identifies the resources that will be invested in that program or activity and the units that will be used to quantify them. Then the manager anticipates what the activity or program will produce and identifies the units that will be used to quantify this production. Finally, the manager considers how he or she will determine which objectives have been achieved, what progress has been made and the tipping point at which the program or activity has produced sufficient output to merit program attainment.

The logic model helps the manager consider the impacts that changes to inputs might have on outputs and ultimately outcomes. Because the logic model outlines the relationships across metrics, managers are able to view the different components of measurement as a system. For instance, a comparison of inputs and outputs can provide managers with useful information like cost per unit. A comparison of outputs over time may provide the manager with useful trend information related to performance such as a decrease in the number of defective parts, which could translate into an increase in quality.

When the manager makes a change to inputs, the logic model helps define anticipated consequences with respect to outputs and outcomes. If the anticipated consequences are not realized, the manager then knows that there are factors at play within the model that have not yet been defined and can do the necessary research to identify them and their place within the system. These factors may be the outputs of related programs or activities which have high potential to influence the targeted activity or program but over which the manager has no control.

The W.K. Kellogg Foundation’s “Logic Model Development Guide” is a good reference for managers of social programs seeking to identify strategies for assessing program impacts. The guide characterizes the logic model as “a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve.” When unpacked, this definition describes 1) a thoughtful plan that 2) identifies relationships and 3) creates a common understanding of purpose and expectations among stakeholders.

Another useful resource on the fundamentals of program evaluation is the Handbook of Practical Program Evaluation. Edited by Kathryn E. Newcomer, Harry P. Hatry and Joseph S. Wholey, this handbook provides an overview of a number of key aspects of program evaluation and is designed specifically for use by public sector and nonprofit managers.

A final resource on performance management topics for managers of social programs is Mario Morino’s Leap of Reason: Managing to Outcomes in an Era of Scarcity. This book is a collection of thoughtful essays that highlight best practices for assessment.

Effective evaluation begins with the development of a performance measurement framework. This framework creates a plan for what the manager intends to measure and how different aspects of the measurement strategy relate to one another. Logic models are useful tools for organizing measures into a framework and establishing a measurement strategy.

---

Author: Amy Johnson has almost two decades of experience working with public and private sector clients in the areas of program and process evaluation, strategic planning and organizational performance management. She serves as the director for nonprofit and civilian government programs at Coverent, a consulting firm focused on helping clients to use data analytics to measure program performance and maximize mission impact. Contact at [email protected].

(1 votes, average: 5.00 out of 5)
Loading...