How to Measure the Performance Measurements

The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.

By Minch Lewis
November 25, 2014

Measuring performance in government is challenging. We need to consider the metrics, the performance contributing to the metrics and the outcomes of the performance. As examples, we will look at performance measurement in education and law enforcement.

In education, the Common Core State Standards Initiative attempts to improve educational outcomes by improving teacher performance based on a system of measurements. We are engaged in a national debate on the performance measures that count for education.

In law enforcement, a police department counts its arrests. But is success a high number or a low number?  Nobody wants to live in a neighborhood where the police are making numerous arrests. Therefore, a high number of arrests can be a negative indicator. This statistic may work against a city’s more important goal of attracting residents to a neighborhood. That does not mean that arrests should not be counted as an operational metric. It means that arrests are not a metric for measuring performance if the goal is to strengthen a neighborhood. We should be counting other indicators that affect security.

Another difficulty in counting arrests is the “whac-a-mole” effect. A concentrated law enforcement effort in one neighborhood can be effective in eliminating crime in that neighborhood. The criminal activity merely moved to another neighborhood.

Measuring performance, “getting it right,” is critically important for resource allocations. Budgets typically are set each year. In a rational system, resources would be allocated based on performance data. Measuring performance is the key. So how do we measure the measurements?

Performance measurement begins with strategic goals.

Measuring operational data like dropout rates and arrests isn’t sufficient. Performance should be measured as it relates to the strategic goals. Strategic performance metrics include:

• Monitoring the implementation and effectiveness of an organization’s strategies.
• Determining the gap between actual and targeted performance.
• Estimating an organization’s  effectiveness and efficiency. 

Here are some proposed metrics for measuring the performance of the performance management system.

Strategic goals should be identified.

The first criterion for a performance measurement system is a set of strategic goals. Creating strategic goals is not an easy task. In a 2014 Harvard Business Review article titled “A list of goals is not a strategy,” Graham Kenny explains, “Without the support of stakeholders such as customers, suppliers, employees and shareholders, for example, you have no organization. But you have to identify those who are key to the long-term survival and prosperity of your organization — and then satisfy them.” Strategic goals in government should be based on the same considerations.

The achievement of those strategic goals should be measured.

Robert Behn made the case in a 2003 Public Administration Review article titled “Why Measure Performance.” He also pointed out the predictable consequences when measurements exist in a strategic vacuum. “No wonder many public managers are alarmed by the evaluative nature of performance measurement. If there existed a clear, universal understanding of their policy objectives, and if they could manage within a rational program structure, they might find performance measurement less scary.” 

This observation may have been missed in the development of the Common Core State Standards Initiative. In a Center for American Progress article, “Using Teacher Evaluation Reform and Professional Development to Support Common Core Assessments,” Peter Youngs outlines the challenges in the process. Many of the details for measuring teacher performance are being developed as the Common Core Initiative is being implemented.

The goals should also be end-points.

Unintended consequences, like “teaching to the test,” should be eliminated from the formulation of the strategic goals. This is particularly difficult in the Common Core Initiative.

Performance measurements should be comprehensive in scope.

The “Balanced Scorecard” is a systematic approach to performance measurement. It was developed at the Harvard Business School by Robert Kaplan and David Norton. It goes beyond the simplistic notion that results are measured solely in financial terms. The model calls for strategic goals that specify traditional financial success but also consider:

• What must be done to meet customers’ expectations.
• What business processes must be used to accomplish those activities.
• What a public agency does to learn and grow.

A “Balanced Scorecard” in education would include significant variables like the effects of concentrated poverty on teacher performance.

Measurements should be selected in a rational process.

The “Balanced Scorecard” approach has been formalized in nine steps. They spell out a logical sequence in developing a performance management system. Whatever process is used should be documented.

Each of the measurement metrics should be:

• Reliable—accurately linked to reality. They should be verifiable by multiple sources generating substantially the same results. 

• Timely—available in a period that is connected to decision-making. Data collection should be current. 

• Consistent—stable over time. The same factors should be measured from each period to provide a base for comparison. 

• Measurable—quantifiable. The metrics should be expressed in numeric values rather than abstract concepts. 

• Relevant—linking cause and effect. What is being measured should be a significant factor in achieving the strategic goal. 

• Helpful—used to improve performance. Analyzing the metrics can lead to decisions that enhance the achievement of the strategic goal. Communication is a key dynamic. The metrics need to be communicated to stakeholders and participants in terms that are meaningful. One option is a data “dashboard” that provides immediate access to critical performance data. 

• Collaborative—created in an environment that transcends the distinction between the evaluator and the evaluated. A teamwork orientation may be the most important characteristic of a performance measurement system. Governmental performance measurement systems should be designed by all participants working together. The alternatives are unpleasant. Look at the frustration over the way Common Core has been linked to teacher evaluations or consider the falsification of performance data recently uncovered in the Veterans’ Administration. 

The performance measurement system should be evaluated.

Performance measurement systems in local governments should be evaluated by an internal audit function. Performance measurements influence the most significant political decisions about resource allocation. Jobs and lives can be affected. The metrics described above are a starting point.

---

Author: Minch Lewis is a part-time instructor at Syracuse University’s Maxwell School. He served as the elected city auditor in Syracuse, NY, for nine years. He has extensive experience nationally in developing and installing financial management systems for the affordable housing industry. He earned his master’s degree in Public Administration at the Maxwell School. He holds a certificate from the Association of Government Accountants as a Certified Government Financial Manager.  He can be reached at [email protected].

(3 votes, average: 4.33 out of 5)
Loading...