Go to Admin » Appearance » Widgets » and move Gabfire Widget: Social into that MastheadOverlay zone
The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.
By Erik Devereux
July 12, 2016
My previous PA Times column criticized the current application of randomized control trials (RCTs) in the development of federal policies on poverty and inequality. I turn now to making a constructive suggestion for how related techniques of policy analysis and program evaluation may become much more useful for addressing those longstanding problems in our society.
One inspiration for the suggestion comes out of the insightful books by Nassim Nicholas Taleb (see Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets and The Black Swan: The Impact of the Highly Improbable). Taleb repeatedly makes a powerful case for replacing information-deprived assertions with information-rich empirical evidence NOT guided by our biased impressions of “how the world works.”
One of the most important defects of RCTs is that they most often do not provide information-rich empirical evidence of durable value to policy makers. On their own, RCTs do not magically escape the biases that Taleb so convincingly shows can make our conclusions about causal relationships unreliable. In particular, we should not trust our instincts or our professionally honed determinations about which combinations of policies to try in response to something as complex as poverty and inequality.
Here is an alternative approach, also based in rigorous experimentation, that would expand the information available about how to address poverty and inequality. When seeking to assess the net benefits of a set of policy interventions, federal policy makers would randomly assign combinations of those interventions across various subgroups of the affected population. The combinations would avoid interactions among interventions that contradict each other. Otherwise, a wide range of random combinations would be deployed, including some subgroups receiving just one intervention among the choices available and other subgroups conceivably receiving all of the interventions simultaneously.
Subgroups would be defined through combinations of demographic characteristics, socioeconomic measures and geography. The overall research might be intricate and detailed, but also capable of producing a very high quality “signal” of policy responses among the poor and disadvantaged.
A primary reason for this form of experimentation would be to empirically determine potential interaction effects among the interventions that simply are impossible to anticipate in advance. If across a range of subgroups there appeared to be a powerful, positive interaction among a specific combination of interventions, then additional research could be conducted by deploying that combination more broadly and studying the impacts in greater detail. The random assignment of combinations of interventions would assist with identifying factors that might otherwise go undetected in standard evaluations of one intervention at a time.
I have no doubt that some of the interactions that would emerge as worthy of further investigation would be surprising even to the most seasoned of social policy analysts.
Another important feature of this approach would be that no subgroup would be denied access to at least one intervention. American democratic policymaking remains highly resistant to the federal government explicitly designating a subgroup as a randomized control to be denied a federal benefit. In many situations, it simply is unethical to deny some form of benefit. That resistance and those ethical concerns have been major impediments to vastly expanding the highest quality empirical research on how to reduce poverty and inequality. Such impediments would be removed by a research design in which all subgroups received at least one intervention.
Finally, this approach might greatly increase the rapidity with which research could improve responses to poverty and inequality. In his excellent presidential address to the Association for Public Policy Analysis and Management, Professor Douglas Besharov at the University of Maryland showed how the glacial pace of social policy research using RCTs was itself undermining the value of such research in policymaking. A randomized design of combinations of interventions across subgroups would greatly accelerate the research process while pointing the way to additional, high-value research. To emphasize a key point, the federal government could deploy combinations of interventions that showed much promise while continuing to support efforts to better understand why the combinations worked.
The federal government needs to become more focused on empirical tests of innovative solutions to issues such as poverty and inequality. Taleb’s powerful insights show how important it is to use research designs that, “cast a wide net,” and open up our ability to see relationships in the world that we might have overlooked. By moving well beyond the limits of RCTs, it will be possible to make serious progress on poverty and inequality by leveraging the potential of research.
Author: Erik Devereux has worked for 25 years in the public policy and management field. Erik currently is an independent consultant to nonprofit organizations and to higher education and teaches applied policy analysis at American University. He has a B.S. from the Massachusetts Institute of Technology and a Ph.D. from the University of Texas at Austin. Contact Erik at [email protected].