Public Administration Scholarship and the Epidemic of Academic Fraud and Dishonesty Part 5: Restoring the Sanctity of Basic Research
The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.
By Erik Devereux
February 23, 2024
This column is the fifth in a series on the epidemic of academic fraud and dishonesty across the social sciences that encompasses public administration (the first column; the second column ; the third column; the fourth column). One of the fundamental issues at play in this unfolding disaster (and here is some of the latest news on this topic pertaining to medical science) is the continuing decline of values in the university regarding the sanctity of basic research. Under a barrage of ideological attacks, universities over the past decades have pivoted away from “controversial research” or “useless research” and towards politically palatable applied research. The messaging now emphasizes to the public that universities are places that generate immediately commercializable solutions to noncontroversial problems. Erectile disfunction, anyone?
Considerable evidence exists that universities are not good incubators for commercial products or services. Their best function is basic research, the long-term value of most probably being very low, but some efforts end up generating trillions of dollars in economic activity or solving vexatious problems that previously seemed intractable. And it is almost impossible to predict in advance how this will play out.
Universities must be places where highly intelligent, tenured faculty have free reign to work on any problems they find interesting without regard to politics or economics. If you disrupt that basic research process, you cannot be surprised when negative consequences arise inside and outside of academe, such as systematic fraud.
Here is one of my favorite examples. When I joined the faculty of the public policy school at Carnegie Mellon University in 1991, serious tension existed between a senior faculty member and the dean at the time. The source of tension was that the senior faculty member (with tenure) had become fascinated with the problem of ranking competitive athletes (in this case professional golfers) in a way that produced a fundamentally better result than all other existing rankings. That faculty member gently suggested that if one golfer was ranked above another, then whenever there was head-to-head competition, the higher ranked player should defeat the lower ranked player. This may sound obvious but every published ranking at that time could not produce that result.
What the researcher then realized was that it was necessary to simultaneously rank the difficulty of golf tournaments (based on the quality of golfers who chose to compete in each) and the quality of the individual golfers. The researcher developed an iterative algorithm that did this estimation and it produced extraordinarily high-quality results. The resulting rankings did predict the outcomes of head-to-head competition. A key insight here is that some professional golfers stay away from the most competitive tournaments because they know they cannot succeed against a strong field and thus are unlikely to win much money. Worse yet, any professional golfer who performs poorly in numerous tournaments might forfeit the right to play at all.
The dean was livid and openly opposed having a senior faculty member of a public policy school researching golf statistics. While completely understandable from a political perspective, from an intellectual perspective, that anger proved to be completely misplaced.
Soon after developing the ranking algorithm, the senior faculty member and some colleagues realized that this solved the problem of adjusting the GPAs of undergraduates based on the difficulty of the college courses they elected to take. The students were the “players” and the classes were the “tournaments”. Those students who had doubts about their intellectual abilities could be avoiding the tough classes to protect their GPAs. When the algorithm was applied, one of the results was to adjust GPAs in a way that greatly increased the correlation between GPAs and SAT scores. Two major universities actively considered adopting the technology but were blocked by concerted opposition among the faculty in the humanities. Remember: The algorithm also ranked the courses, clearly communicating to the university administration which subjects were the easiest to get good grades in.
Then those same researchers realized that this approach also provided an alternative way to rank airlines based on their on-time arrivals. In this instance, the airports are the tournaments and airlines are the players. Some airports are much more difficult to fly into than others; adjusting the on-time statistics produced a more reliable measure of a factor with much influence over consumer choices in air travel. Consumers could know that a higher ranked airline actually is more likely to get its flights into those airports on time compared with lower ranked rivals.
There are numerous famous examples of how basic research like this, shielded from political and economic pressures, had similar unexpected applications. But if basic research becomes highly scrutinized by bureaucrats more than by scholars, subject to constant political harassment, and otherwise held to unreasonable standards, we lose access to the powerful overall benefits. We eat the fictive “seed corn” and then wonder why there is no food later. And we reward fraudulent research because we no longer value and protect the basic research enterprise. As the kids say, “good luck with that.”
Author: Erik Devereux is a consultant to nonprofits and higher education and is adjunct faculty in the Security Studies Program at Georgetown University. He has a B.S. from the Massachusetts Institute of Technology (Political Science, 1985) and a Ph.D. from the University of Texas at Austin (Government, 1993). He is the author of Methods of Policy Analysis: Creating, Deploying, and Assessing Theories of Change (available for free here). Email: [email protected]. More content is available here.
(No Ratings Yet)
Loading...
Public Administration Scholarship and the Epidemic of Academic Fraud and Dishonesty Part 5: Restoring the Sanctity of Basic Research
The views expressed are those of the author and do not necessarily reflect the views of ASPA as an organization.
By Erik Devereux
February 23, 2024
This column is the fifth in a series on the epidemic of academic fraud and dishonesty across the social sciences that encompasses public administration (the first column; the second column ; the third column; the fourth column). One of the fundamental issues at play in this unfolding disaster (and here is some of the latest news on this topic pertaining to medical science) is the continuing decline of values in the university regarding the sanctity of basic research. Under a barrage of ideological attacks, universities over the past decades have pivoted away from “controversial research” or “useless research” and towards politically palatable applied research. The messaging now emphasizes to the public that universities are places that generate immediately commercializable solutions to noncontroversial problems. Erectile disfunction, anyone?
Considerable evidence exists that universities are not good incubators for commercial products or services. Their best function is basic research, the long-term value of most probably being very low, but some efforts end up generating trillions of dollars in economic activity or solving vexatious problems that previously seemed intractable. And it is almost impossible to predict in advance how this will play out.
Universities must be places where highly intelligent, tenured faculty have free reign to work on any problems they find interesting without regard to politics or economics. If you disrupt that basic research process, you cannot be surprised when negative consequences arise inside and outside of academe, such as systematic fraud.
Here is one of my favorite examples. When I joined the faculty of the public policy school at Carnegie Mellon University in 1991, serious tension existed between a senior faculty member and the dean at the time. The source of tension was that the senior faculty member (with tenure) had become fascinated with the problem of ranking competitive athletes (in this case professional golfers) in a way that produced a fundamentally better result than all other existing rankings. That faculty member gently suggested that if one golfer was ranked above another, then whenever there was head-to-head competition, the higher ranked player should defeat the lower ranked player. This may sound obvious but every published ranking at that time could not produce that result.
What the researcher then realized was that it was necessary to simultaneously rank the difficulty of golf tournaments (based on the quality of golfers who chose to compete in each) and the quality of the individual golfers. The researcher developed an iterative algorithm that did this estimation and it produced extraordinarily high-quality results. The resulting rankings did predict the outcomes of head-to-head competition. A key insight here is that some professional golfers stay away from the most competitive tournaments because they know they cannot succeed against a strong field and thus are unlikely to win much money. Worse yet, any professional golfer who performs poorly in numerous tournaments might forfeit the right to play at all.
The dean was livid and openly opposed having a senior faculty member of a public policy school researching golf statistics. While completely understandable from a political perspective, from an intellectual perspective, that anger proved to be completely misplaced.
Soon after developing the ranking algorithm, the senior faculty member and some colleagues realized that this solved the problem of adjusting the GPAs of undergraduates based on the difficulty of the college courses they elected to take. The students were the “players” and the classes were the “tournaments”. Those students who had doubts about their intellectual abilities could be avoiding the tough classes to protect their GPAs. When the algorithm was applied, one of the results was to adjust GPAs in a way that greatly increased the correlation between GPAs and SAT scores. Two major universities actively considered adopting the technology but were blocked by concerted opposition among the faculty in the humanities. Remember: The algorithm also ranked the courses, clearly communicating to the university administration which subjects were the easiest to get good grades in.
Then those same researchers realized that this approach also provided an alternative way to rank airlines based on their on-time arrivals. In this instance, the airports are the tournaments and airlines are the players. Some airports are much more difficult to fly into than others; adjusting the on-time statistics produced a more reliable measure of a factor with much influence over consumer choices in air travel. Consumers could know that a higher ranked airline actually is more likely to get its flights into those airports on time compared with lower ranked rivals.
There are numerous famous examples of how basic research like this, shielded from political and economic pressures, had similar unexpected applications. But if basic research becomes highly scrutinized by bureaucrats more than by scholars, subject to constant political harassment, and otherwise held to unreasonable standards, we lose access to the powerful overall benefits. We eat the fictive “seed corn” and then wonder why there is no food later. And we reward fraudulent research because we no longer value and protect the basic research enterprise. As the kids say, “good luck with that.”
Author: Erik Devereux is a consultant to nonprofits and higher education and is adjunct faculty in the Security Studies Program at Georgetown University. He has a B.S. from the Massachusetts Institute of Technology (Political Science, 1985) and a Ph.D. from the University of Texas at Austin (Government, 1993). He is the author of Methods of Policy Analysis: Creating, Deploying, and Assessing Theories of Change (available for free here). Email: [email protected]. More content is available here.
Follow Us!