Numbers tend to be the gold-standard of policy analysis. Master's and doctoral policy programs rarely discuss qualitative analysis and tend to focus on the quantification of phenomena. As we learned last week, stories play a very important role in policy advocacy, but they tend to be ignored in the study of public policy. In my Master's of Public Policy program at Georgetown, for example, we took 3 required semesters of quantitative methods, but were not offered a qualitative course as an elective. Numbers are considered so important because they imply scientific analysis and objectivity. While the story of an individual is considered subjective and unrelated to the experiences of the population, a number is perceived as the objective picture of the issue as a whole.
Of course, anyone who has paid recent attention to political debate knows that both sides have numbers to support their point. Sometimes these numbers are directly contradictory. Occasionally, the number is completely made up like Senator Kyl's "not intended to be factual" statement that abortions are well over 90 percent of what planned parenthood does. Usually, both numbers contain some validity but vary in terms of how they count phenomena.
One of the major questions currently being debated is the extent of poverty in our society. In the section on security, I talked about relative versus absolute poverty and that certainly comes into play here. If you believe that relative poverty is important you would likely consider many more people to be poor than if you think absolute poverty is all that matters. We have two federal measures of poverty, the federal poverty threshold and the poverty guidelines. The federal policy threshold was developed by Mollie Orshansky in the 1970s and counts the pre-tax income, including any income provided by government assistance programs, of all family members and adjusts the count by the number and age of the family members. Non-cash benefits, benefits delivered through the tax-code and capital gains and losses are not counted. If your income falls below the poverty threshold, which is based on the cost of the basic basket of food necessary to meet dietary needs (adjusted for inflation) and an estimate (based on data from the 1960s) of what percent of their income families spend of food, then you are considered to be in poverty. The HHS poverty guidelines are used for program administration and simplify the census measure so that the age of family members is no longer taken into account.
Of course, many people find this measure to be problematic. Much of our aid to the poor is delivered through non-cash assistance and tax credits. Many scholars argue that without taking these sources of income into account the federal poverty level over-estimates the number of families in poverty. Others argue that families spend a much lower percentage of their income on food than they did in the 1960s due to the rising costs of housing and transportation so the federal poverty level under-estimates the number of people in poverty. Still others would argue that the federal poverty line doesn't take into account geographic variations in the cost of living so it under-estimates the number of people in poverty in large cities with high costs of living. When you read political and scholarly discussions of poverty in America (or of any issue) you have to pay attention to how they count who is in poverty and whether or not that measure is appropriate for the given context.
Stone also discusses some of the paradoxes of counting in the polis. Cost to government is often income to someone else. This is especially important in an era of contracted services. Whole industries have grown up to serve the needs of citizens that had once been fulfilled by government. As fiscal austerity measures are put in place, businesses who rely on government contracts will likely fail.
Averages become a paradox in the polis, as well. An average is a moving number. When a politician says "I want all students with low test scores to be brought up to the average", this becomes an impossible goal because the average test score moves (unless all high scoring students also come down to the average). All people want to be considered middle-class in America so we end up with a theoretical middle-class which bears little relationship to the actual statistical middle-class. We also end up with different definitions of the middle-class by party.
Finally, we have to remember that in the polis people adjust their behavior to adapt to measurement. There is an old adage in program evaluation that says "you get what you measure." If you measure one aspect of performance, that aspect will likely improve whereas other aspects will likely get worse. Measuring quantity of service will likely reduce the quality of service. Measuring pure numbers also creates the perverse incentives to cheat the system. The recent cheating scandal in Atlanta Public Schools illustrates how measurement can lead to corruption at both individual and systemic levels.
Numbers are often imbued with a sense of objectivity and scientific validity, but we have to remember that in the polis they tell stories just like symbols. You have to be just as skeptical when you hear a number as you are when you hear a story. Of course, just because numbers can be manipulated does not mean that they do not often have a degree of truth to them. My advice is to think about the source of the number, how it was measured and counted, and how other ways of measuring could have changed the statistic. Once you have skeptically examined the number, you can decide whether or not you think it is a valid measurement, or at least more or less valid than competing claims.
No comments:
Post a Comment