Section III
Estimating How the World Works: Testing Claims and Drawing Inferences
Section III Preview
Estimating How the World Works: Testing Claims and Drawing Inferences
Describing Those Data You've Collected
The previous section provided a useful discussion of the basic operations of gathering data. We covered how to structure a data collection effort (research design), how to measure social phenomenon (measurement), and how to draw a sample from a population of interest (sampling). We then moved on to explanations of data gathering and data use techniques, such as performing a case study or conducting interviews with subject matter experts. While all of that material is extremely important, here in section III, we are moving on to some even better stuff! You might think about where we are at this point in the text by considering this statement: ultimately we want to make claims about how the world is and how it works in the area of public affairs. We are at the point in this book where we move from data gathering to description and to inference. So we can say that one really good way of explaining how the world of public affairs works is to measure social phenomena that have importance to governance of the public sector. Once you collect measures, we are going to need to describe the nature of those variable distributions. Doing so is what descriptive statistics is all about: providing useful explanations of what distributions tell us about what we have collected. Effective descriptions are powerful: they give insight to what is happening with policies and programs.
Here’s a true, real-world story, that the authors of this book have direct personal knowledge of due to applied project work. Some of those charged with oversight for a state agency that distributed community development funds did not have full or precise awareness of the overall trend in how those funds had been geographically dispersed across the state. (That may seem a little hard to believe, but in abbreviated form, that was the situation.) A simple map showing the geographic patterns of where the development funds were delivered revealed important implications for how well the program was working in practice. In short, a simple mapping exercise—a visual display of data— produced a much clearer understanding of the nature of policy implementation; in turn the oversight board had a stronger basis for considering possible program adjustments. The lesson: do not underestimate the power of effective data description.
The first two chapters in this section cover the topics of coding and displaying data, as well as an introduction to the basics of descriptive statistics.
After Describing Your Distributions: Moving on to Testing and Drawing Inferences
After learning about how to describe and present variable distributions, our next step is to learn about inferential statistics. We would like to be fully honest here (given we’re honest people, more or less): this is where students often start to find the material challenging. It seems there tends to be two major hurdles in comprehension of the material. The first is understanding why we are employing a particular type of hypothesis test. The second is understanding what statistical significance means. Chapter 15 in this section gives you a clear and concise explanation of both, but before you get to that chapter, we would like to offer a quick word or two here to help with the intuition of what you are about to study.
If you think about observing outcomes in the real world, you might think of things in terms of a bell curve, which presumably everyone has seen. A bell curve shows in graphical form that most observations of a phenomenon occur around the central point of a distribution (the big middle chunk of the curve). Moving away from that central point (where most observations are found) we can think of those observations as more extreme and less likely to be seen (hence the tails of the curve get really small; hence the bell shape). Here’s a silly example: the average adult male in the United States is about 5 feet 9 inches or so. Say you enter a classroom for this course, and there are about 30 students enrolled and in the room. But as you walk in, you see a group of guys in the room, say four or five, taller than 6 feet 10 inches. Now, just due to random variation you’d expect absolutely to see in a classroom of 30 students some a little taller and some a little shorter than 5 feet 9 inches; that is, you wouldn’t expect all the males in the course to be exactly the average height of an adult male in the United States. That is important to recognize: some amount of variation around that average height is to be expected—but most people are going to be fairly close to the population average (which we’ve defined as about 5 feet 9 inches). But you really wouldn’t expect to see one-sixth of the classroom to be nearly 7 feet tall! There is a really low chance that you get that many tall people in a small group due to random chance; i.e., it is unlikely that your course of 30 students—just by wild coincidence—had five guys of such impressive and unusual height. So what do you make of the situation? Well, you might guess that for some reason, a group of members of the university’s men’s basketball team ended up enrolling in that course. That might be a good guess for what is going on because you are pretty sure it is unlikely to see that many tall people in a classroom of 30 students just by random chance.
While this is a silly little example, in essence it reveals what the process of hypothesis testing and ...