Statistics for Psychologists

An Intermediate Course

Brian S. Everitt

• Publisher: Psychology Press
• Year: 2001
• ISBN:
  9781135648350
• Edition: 1
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Built around a problem solving theme, this book extends the intermediate and advanced student's expertise to more challenging situations that involve applying statistical methods to real-world problems. Data relevant to these problems are collected and analyzed to provide useful answers. Building on its central problem-solving theme, a large number of data sets arising from real problems are contained in the text and in the exercises provided at the end of each chapter. Answers, or hints to providing answers, are provided in an appendix. Concentrating largely on the established SPSS and the newer S-Plus statistical packages, the author provides a short, end-of-chapter section entitled Computer Hints that helps the student undertake the analyses reported in the chapter using these statistical packages.

1

Statistics in Psychology: Data, Models, and a Little History

1.1. Introduction

Psychology is a uniquely diverse discipline, ranging from biological aspects of behaviour on the one hand to social psychology on the other, and from basic research to the applied professions of clinical, counselling, educational, industrial, organizational and forensic psychology.

—Andrew M. Colman, Companion Encyclopedia of Psychology, 1994.

As Dr. Colman states, psychology is indeed a diverse and fascinating subject, and many students are attracted to it because of its exciting promise of delving into many aspects of human behavior, sensation, perception, cognition, emotion, and personality, to name but a few. It probably comes as a disagreeable shock to many such students that they are often called on, early in their studies, to learn about statistics, because the subject (and, sadly, its practitioners, statisticians), are mostly seen as anything but exciting, and the general opinion seems to be that both should be avoided as far as possible.

However, this “head in the sand” attitude toward statistics taken by many a psychology student is mistaken. Both statistics (and statisticians) can be very exciting (!), and anyway, a knowledge and understanding of the subject is essential at each stage in the often long, and frequently far from smooth, road from planning an investigation to publishing its results in a prestigious psychology journal. Statistical principles will be needed to guide the design of a study and the collection of data. The initial examination of data and their description will involve a variety of informal statistical techniques. More formal methods of estimation and significance testing may also be needed in building a model for the data and assessing its fit.

Nevertheless, most psychologists are not, and have no desire to be, statisticians (I can’t think why). Consequently, questions have to be asked about what and how much statistics the average psychologist needs to know. It is generally agreed that psychologists should have some knowledge of statistics, at least, and this is reflected in the almost universal exposure of psychology students to an introductory course in the subject, covering topics such as

• descriptive statistics—histograms, means, variances, and standard deviations;
• elementary probability;
• the normal distribution;
• inference; t tests, chi-squared tests, confidence intervals, and so on;
• correlation and regression; and
• simple analyses of variance.

Although such a course often provides essential grounding in statistics, it frequently gives the wrong impression of what is and what is not of greatest importance in tackling real life problems. For many psychology students (and their teachers), for example, a p value is still regarded as the holy grail and almost the raison d’ être of statistics (and providing one, the chief role of statisticians). Despite the numerous caveats issued in the literature, many psychologists still seem determined to experience joy on finding a p value of .049 and despair on finding one of .051. Again, psychology students may, on their introductory course, learn how to perform a t test (they may, poor things, even be made to carry out the arithmetic themselves), but they may still be ill equipped to answer the question, How can I summarize and understand the main features of this set of data?

The aim of this text is essentially twofold. First it will introduce the reader to a variety of statistical techniques not usually encountered in an introductory course; second, and equally, if not more importantly, it will attempt to transform the knowledge gained in such a course into a more suitable form for dealing with the complexities of real data. Readers will, for example, be encouraged to replace the formal use of significance tests and the rigid interpretation of p values with an approach that regards such tests as giving informal guidance on possible evidence of an interesting effect. Readers may even be asked to abandon the ubiquitous signficance test altogether in favor of, for example, a graphical display that makes the structure in the data apparent without any formal analyses. By building on and reshaping the statistical knowledge gained in their first-level course, students will be better equipped (it is hoped) to overcome the criticisms of much current statistical practice implied in the following quotations from two British statisticians:

Most real-life statistical problems have one or more nonstandard features. There are no routine statistical questions; only questionable statistical routines.

—Sir David Cox.

Many statistical pitfalls lie in wait for the unwary. Indeed statistics is perhaps more open to misuse than any other subject, particularly by the nonspecialist. The misleading average, the graph with “fiddled axes,” the inappropriate p-value and the linear regression fitted to nonlinear data are just four examples of horror stories which are part of statistical folklore.

—Christopher Chatfield.

1.2. Statistics Descriptive, Statistics Inferential, and Statistical Models

This text will be primarily concerned with the following overlapping components of the statistical analysis of data.

1. The initial examination of the data, with the aim of making any interesting patterns in the data more visible.
2. The estimation of parameters of interest.
3. The testing of hypotheses about parameters.
4. Model formulation, building, and assessment.

Most investigations will involve little clear separation between each of these four components, but for now it will be helpful to try to indicate, in general terms, the unique parts of each.

1.2.1. The Initial Examination of Data

The initial examination of data is a valuable stage of most statistical investigations, not only for scrutinizing and summarizing data, but often also as an aid to later model formulation. The aim is to clarify the general structure of the data, obtain simple descriptive summaries, and perhaps get ideas for a more sophisticated analysis. At this stage, distributional assumptions might be examined (e.g., whether the data are normal), possible outliers identified (i.e., observations very different from the bulk of the data that may be the result of, for example, a recording error), relationships between variables examined, and so on. In some cases the results from this stage may contain such an obvious message that more detailed analysis becomes largely superfluous. Many of the methods used in this preliminary analysis of the data will be graphical, and it is some of these that are described in Chapter 2.

1.2.2. Estimation and Significance Testing

Although in some cases an initial examination of the data will be all that is necessary, most investigations will proceed to a more formal stage of analysis that involves the estimation of population values of interest and/or testing hypotheses about particular values for these parameters. It is at this point that the beloved significance test (in some form or other) enters the arena. Despite numerous attempts by statisticians to wean psychologists away from such tests (see, e.g., Gardner and Altman, 1986), the p value retains a powerful hold over the average psychology researcher and psychology student. There are a number of reasons why it should not.

First, p value is poorly understood. Although p values appear in almost every account of psychological research findings, there is evidence that the general degree of understanding of the true meaning of the term is very low. Oakes (1986), for example, put the following test to 70 academic psychologists:

Suppose you have a treatment which you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further suppose you use a simple independent means t-test and your result is t = 2.7, df = 18, P = 0.01. Please mark each of the statements below as true or false.

• You have absolutely disproved the null hypothesis that there is no difference between the population means.

• You have found the probability of the null hypothesis being true.

• You have absolutely proved your experimental hypothesis.

• You can deduce the probability of the experimental hypothesis being true.

• You know, if you decided to reject the null hypothesis, the probability that you are making the wrong decision.

• You have a reliable experiment in the sense that if, hypothetically, the experiment were repeated a great number of times, you would obtain a significant result on 99% of occasions.

The subjects were all university lecturers, research fellows, or postgraduate students. The results presented in Table 1.1 are illuminating.

Table 1.1 Frequencies and Percentages of “True” Responses in a Test of Knowledge of p Values

Statement	f	%
1. The null hypothesis is absolutely disproved.	1	1.4
2. The probability of the null hypothesis has been found.	25	35.7
3. The experimental hypothesis is absolutely proved.	4	5.7
4. The probability of the experimental hypothesis can be deduced.	46	65.7
5. The probability that the decision taken is wrong is known.	60	85.7
6. A replication has a .99 probability of being significant.	42	60.0

Under a relative frequency view of probability, all six statements are in fact false. Only 3 out of the 70 subjects came to the conclusion. The correct interpretation of the probability associated with the observed t value is

the probability of obtaining the observed data (or data that represent a more extreme departure from the null hypothesis) if the null hypothesis is true.

Clearly the number of false statements described as true in this experiment would have been reduced if the true interpretation of a p value had been included with the six others. Nevertheless, the exercise is extremely interesting in highlighting the misguided appreciation of p values held by a group of research psychologists.

Second, a p value represents only limited information about the results from a study. Gardner and Altman (1986) make the point that the excessive use of p values in hypothesis testing, simply as a means of rejecting or accepting a particular hypothesis, at the expense of other ways of assessing results, has reached such a degree that levels of significance are often quoted alone in the main text and in abstracts of papers with no mention of other more relevant and important quantities. The implications of hypothesis testing—that there can always be a simple yes or no answer as the fundamental result from a psychological study—is clearly false, and used in this way hypothesis testing is of limited value.

The most common alternative to presenting results in terms of p values, in relation to a statistical null hypothesis, is to estimate the magnitude of some parameter of interest along with some interval that includes the population value of the parameter with some specified probability. Such confidence intervals can be found relatively simply for many quantities of interest (see Gardner and Altman, 1986 for details), and although the underlying logic of interval estimaton is essentially similar to that of significance tests, they do not carry with them the pseudoscientific hypothesis testing language of such tests. Instead they give a plausible range of values for the unknown parameter. As Oakes (1986) rightly comments,

the significance test relates to what the population parameter is not: the confidence interval gives a plausible range for what the parameter is.

So should the p value be abandoned completely? Many statistician would answer yes, but I think a more sensible response, at least for psychologists, would be a resounding “maybe.” Such values should rarely be used in a purely confirmatory way,...