A Panorama of Statistics

Perspectives, Puzzles and Paradoxes in Statistics

Eric Sowey, Peter Petocz

• Publisher: Wiley
• Year: 2017
• ISBN:
  9781119075844
• Edition: 1
• Topic: Mathématiques
• Subtopic: Probabilités et statistiques

This book is a stimulating panoramic tour – quite different from a textbook journey – of the world of statistics in both its theory and practice, for teachers, students and practitioners. At each stop on the tour, the authors investigate unusual and quirky aspects of statistics, highlighting historical, biographical and philosophical dimensions of this field of knowledge.Each chapter opens with perspectives on its theme, often from several points of view. Five original and thought-provoking questions follow. These aim at widening readers' knowledge and deepening their insight. Scattered among the questions are entertaining puzzles to solve and tantalising paradoxes to explain. Readers can compare their own statistical discoveries with the authors' detailed answers to all the questions.

The writing is lively and inviting, the ideas are rewarding, and the material is extensively cross-referenced.

A Panorama of Statistics:

• Leads readers to discover the fascinations of statistics.
• Is an enjoyable companion to an undergraduate statistics textbook.
• Is an enriching source of knowledge for statistics teachers and practitioners.
• Is unique among statistics books today for its memorable content and engaging style.

Lending itself equally to reading through and to dipping into, A Panorama of Statistics will surprise teachers, students and practitioners by the variety of ways in which statistics can capture and hold their interest. Reviews:
"As befits the authors' statement that 'this is not a textbook', the structure is unusual. There are twenty-five chapters organised in five sections, each beginning with a brief perspective of a theme in statistics and finishing with five questions related to that theme. The answers provided to the questions, in section six, are as discursive and illuminating as the main body of the text. Even if you are pretty sure you know the answer, it is always worth checking what the authors have to say. Chances are that you will learn something every time. The glimpses and insights given into this enormous and far-reaching discipline succeed in being bewitching, entertaining and inviting; coverage was never the aim." "In summary, this splendid book lives up to the four 'p-values' of its title. It is panoramic in the scope of its survey of statistics, it is full of illuminating perspectives, it sets entertaining and challenging puzzles, and it explores fascinating paradoxes. Read it, enjoy it and learn from it." From Neil Sheldon, Teaching Statistics, volume 9, no. 2, May 2017

Part I
Introduction

1
Why is statistics such a fascinating subject?

In the real world, little is certain. Almost everything that happens is influenced, to a greater or lesser degree, by chance. As we shall explain in this chapter, statistics is our best guide for understanding the behaviour of chance events that are, in some way, measurable. No other field of knowledge is as vital for the purpose. This is quite a remarkable truth and, statisticians will agree, one source of the subject’s fascination.

You may know the saying: data are not information and information is not knowledge. This is a useful reminder! Even more useful is the insight that it is statistical methods that play the major role in turning data into information and information into knowledge.

In a world of heavily promoted commercial and political claims, a familiarity with statistical thinking can bring enormous personal and social benefits. It can help everyone to judge better what claims are trustworthy, and so become more competent and wiser as citizens, as consumers and as voters. In short, it can make ours not only a more numerate, but also a more accurately informed, society. This is an ideal we shall return to in CHAPTER 3.

Chance events are studied in the physical, biological and social sciences, in architecture and engineering, in medicine and law, in finance and marketing, and in history and politics. In all these fields and more, statistics has well‐established credentials. To use John Tukey’s charming expression, ‘being a statistician [means] you get to play in everyone’s backyard’. (There is more about this brilliant US statistician in CHAPTER 22, FIGURE 22.2.)

‐‐‐oOo‐‐‐

To gain a bird’s eye view of the kinds of practical conclusions this subject can deliver, put yourself now in a situation that is typical for an applied statistician.

Suppose you have collected some data over a continuous period of 150 weekdays on the daily number of employees absent from work in a large insurance company. These 150 numbers will, at first, seem to be just a jumble of figures. However, you – the statistician – are always looking for patterns in data, because patterns suggest the presence of some sort of systematic behaviour that may turn out to be interesting. So you ask yourself: can I find any evidence of persisting patterns in this mass of figures? You might pause to reflect on what sorts of meaningful patterns might be present, and how you could arrange the data to reveal each of them. It is clear that, even at this early stage of data analysis, there is lots of scope for creative thinking.

Exercising creativity is the antithesis of following formalised procedures. Unfortunately, there are still textbooks that present statistical analysis as no more than a set of formalised procedures. In practice, it is quite the contrary. Experience teaches the perceptive statistician that a sharpened curiosity, together with some preliminary ‘prodding’ of the data, can often lead to surprising and important discoveries. Tukey vigorously advocated this approach. He called it ‘exploratory data analysis’. Chatfield (2002) excellently conveys its flavour.

In this exploratory spirit, let’s say you decide to find out whether there is any pattern of absenteeism across the week. Suppose you notice at once that there seem generally to be more absentees on Mondays and Fridays than on the other days of the week. To confirm this impression, you average the absentee numbers for each of the days of the week over the 30 weeks of data. And, indeed, the averages are higher for Mondays and Fridays.

Then, to sharpen the picture further, you put the Monday and Friday averages into one group (Group A), and the Tuesday, Wednesday and Thursday averages into a second group (Group B), then combine the values in each group by averaging them. You find the Group A average is 104 (representing 9.5% of staff) and the Group B average is 85 (representing 7.8% of staff).

This summarisation of 30 weeks of company experience has demonstrated that staff absenteeism is, on average, 1.7 percentage points higher on Mondays and Fridays as compared with Tuesdays, Wednesdays and Thursdays. Quantifying this difference is a first step towards better understanding employee absenteeism in that company over the longer term – whether your primary interest is possible employee discontent, or the financial costs of absenteeism to management.

Creating different kinds of data summaries is termed statistical description. Numerical and graphical methods for summarising data are valuable, because they make data analysis more manageable and because they can reveal otherwise unnoticed patterns.

Even more valuable are the methods of statistics that enable statisticians to generalise to a wider setting whatever interesting behaviour they may have detected in the original data. The process of generalisation in the face of the uncertainties of the real world is called statistical inference. What makes a statistical generalisation so valuable is that it comes with an objective measure of the likelihood that it is correct.

Clearly, a generalisation will be useful in practice only if it has a high chance of being correct. However, it is equally clear that we can never be sure that a generalisation is correct, because uncertainty is so pervasive in the real world.

To return to the example we are pursuing, you may be concerned that the pattern of absenteeism detected in 30 weeks of data might continue indefinitely, to the detriment of the company. At the same time, you may be unsure that that pattern actually is a long‐term phenomenon. After all, it may have appeared in the collected data only by chance. You might, therefore, have good reason to widen your focus, from absenteeism in a particular 30‐week period to absenteeism in the long term.

You can test the hypothesis that the pattern you have detected in your data occurred by chance alone against the alternative hypothesis that it did not occur by chance alone. The alternative hypothesis suggests that the pattern is actually persistent – that is, that it is built into the long‐term behaviour of the company if there are no internal changes (by management) or external impacts (from business conditions generally). As just mentioned, the statistical technique for performing such a hypothesis test can also supply a measure of the likelihood that the test result is correct. For more on hypothesis testing, see CHAPTER 16.

When you do the test, suppose your finding is in favour of the alternative hypothesis. (Estimating the likelihood that this finding is correct requires information beyond our scope here, but there are ways of testing which optimise that likelihood.) Your finding suggests a long‐term persisting pattern in absenteeism. You then have grounds for recommending a suitable intervention to management.

Generalising to ‘a wider setting’ can also include to ‘a future setting’, as this example illustrates. In other words, statistical inference, appropriately applied, can offer a cautious way of forecasting the future – a dream that has fascinated humankind from time immemorial.

In short, statistical inference is a logical process that deals with ‘chancy’ data and generalises what those data reveal to wider settings. In those wider settings, it provides precise (as opposed to vague) conclusions which have a high chance of being correct.

‐‐‐oOo‐‐‐

But this seems paradoxical! What sort of logic is it that allows highly reliable conclusions to be drawn in the face of the world’s uncertainties? (Here, and in what follows, we say ‘highly reliable’ as a...

Table of contents

• Cover
• Title Page
• Table of Contents
• Preface
• Acknowledgments
• Part I: Introduction
• Part II: Statistical description
• Part III: Preliminaries to inference
• Part IV: Statistical inference
• Part V: Some statistical byways
• Part VI: Answers
• Index
• End User License Agreement