Some Basic Theory for Statistical Inference

Monographs on Applied Probability and Statistics

E.J.G. Pitman

• Publisher: Chapman and Hall/CRC
• Year: 2018
• ISBN:
  9781351093675
• Edition: 1
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

In this book the author presents with elegance and precision some of the basic mathematical theory required for statistical inference at a level which will make it readable by most students of statistics.

CHAPTER ONE

BASIC PRINCIPLES OF THE THEORY OF INFERENCE THE LIKELIHOOD PRINCIPLE SUFFICIENT STATISTICS

In developing the theory of statistical inference, I find it helpful to bear in mind two considerations. Firstly, I take the view that the aim of the theory of inference is to provide a set of principles, which help the statistician to assess the strength of the evidence supplied by a trial or experiment for or against a hypothesis, or to assess the reliability of an estimate derived from the result of such a trial or experiment. In making such an assessment we may look at the results to be assessed from various points of view, and express ourselves in various ways. For example, we may think and speak in terms of repeated trials as for confidence limits or for significance tests, or we may consider the effect of various loss functions. Standard errors do give us some comprehension of reliability; but we may sometimes prefer to think in terms of prior and posterior distributions. All of these may be helpful, and none should be interdicted. The theory of inference is persuasive rather than coercive.

Secondly, statistics being essentially a branch of applied mathematics, we should be guided in our choice of principles and methods by the practical applications. All actual sample spaces are discrete, and all observable random variables have discrete distributions. The continuous distribution is a mathematical construction, suitable for mathematical treatment, but not practically observable. We develop our fundamental concepts, principles and methods, in the study of discrete distributions. In the case of a discrete sample space, it is easy to understand and appreciate the practical or experimental significance and value of conditional distributions, the likelihood principle, the principles of sufficiency and conditionality, and the method of maximum likelihood. These are then extended to more general distributions by means of suitable definitions and mathematical theorems.

Let us consider the likelihood principle. If the sample space is discrete, its points may be enumerated,

$x_1,x_2,\dots$

Suppose that the probability of observing the point x_r is f(x_r, θ), and that θ is unknown. An experiment is performed, and the outcome is the point x of the sample space. All that the experiment tells us about θ is that an event has occurred, the probability of which is f(x, θ), which for a given x is a function of θ, the likelihood function.

Consider first the case where θ takes only two values θ₀, θ₁. To decide between θ₀ and θ₁, all that the experiment gives us is the pair of likelihoods f(x, θ₀), f(x, θ₁). Suppose that θ is a random variable, taking the v...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• Preface
• Chapter one Basic Principles of the Theory of Inference, The Likelihood Principle, Sufficient Statistics
• Chapter two Distance between Probability Measures
• Chapter three Sensitivity of a Family of Probability Measures with respect to a Parameter
• Chapter four Sensitivity Rating, Conditional Sensitivity, The Discrimination Rate Statistic
• Chapter five Efficacy, Sensitivity, The Cramér–Rao Inequality
• Chapter six Many Parameters, The Sensitivity Matrix
• Chapter seven Asymptotic Power of a Test, Asymptotic Relative Efficiency
• Chapter eight Maximum Likelihood Estimation
• Chapter nine The Sample Distribution Function
• Appendix: Mathematical Preliminaries
• References
• Index