1
Introduction
1.1 Objectives
The aim of this book is to provide a guide to elementary matrix algebra sufficient for undertaking intermediate and advanced statistical courses such as multivariate data analysis and linear models. Starting from a definition of a matrix and covering the basic rules of addition, subtraction, multiplication and inversion, the later topics include determinants, calculation of eigenvectors and eigenvalues and differentiation of linear and quadratic forms with respect to vectors. These later topics are sometimes not included in basic courses on linear algebra but are virtually essential for full discussion of statistical topics in multivariate analysis and linear models. The notes go a little beyond meeting just this need, providing an initial guide to more advanced topics such as generalized inverses of singular and rectangular matrices and manipulation of partitioned matrices. This is to provide a second step for those who need to go a little further than standard lecture courses on advanced statistics, for example, when embarking on a dissertation. As well as describing the basics of matrix algebra, including numerical calculations “by hand”, for example, of matrix multiplication and inversion, the notes give guidance on how to do numerical calculations in R (R Core Team 2014). R is broadly similar in operation to the package MATLAB® but oriented specifically towards statistical applications rather than more general areas of applied mathematics. R is an open source system and is available free. It is closely similar to the commercial package S-PLUS: the prime difference is that R is command-line driven without the standard menus and dialogue boxes for statistical operations in S-PLUS. Otherwise, most code written for the two systems is interchangeable. There are however a few differences, for example there may be differences in the available optional arguments for some functions. Generally, all options available in S-PLUS are also available in R but not necessarily vice versa. These are quickly verified by use of the help system. The sites from which R and associated software (extensions and libraries) and manuals can be found are listed at http://www.ci.tuwien.ac.at/R/mirrors.html.
There are many available user interfaces for R, such as RStudio (Verzani, 2011), some of which provide a menu-driven facility, but this text assumes use of only the standard version of R without any such special facility.
Free versions of full manuals for R (mostly in PDF format) can be found at any of these mirror sites. There is a wealth of contributed documentation.
Some sections are considerably more advanced in concept and technical requirements than others, for example, those on partitioned matrices (§2.6.1) and generalized inverses (§8.3), but these have not been starred or marked since what may seem technically complex to one may seem straightforward to another with a different background. It is left to the readers to glean what they can from them on initial readings and then return later if a complete understanding is wanted. Some topics often covered in texts such as this one are not included (for example linear spaces) which means that there are a few results (e.g. equality of row and column rank, §3.1) which will need to be taken on trust or else checked in one of the more specialist texts referenced in §1.2
Since the applications envisaged for the majority of users of these notes are within statistics, most emphasis is given to real matrices and mostly to real symmetric matrices. Unless specifically indicated, all matrices are presumed to be real.
In addition to the algebraic manipulation of matrices the notes give numerical examples since it is necessary to be able to do numerical calculations, both by hand and by using R. Much of the exposition is by presenting examples with solutions. There are additionally some examples for self study and solutions are available separately (for the desperate) to these. For numerical questions, the calculations are given only in R and in some cases only the numerical answers are given. A brief guide to installing and running R is included.
1.2 Further Reading
1.2.1 Matrix algebra
There are many books on matrix algebra and a search will reveal hundreds if not thousands aimed at readers in the later years at school, throughout university and at advanced levels of mathematical research. A particular one consulted in the preparation of this text is Abadir and Magnus (2005) which consists largely of worked exercises. This will complete many of the details omitted from this book (in particular complex matrices) but, as is common with all post-school level texts, it does require a good understanding of linear spaces. It is not specifically written for any particular discipline but does contain several sections on statistical topics such as least squares and maximum likelihood estimation. A rather more theoretical book by Magnus and Neudecker (1988) goes into further details on vector and matrix calculus. Many statistical texts on topics such as linear models and multivariate analysis, such as Draper and Smith (1998), Seber and Lee (2012), Anderson (2003), Cox (2005) and Mardia et al. (1979), contain brief sections or appendices on matrix algebra but there are also many books aimed at the statistical reader. Three such are Searle (1982), Basilevsky (2013) and Harville (2008). A comprehensive account of linear algebra...