Probability and Statistical Inference: From Basic Principles to Advanced Models

Citation

Mavrakakis, M. C., & Penzer, J. (2021). Probability and Statistical Inference: From Basic Principles to Advanced Models (1st ed.). Chapman & Hall/CRC.

Chapter Summary

The book has twelve chapters and about 444 pages. It grew out of a second-year statistical inference course at the London School of Economics (ST202), and the publisher describes it as bridging the gap between introductory books and advanced graduate-level texts, presenting probability, distribution theory and inference accessibly without giving up mathematical rigour. Chapters 2–5 build probability and distribution theory, Chapters 6–10 move to models and classical inference, and Chapters 11–12 cover Bayesian and simulation methods.

Chapter 1: Introduction

The chapter sets out why a theoretical understanding of inference matters for analysing and modelling data, and the role probability plays in drawing conclusions from data.

Chapter 2: Probability

Mathematical probability is introduced together with the intuitive ideas behind it: the formal definition of a probability measure, methods for counting outcomes, and conditional probability and independence.

Chapter 3: Random Variables and Univariate Distributions

Outcomes are mapped to real numbers. The chapter treats cumulative distribution functions, discrete and continuous random variables, expectation and variance, generating functions, and functions of a random variable.

Chapter 4: Multivariate Distributions

The chapter extends the previous one to several variables: joint, marginal and conditional distributions, covariance and correlation, and transformations of random variables.

Chapter 5: Conditional Distributions

Discrete and continuous conditional distributions are treated in more depth, together with conditional expectation.

Chapter 6: Statistical Models

A survey of the models used in statistical practice: linear models, generalised linear models, survival analysis, time series, Poisson processes and Markov chains. The publisher describes this as a concise but broad account of statistical modelling.

Chapter 7: Sample Moments and Quantiles

Sample moments, the central limit theorem, and order statistics and quantiles.

Chapter 8: Estimation, Testing, and Prediction

The fundamentals of classical inference: point estimation, interval estimation, hypothesis testing and prediction.

Chapter 9: Likelihood-Based Inference

The likelihood function, maximum-likelihood estimation and likelihood-ratio tests.

Chapter 10: Inferential Theory

The theory behind inference: the sufficiency principle, properties of estimators and the construction of most powerful tests.

Chapter 11: Bayesian Inference

Prior and posterior distributions, Bayesian estimation and hierarchical models.

Chapter 12: Simulation Methods

Simulation for inference, including Monte Carlo integration and Markov chain Monte Carlo. The publisher also lists the bootstrap among the computational methods illustrated in the book.

Key Concepts

Chapters 2–5: Probability and distribution theory

• Probability measure, conditional probability and independence (Chapter 2).
• Cumulative distribution function, expectation, variance and generating functions (Chapter 3).
• Joint distributions, covariance and correlation (Chapter 4); conditional expectation (Chapter 5).

Chapters 6–10: Models and classical inference

• Generalised linear models, survival analysis, time series, Poisson processes (Chapter 6).
• Central limit theorem (Chapter 7).
• Point and interval estimation, hypothesis testing and prediction (Chapter 8).
• Maximum-likelihood estimation (Chapter 9); sufficiency and most powerful tests (Chapter 10).

Chapters 11–12: Bayesian and computational methods

• Prior, posterior and hierarchical models (Chapter 11).
• Monte Carlo integration and Markov chain Monte Carlo (Chapter 12).

Critical Analysis

Strengths

• A tested course behind it. The authors describe the book as based on a popular undergraduate course at the London School of Economics, and reviewer Fiona Steele (London School of Economics) writes that the content and presentation have been honed over many years of teaching.
• Rigour with explanation. Reviewers, including Stuart Coles, praise the balance of formal mathematics and statistical insight and the clarity of the explanations. The publisher states the aim of providing explanations that are more detailed and approachable than those in advanced texts.
• Breadth of scope. One volume spans probability, distribution theory, classical inference, likelihood, Bayesian statistics and simulation, with a survey of models in the middle.
• Practice built in. Steele's review notes that concepts are illustrated with real-world examples and that each chapter includes an extensive and graduated set of exercises.

Limitations

• Not a first course. The publisher addresses students who have already completed a first course in probability and statistics. Readers without that background should start with an introductory text.
• Models are a survey. Chapter 6 covers linear and generalised linear models, survival analysis, time series, Poisson processes and Markov chains in one chapter, so it introduces rather than develops each of them.
• A bridge, not the final level. As a book positioned between introductory and advanced graduate texts, it does not replace a measure-theoretic treatment of probability or a full graduate course in inference.

Where It Fits in the Course

Among the probability and statistics books on the list, this is an intermediate-to-advanced text that follows a first course. For that first course, Introductory Statistics 2e and The Art of Statistics are gentler, and Introduction to Probability by Blitzstein and Hwang is an undergraduate probability text built on stories and simulation. An Introduction to Mathematical Statistics and Its Applications covers probability, estimation, hypothesis testing and regression.

Beyond it lie Statistical Inference, a graduate treatment of inference with detailed proofs and exercises, and Probability with Martingales, which develops probability on measure-theoretic foundations. Probability and Computing pursues probability through algorithms and randomisation. A useful sequence is a first course, then this book for a unified account of probability, inference and models, then a graduate text on either inference or measure-theoretic probability.

How to Use This Book

The book is for students in advanced undergraduate or postgraduate courses who have completed a first course in probability and statistics, and for data scientists and applied practitioners who want to understand the theory behind the methods they use. Comfortable calculus is the practical mathematical prerequisite.

A practical plan:

• Core: Chapters 2–5 for probability and distributions, then Chapters 7–10 for estimation, testing, likelihood and inferential theory. These are the backbone of a one-semester course.
• Selective reading: Chapter 6 on models can be taken as a survey and revisited when a specific model family is needed; Chapters 11 and 12 suit a second pass or a course that includes Bayesian and computational statistics.
• Practice: work the exercises at the end of each chapter, and reproduce the simulation methods of Chapter 12 in code.