Introduction to Probability

Citation

Blitzstein, J. K., & Hwang, J. (2019). Introduction to Probability (2nd ed.). Chapman & Hall/CRC.

Chapter Summary

The book has 13 chapters and three appendices, 634 pages. According to the contents, every chapter ends with a recap, a section on doing the relevant simulations and calculations in R, and exercises. The publisher says the authors use stories to uncover connections between the fundamental distributions and conditioning to reduce complicated problems to manageable pieces. The book grew out of Harvard's Stat 110 lectures.

Chapter 1: Probability and Counting

Opens with why probability is worth studying, then sample spaces and "Pebble World", the naive definition of probability, how to count, story proofs and the non-naive definition of probability.

Chapter 2: Conditional Probability

Starts from the importance of thinking conditionally and builds the definition and intuition, Bayes' rule and the law of total probability, the fact that conditional probabilities are probabilities, independence of events and the coherency of Bayes' rule. It then treats conditioning as a problem-solving tool and ends with pitfalls and paradoxes.

Chapter 3: Random Variables and Their Distributions

Introduces random variables, distributions and probability mass functions, and the Bernoulli, Binomial, Hypergeometric and Discrete Uniform families. Cumulative distribution functions, functions of random variables, independence of random variables and the connections between the Binomial and Hypergeometric follow.

Chapter 4: Expectation

Covers the definition of expectation and its linearity, the Geometric and Negative Binomial distributions, indicator random variables and the fundamental bridge, the law of the unconscious statistician (LOTUS), variance, the Poisson distribution and its connection to the Binomial. A starred section uses probability and expectation to prove existence.

Chapter 5: Continuous Random Variables

Develops probability density functions and the Uniform, Normal and Exponential distributions, including the universality of the Uniform, a first look at Poisson processes and the symmetry of iid continuous random variables.

Chapter 6: Moments

Discusses summaries of a distribution, interpreting moments, sample moments, moment generating functions, generating moments with them and finding sums of independent random variables through them. A starred section covers probability generating functions.

Chapter 7: Joint Distributions

Treats joint, marginal and conditional distributions, a two-variable form of LOTUS, covariance and correlation, and the Multinomial and Multivariate Normal distributions.

Chapter 8: Transformations

Covers change of variables, convolutions, the Beta and Gamma distributions and their connections, and order statistics.

Chapter 9: Conditional Expectation

Treats conditional expectation given an event and given a random variable, its properties, conditional variance and the "Adam and Eve" examples. A starred section gives a geometric interpretation.

Chapter 10: Inequalities and Limit Theorems

Presents inequalities, the law of large numbers, the central limit theorem, and the Chi-Square and Student-t distributions.

Chapters 11–13: Markov Chains; Markov Chain Monte Carlo; Poisson Processes

Chapter 11 covers the Markov property and transition matrices, classification of states, stationary distributions and reversibility. Chapter 12 is built around Metropolis–Hastings. Chapter 13 treats Poisson processes in one dimension, conditioning, superposition and thinning, and processes in several dimensions.

Appendices

Appendix A, "Math", reviews sets, functions, matrices, difference and differential equations, partial derivatives, multiple integrals and sums, and ends with sections on pattern recognition and on common sense and checking answers. Appendix B introduces R (vectors, matrices, sampling and simulation, plotting, programming, summary statistics and distributions). Appendix C is a table of distributions.

Key Concepts

Chapters 1–2: Counting and Conditioning

• Sample space and naive definition: probability as a ratio of equally likely outcomes, which counting methods make usable.
• Story proof: an argument that proves an identity by interpreting both sides as counting the same thing.
• Bayes' rule and the law of total probability: the two basic tools for updating beliefs and splitting a problem into cases.

Chapters 3–6: Random Variables, Expectation and Moments

• Named distributions: Bernoulli, Binomial, Hypergeometric, Geometric, Negative Binomial, Poisson, Uniform, Normal and Exponential, each tied to a situation it models.
• Linearity of expectation and indicator random variables: computing means by adding up simple pieces without needing independence.
• Moment generating function: a function that encodes all moments and turns sums of independent variables into products.

Chapters 7–10: Joint Distributions, Transformations, Conditional Expectation, Limits

• Covariance and correlation: measures of linear dependence between variables.
• Change of variables and convolutions: finding the distribution of a function or sum of random variables.
• Conditional expectation: expectation given information, treated as a random variable.
• Law of large numbers and central limit theorem: the long-run behaviour of averages and sums.

Chapters 11–13: Markov Chains, MCMC and Poisson Processes

• Stationary distribution: a distribution that a Markov chain keeps unchanged.
• Metropolis–Hastings: a method for sampling from a target distribution by constructing a Markov chain.
• Poisson process: random points in time or space with independent, Poisson-distributed counts.

Critical Analysis

Strengths

• A story-driven approach. The publisher presents the book as built on stories that connect the fundamental distributions and on conditioning to reduce problems to pieces. The contents reflect this: "Story proofs" in Chapter 1, "Conditioning as a problem-solving tool" in Chapter 2, "Adam and Eve examples" in Chapter 9.
• Simulation with the text. Each chapter ends with an R section, and Appendix B teaches the R needed, so numerical experiments are part of the study rather than an add-on.
• Practice and support. Every chapter ends with exercises, and the publisher lists solutions to selected exercises, additional practice problems, handouts with review material and sample exams, animations and interactive visualizations, and links to lecture videos, all on Joseph Blitzstein's website (stat110.net).
• Modern applications within an introductory text. The publisher notes applications from coincidences and paradoxes to Google PageRank and MCMC, and the final chapters on Markov chains, Metropolis–Hastings and Poisson processes extend the book beyond the basics of probability.

Limitations

• Measure theory is absent. The contents show no measure-theoretic material. Readers heading toward graduate probability will need a measure-theoretic text.
• Statistical inference is outside its scope. The book covers probability, with the Chi-Square and Student-t distributions as the closest approach to statistics; estimation and hypothesis testing need a companion text.
• Long for one course. At 634 pages, a course must choose chapters; the later chapters on Markov chains, MCMC and Poisson processes are natural candidates to leave for another course.
• Full solutions are limited. The publisher lists solutions to selected exercises only; the complete solutions manual is restricted to instructors.

Where It Fits in the Course

Among the probability and statistics books on this site, Blitzstein and Hwang is the calculus-based first course in probability itself. Introductory Statistics 2e is a free, one-semester statistics text that needs only algebra and is the right start for students who have not studied calculus, and Spiegelhalter's The Art of Statistics is a non-technical book about statistical thinking.

After this book come the books that build on probability. Larsen and Marx's An Introduction to Mathematical Statistics and Its Applications and Casella and Berger's Statistical Inference move to estimation and hypothesis testing; Mavrakakis and Penzer's Probability and Statistical Inference is a more rigorous treatment of both; Mitzenmacher and Upfal's Probability and Computing applies probability to algorithms; and Williams's Probability with Martingales re-establishes the subject on measure-theoretic foundations.

How to Use This Book

The book is written for undergraduate readers meeting probability for the first time; the site lists single- and multivariable calculus as the prerequisite, with some linear algebra helping in the later chapters, and Appendix A gives a refresher on the mathematics used.

A practical plan:

• Core: Chapters 1 to 5 (counting, conditioning, random variables, expectation, continuous distributions), then Chapters 7 to 10 (joint distributions, transformations, conditional expectation, limit theorems). Chapter 6 on moments can be read alongside Chapter 7.
• Optional by interest: the starred sections, and Chapters 11 to 13 for Markov chains, MCMC and Poisson processes.
• Practice: work the R section at the end of each chapter, do exercises on every topic, and use the supplementary material at stat110.net for extra problems and lecture videos.