An Introduction to Statistical Learning: with Applications in Python

Citation

James, G., Witten, D., Hastie, T., Tibshirani, R., & Taylor, J. (2023). An Introduction to Statistical Learning: with Applications in Python. Springer (Springer Texts in Statistics).

Chapter Summary

The Python edition (ISLP) runs to about 600 pages in thirteen chapters. It follows the second edition of the R book (ISLR, 2021), whose first edition appeared in 2013. Jonathan Taylor joined the four original authors for this edition. The authors describe the book as a broad and less technical treatment of statistical learning for anyone who wants to use contemporary tools for data analysis. Every chapter after the introduction ends with a lab in Python, and the authors' ISLP package supplies the datasets and helper functions the labs use. Companion courses with video lectures are offered through edX.

Chapter 1: Introduction

A thirteen-page introduction that opens the book. Unlike the other chapters, it has no lab.

Chapter 2: Statistical Learning

At about fifty pages, this chapter sets out what statistical learning is and the framework the later chapters use. The lab is an introduction to Python, covering NumPy arrays, indexing, loading data with pandas, plotting and loops.

Chapter 3: Linear Regression

Regression in depth over some sixty-five pages. The lab fits simple and multiple linear regression, checks goodness of fit, and adds interaction terms, non-linear transformations of the predictors and qualitative predictors.

Chapter 4: Classification

Logistic regression, linear and quadratic discriminant analysis, naive Bayes and K-nearest neighbours, compared on stock market data. The lab also fits linear and Poisson regression to bike-sharing counts.

Chapter 5: Resampling Methods

The validation-set approach, cross-validation and the bootstrap, used to estimate test error and the variability of estimates.

Chapter 6: Linear Model Selection and Regularization

Best-subset and stepwise selection, ridge regression and the lasso, and dimension-reduction methods (principal components regression and partial least squares), with cross-validation used to choose among models.

Chapter 7: Moving Beyond Linearity

Polynomial regression, step functions, splines, smoothing splines, local regression and generalised additive models.

Chapter 8: Tree-Based Methods

Classification and regression trees, bagging, random forests, boosting and Bayesian additive regression trees.

Chapter 9: Support Vector Machines

The support vector classifier and the support vector machine, ROC curves, multi-class problems and an application to gene expression data.

Chapter 10: Deep Learning

Single-layer and multilayer networks, convolutional networks for images, document classification and recurrent networks for sequences. The lab uses PyTorch (with PyTorch Lightning) on the Hitters, MNIST, CIFAR-100 and IMDb data. It compares a network with the lasso, uses a pretrained convolutional network and fits a time-series model.

Chapter 11: Survival Analysis and Censored Data

Methods for time-to-event data subject to censoring, applied in the lab to brain cancer, publication-time and call-centre data.

Chapter 12: Unsupervised Learning

Principal components analysis, matrix completion, and K-means and hierarchical clustering, illustrated with the NCI60 cancer cell-line data.

Chapter 13: Multiple Testing

Testing many hypotheses at once: a review of hypothesis tests, the family-wise error rate, the false discovery rate and a resampling approach.

Key Concepts

Chapters 1–5: Foundations

• Statistical learning: a set of tools for modelling and understanding the relationship between predictors and a response, the framework set out in Chapter 2.
• Linear regression extensions: interaction terms, transformed predictors and qualitative predictors within the linear model.
• Classification methods: logistic regression, discriminant analysis, naive Bayes and K-nearest neighbours for categorical outcomes.
• Cross-validation and the bootstrap: resampling to estimate test error and the uncertainty of estimates.

Chapters 6–9: Beyond Least Squares

• Regularisation: shrinking coefficients (ridge, lasso) to improve prediction and, for the lasso, select variables.
• Splines and generalised additive models: flexible non-linear fits that remain interpretable.
• Ensembles of trees: bagging, random forests and boosting combine many trees for accuracy.
• Support vector machines: classifiers based on separating hyperplanes and kernels.

Chapters 10–13: Modern Topics

• Neural networks: layered models, including convolutional and recurrent networks, fitted by gradient-based training.
• Censoring: incomplete observation of event times, handled by survival methods.
• Principal components and clustering: finding structure in data without an outcome variable.
• False discovery rate: controlling the expected share of false positives among rejected hypotheses.

Critical Analysis

Strengths

• Accessible without heavy mathematics. The authors aim at a broad, less technical treatment for anyone using modern data-analysis tools. A review of the R edition in the Journal of the American Statistical Association, quoted on the book's site, called it well written and easy to read. The same review judged it suitable both for undergraduates and for less quantitatively oriented graduate students and practitioners.
• Labs in every chapter. Each chapter's Python lab works through the methods on real datasets, from linear regression to deep learning in PyTorch, so readers learn to apply what they read. The ISLP package and its online documentation make the datasets and helper functions easy to install.
• Broad, current coverage. Beyond the classical methods, the book covers deep learning, survival analysis, matrix completion, Bayesian additive regression trees and multiple testing.
• Free resources. The authors host a free PDF of each edition on statlearning.com, and the companion edX courses provide video lectures covering all chapters.
• Shared conceptual core with the R course. The Python and R companion courses share chapter lectures and differ mainly in their computing sessions (see the companion course below).

Limitations

• Less theory. As a deliberately less technical treatment, the book is not the place for full derivations. An endorsement by a Carnegie Mellon statistician on the book's site describes it as inspired by Hastie, Tibshirani and Friedman's The Elements of Statistical Learning, which is the more technical book to read for the mathematics.
• Statistics more than computer science. The emphasis is on regression, classification and resampling from a statistical viewpoint. Topics such as reinforcement learning, large-scale optimisation or the engineering of machine-learning systems are outside its scope.
• Software changes. The labs depend on specific versions of Python packages, which the documentation lists, so code may need adjusting as libraries change.

Where It Fits in the Course

ISLP is the entry point to machine learning in this course. Melanie Mitchell's Artificial Intelligence: A Guide for Thinking Humans is a non-technical account of what current AI can and cannot do. It explains neural networks and deep learning at a conceptual level and makes a good companion read. Kevin Murphy's Probabilistic Machine Learning: An Introduction covers similar ground through probabilistic modelling and Bayesian decision theory, with considerably more mathematics. It is the natural next step after ISLP.

Russell and Norvig's Artificial Intelligence: A Modern Approach is a broad survey of AI, from search and logic to planning, learning, language and robotics, in which machine learning is one part among many. ISLP gives far deeper practical training in the statistical learning chapters of such a course. Szeliski's Computer Vision: Algorithms and Applications is a specialised text, and ISLP's Chapter 10 on convolutional networks is useful preparation for its deep-learning material.

How to Use This Book

The book suits upper-level undergraduates, graduate students outside statistics and practitioners who know basic statistics and some Python.

A practical plan:

• Core sequence: Chapters 2–6 form the foundation: the statistical learning framework, regression, classification, resampling and regularisation. Read them in order.
• Then by interest: Read Chapters 7–9 for non-linear methods, trees and SVMs, Chapter 10 for deep learning, and Chapter 12 for unsupervised learning. Chapters 11 and 13 suit readers in biostatistics, medicine or any field with many simultaneous tests.
• Do every lab. Run each chapter's lab in a notebook after reading the chapter. Then repeat the analysis on a different dataset from the ISLP package.
• Use the online courses for video lectures, and check the errata page for corrections.

Readers who finish ISLP and want the theory should continue with a more mathematical text.

The authors' companion course

Stanford Online's Statistical Learning with Python (SOHS-YSTATSLEARNINGP, School of Humanities and Sciences) offers a structured route through ISLP. It is a fully online, on-demand edX course rather than a campus class, and its course page gives no teaching term. The book is the course's companion text. According to the course page, the lectures cover all the material in the 2023 Python volume. That material includes regression and classification, resampling, regularisation, nonlinear methods, trees, support vector machines, deep learning, survival models, multiple testing and some unsupervised methods. The course page does not give chapter numbers, page ranges or a weekly reading schedule.

All computing in the course is done in Python. Its tutorials move from Python basics to implementations of each chapter's methods, which mirrors the book's progression from the Chapter 2 Python lab onward. The course page does not state formal prerequisites. The prerequisites suggested on this page are this guide's own recommendation.

The authors also offer a parallel R course tied to the second edition of the R book. It shares the chapter lectures but has different, language-specific computing sessions. A reader working in R can follow the same lectures but should pair them with the second R edition's labs rather than ISLP's.

To combine the course and the book, watch a chapter's lectures and read the matching ISLP chapter. Then work the Python lab yourself before watching the course's computing session.

Focused reading and edition boundary

The exercise below uses the guide's companion problem. Open its assumptions and data before answering.

Reading route: In the 2023 Python volume, read Statistical Learning, Classification and Resampling Methods, followed by parts of the model-selection discussion.

Try before continuing: Write two legitimate validation questions for the same repeated-device table and choose a split unit for each. Checkpoint: Withholding a row is not the same as withholding the device that generated it.

Edition and access: The edition listed here (Springer, 2023, ISBN 9783031387463) is the Python volume, the same one covered by the Stanford Online course above. It is distinct from the R volumes. The authors' site lists it separately from the R first (2013) and second (2021) editions and gives it no edition number, so cite it as the 2023 Python edition. The authors provide a free PDF and lab notebooks. Use the package versions specified in the notebook documentation.

Springer book page