Python Programming and Numerical Methods: A Guide for Engineers and Scientists

Citation

Kong, Q., Siauw, T., & Bayen, A. M. (2020). Python Programming and Numerical Methods: A Guide for Engineers and Scientists. Academic Press (Elsevier). Free official edition: https://pythonnumericalmethods.berkeley.edu/notebooks/Index.html

Chapter Summary

The book has two parts: Part I (Chapters 1–13) teaches programming, and Part II (Chapters 14–24) introduces numerical methods. Chapter 25, on machine learning, is an extension available only in the online version.

Chapter 1: Python Basics

The opening chapter gets students running: installing Python, using it as a calculator, managing packages and working in Jupyter Notebook. It introduces the basic data types (integers, floats, complex numbers, strings and booleans) and logical expressions and operators.

Chapter 2: Variables and Basic Data Structures

Variables and assignment are followed by Python's built-in containers—strings, lists, tuples, sets and dictionaries—and by NumPy arrays, which the summary singles out as the central object in scientific computing.

Chapters 3–4: Functions; Branching Statements

Chapter 3 explains how to define functions, how each function has its own memory space (local versus global variables), nested functions, lambda functions and passing functions as arguments. Chapter 4 adds if-else branching and the one-line ternary operator.

Chapters 5–6: Iteration; Recursion

Chapter 5 covers for-loops, while-loops and comprehensions as tools for iterative solutions. Chapter 6 introduces recursive functions and the divide-and-conquer strategy, with examples such as Fibonacci numbers, the Towers of Hanoi and quicksort.

Chapter 7: Object Oriented Programming (OOP)

The chapter contrasts procedure-oriented and object-oriented programming and introduces classes, objects, and the ideas of inheritance, encapsulation and polymorphism.

Chapters 8–9: Complexity; Representation of Numbers

Chapter 8 defines algorithmic complexity and Big-O notation, shows why complexity matters with examples, and demonstrates the profiler for finding slow code. Chapter 9 explains base-N and binary representation, IEEE 754 floating-point numbers and round-off error.

Chapters 10–11: Errors, Good Programming Practices, and Debugging; Reading and Writing Data

Chapter 10 distinguishes syntax errors, exceptions and logical errors, recommends practices that avoid errors, and covers try/except, type checking and the debugger. Chapter 11 shows how to read and write text, CSV, pickle, JSON and HDF5 files.

Chapters 12–13: Visualization and Plotting; Parallel Your Python

Chapter 12 covers 2D and 3D plotting with Matplotlib, drawing maps with Cartopy, and making animations and movies. Chapter 13 explains processes, threads and Python's global interpreter lock, then uses the multiprocessing package and joblib to run work on multiple cores.

Chapter 14: Linear Algebra and Systems of Linear Equations

Part II opens with a crash course in linear algebra: vectors, norms, linear independence, matrices with determinant, rank and inverse, and linear transformations. It represents systems as Ax = y, relates the number of solutions to ranks, and solves systems by Gaussian elimination, Gauss–Jordan elimination, LU decomposition and the Gauss–Seidel iteration, before showing NumPy and SciPy solvers and matrix inversion.

Chapter 15: Eigenvalues and Eigenvectors

After stating the eigenvalue problem, the chapter presents the power method for the largest eigenvalue, the inverse and shifted power methods, and the QR method for all eigenvalues, and then shows how to compute them with NumPy.

Chapter 16: Least Squares Regression

Least squares is derived twice—once with linear algebra and once with multivariable calculus—leading to the normal-equation formula for the parameters. The chapter fits data in Python and handles nonlinear models through log transformations, polynomial regression and SciPy's curve_fit.

Chapter 17: Interpolation

The chapter states the interpolation problem and presents linear interpolation, cubic splines, and Lagrange and Newton polynomial interpolation, with SciPy implementations.

Chapter 18: Series

Taylor series are used to represent and approximate functions; the chapter discusses linear approximation, estimates truncation error with the Taylor remainder and points out round-off error when summing series.

Chapter 19: Root Finding

After defining the problem and the idea of tolerance, the chapter presents bisection (stable but slower) and Newton–Raphson (fast near the root but potentially unstable), and shows SciPy's root-finding functions.

Chapters 20–21: Numerical Differentiation; Numerical Integration

Chapter 20 approximates derivatives with forward, backward and central finite differences, extends them to higher derivatives, and shows the difficulties caused by noisy data. Chapter 21 approximates integrals with Riemann sums, the trapezoid rule and Simpson's rule, compares their orders of accuracy, and uses SciPy integration routines.

Chapters 22–23: Ordinary Differential Equation – Initial Value Problems; Ordinary Differential Equation – Boundary Value Problems

Chapter 22 reduces higher-order ODEs to first-order systems and covers the explicit, implicit and trapezoidal Euler methods, numerical error and stability, predictor–corrector and Runge–Kutta methods, SciPy's solve_ivp, and brief advanced topics on multistep methods and stiffness. Chapter 23 solves boundary value problems with the shooting method and the finite difference method and discusses their accuracy and stability.

Chapter 24: Fourier Transform

Starting from the basic properties of waves (frequency, period, amplitude, wavelength), the chapter defines the discrete Fourier transform, explains how the fast Fourier transform exploits symmetry, and applies NumPy and SciPy FFT functions to signals.

Chapter 25: Introduction to Machine Learning

An online extension introduces supervised and unsupervised learning, with classification using support vector machines, regression, and k-means clustering, using scikit-learn on the iris data set.

Key Concepts

Chapter 1: Python Basics

• Jupyter Notebook: an interactive environment mixing code, output and text, used throughout the book.
• Data types: int, float, complex, string and boolean values, each with its own operations.

Chapter 2: Variables and Basic Data Structures

• Mutable and immutable containers: lists and dictionaries can change, whereas strings and tuples cannot.
• NumPy array: a fixed-type, multidimensional array supporting fast element-wise operations.

Chapters 3–4: Functions; Branching Statements

• Function scope: each function has its own memory block, receiving information only through inputs and returning it through outputs.
• Lambda function: an anonymous one-line function, often passed as an argument.
• Branching: if-elif-else statements that let code act differently under different conditions.

Chapters 5–6: Iteration; Recursion

• Loop: a for- or while-construct that repeats instructions.
• Recursion and divide and conquer: solving a problem by calling a function on smaller instances of itself.

Chapter 7: Object Oriented Programming (OOP)

• Class and object: a blueprint grouping data and methods, and an instance created from it.
• Inheritance: reusing attributes and methods of a parent class in a child class.

Chapters 8–9: Complexity; Representation of Numbers

• Big-O notation: a machine-independent description of how running time grows with input size.
• IEEE 754: the standard binary floating-point format, which limits precision and causes round-off error.

Chapters 10–11: Errors, Good Programming Practices, and Debugging; Reading and Writing Data

• Exception handling: try/except blocks that respond to runtime errors without stopping a program.
• File formats: text, CSV, pickle, JSON and HDF5 for storing and exchanging data.

Chapters 12–13: Visualization and Plotting; Parallel Your Python

• Matplotlib plotting: building 2D and 3D figures to communicate results.
• Process-based parallelism: using multiple processes (via multiprocessing or joblib) to use several CPU cores.

Chapter 14: Linear Algebra and Systems of Linear Equations

• Rank and solutions: comparing rank(A) with rank([A, y]) determines whether a system has zero, one or infinitely many solutions.
• LU decomposition: factoring A into lower and upper triangular matrices to solve systems efficiently.
• Gauss–Seidel iteration: an iterative method updating unknowns one at a time.

Chapter 15: Eigenvalues and Eigenvectors

• Power method: repeated multiplication by A to find the dominant eigenvalue and eigenvector.
• QR method: an iteration based on QR factorisation that finds all eigenvalues.

Chapter 16: Least Squares Regression

• Normal equations: the formula β = (AᵀA)⁻¹AᵀY for the least squares parameters.
• Linearising transformations: taking logarithms to fit exponential or power-law models with linear least squares.

Chapter 17: Interpolation

• Cubic spline: a piecewise cubic interpolant with continuous first and second derivatives.
• Lagrange and Newton polynomials: two forms of the single polynomial passing through all data points.

Chapter 18: Series

• Taylor series: representing a function as an infinite sum of polynomial terms around a point.
• Truncation error: the error from cutting off a series, estimated with the Taylor remainder.

Chapter 19: Root Finding

• Bisection: repeatedly halving an interval where the function changes sign.
• Newton–Raphson: iterating with the tangent line, converging quickly near a root.

Chapters 20–21: Numerical Differentiation; Numerical Integration

• Finite difference: approximating a derivative by a ratio of function differences on a grid, with accuracy depending on the scheme's order.
• Simpson's rule: integrating with quadratic approximations, more accurate than the trapezoid rule for smooth functions.

Chapters 22–23: ODE Initial and Boundary Value Problems

• Euler and Runge–Kutta methods: step-by-step schemes for initial value problems of increasing order.
• Stability: whether numerical errors grow, depending on the equation, method and step size.
• Shooting method: turning a boundary value problem into an initial value problem combined with root finding.

Chapter 24: Fourier Transform

• Discrete Fourier transform: converting a sampled signal from the time domain to the frequency domain.
• Fast Fourier transform: an efficient algorithm for the DFT that exploits symmetries.

Chapter 25: Introduction to Machine Learning

• Supervised versus unsupervised learning: learning from labelled data (classification, regression) versus finding structure in unlabelled data (clustering).
• Support vector machine: a classifier that separates classes with a maximum-margin boundary.

Critical Analysis

Strengths

• Truly beginner-friendly programming. The preface assumes no programming experience, only high school algebra and trigonometry and introductory calculus. Part I builds from the calculator and data types to functions, OOP, debugging, file formats, plotting and parallel computing, giving engineering students a practical toolkit before any numerical method appears.
• Interactive and runnable. The online edition is a set of Jupyter notebooks with code that readers are encouraged to run; frequent "TRY IT!" blocks (more than 200 across the site), plus TIP, WARNING, WHAT IS HAPPENING? and CONSTRUCTION boxes, guide hands-on work. The code is MIT-licensed.
• Methods paired with library tools. Most Part II chapters first implement a method from scratch and then show the corresponding NumPy or SciPy routine, so students understand both the algorithm and the professional tool.
• Broad, brief coverage by design. Each chapter is designed for no more than two lecture hours, giving a survey of linear algebra, eigenvalues, regression, interpolation, series, roots, differentiation, integration, ODEs and Fourier analysis in one course.

Limitations

• Shallow by intention. The preface states that no topic is covered in great depth; error analysis and convergence proofs are minimal, so students who need theory must turn to a numerical analysis text.
• Version-specific code. The book lists package versions from around 2019 (for example NumPy 1.16 and SciPy 1.2); some functions and outputs may differ in current releases.
• Uneven depth across topics. Some sections (for example multistep methods and stiffness) are only brief "advanced topics," and the machine learning chapter is a short online-only extension.
• Online text versus print. The site notes that the book's copyright belongs to Elsevier and describes the notebooks as an excerpt of the printed book; the machine learning chapter is an online extension, and the only appendix online covers installing Python on Windows.

Applications in the Book

Applications appear as examples and problems rather than separate chapters: plotting and mapping geographic data with Cartopy and making animations (Chapter 12), timing and parallelising computations (Chapters 8 and 13), fitting experimental data with linear and nonlinear least squares (Chapter 16), solving ODE models such as the pendulum and a rocket-launch boundary value problem with hand-written schemes and SciPy (Chapters 22–23), analysing signals with the FFT (Chapter 24), and classifying and clustering the iris data set (Chapter 25).

Where It Fits in the Course

This book sits alongside Cleve Moler's Numerical Computing with MATLAB. The two take a software-first approach, but differ in emphasis: Kong, Siauw and Bayen devote Part I to teaching programming from scratch in Python, whereas Moler assumes some programming and spends more space on how library algorithms work internally, using MATLAB. Students with no programming background or in Python-based courses should start here; those who already program may prefer Moler or use it as a companion.

Burden, Faires and Burden's Numerical Analysis and Süli and Mayers's An Introduction to Numerical Analysis are undergraduate texts generally described as more rigorous treatments with error analysis; they are the natural next step after Part II. Trefethen and Bau's Numerical Linear Algebra and Stoer and Bulirsch's Introduction to Numerical Analysis are graduate-level texts that suit study once the methods introduced here are familiar.

How to Use This Book

The book is aimed at first- or second-year engineering and science students with no programming experience. Prerequisites are high school algebra and trigonometry and introductory college calculus; Chapter 14 supplies the linear algebra needed later.

The preface advises reading Chapters 1–5 first because they cover programming fundamentals; Chapters 6–11 can be read in any order. A one-semester course can cover Part I quickly for students with some coding experience and then focus on Chapters 14–24. Read with a Jupyter notebook open, run every example, attempt all "TRY IT!" blocks, and work the Problems at the end of each chapter; use the Summary lists to check understanding. Because package versions change, compare outputs with current NumPy and SciPy documentation when they differ.

Edition and Sources

This guide is based on the free official online edition of Python Programming and Numerical Methods hosted by the authors at the University of California, Berkeley (https://pythonnumericalmethods.berkeley.edu/notebooks/Index.html; © 2020, print edition published by Academic Press/Elsevier; code under the MIT licence). It draws on the preface (purpose, prerequisites, organisation, reading advice, package versions), the full contents, every chapter outline and section page heading for Chapters 1–25 and the appendix, the end-of-chapter Summary lists for all chapters, and selected sections read more closely, including 9.2, 12.3, 13.1–13.3, 14.4, 15.3, 16.5, 18.3, 22.5, 22.7 and 25.2–25.4.