Adventures in Stochastic Processes

Citation

Resnick, S. I. (2002). Adventures in Stochastic Processes. Birkhäuser.

Chapter Summary

Chapter 1: Preliminaries (Discrete Index Sets and/or Discrete State Spaces)

The opening chapter reviews techniques for non-negative integer-valued random variables and their distributions, then applies them to the simple random walk and to a further example the author calls the ω process. Near the end it introduces stopping times and uses them to obtain Wald's identity and some facts about the random walk. The author tells beginners they may skip the advanced discussion of sigma-fields, needing only a rough idea that sigma-fields organise information within a probability space.

Chapter 2: Markov Chains

A substantial chapter of about 115 pages, it starts from the observation that realistic models must allow dependence: next week's purchases depend on past satisfaction, tomorrow's stock level on today's, a queue on earlier queue lengths. It develops Markov chains on a discrete state space, including the dissection principle, a way of cutting a chain into independent, identically distributed blocks at its returns to a reference state. That regenerative idea is reused in later chapters.

Chapter 3: Renewal Theory

Renewal processes model events whose gaps are independent and identically distributed. The chapter presents renewal theory as one of the basic building blocks of applied probability, because a complicated model often has a renewal process embedded in it. It centres on regeneration, the decomposition of a process into iid blocks of random length. At roughly 125 pages it is the longest chapter in the book.

Chapter 4: Point Processes

Point processes describe a random scattering of points in a subset of R, [0, ∞) or R^d. The chapter builds on two earlier examples: renewal processes, whose gaps are iid, and the Poisson process on [0, ∞), which is the renewal process with exponential gaps. It is a comparatively short chapter of about 65 pages.

Chapter 5: Continuous Time Markov Chains

The Markov property is moved to continuous time while keeping a discrete state space, usually {0, 1, …}. The sequence of visited states is still a discrete-time chain, but time is perturbed by exponentially distributed holding times in each state. A generalisation of the Chapter 2 dissection argument shows that the process regenerates at returns to a fixed state, so renewal theory and regenerative processes become tools here.

Chapter 6: Brownian Motion

The chapter introduces Brownian motion (the Wiener process), first proposed by the botanist Robert Brown as a model for a particle buffeted by molecular collisions. It is presented as central to the modern theory of stochastic processes and of large-sample statistics, and as basic to descriptions of financial markets, the construction of diffusions, queueing approximations and asymptotic distributions in estimation.

Chapter 7: The General Random Walk

The final chapter defines a random walk as the partial sums S_n of iid real-valued steps with a given step distribution. The author labels it advanced material not suitable for beginning readers.

Key Concepts

Chapter 1: Preliminaries

• Stopping time: a random time whose occurrence is decided by the information available so far; the chapter uses it to derive Wald's identity.
• Simple random walk: the first major example, treated with the tools for non-negative integer-valued variables.

Chapters 2–3: Markov Chains; Renewal Theory

• Dissection principle: cutting a Markov chain into iid blocks at returns to a reference state.
• Regeneration: the idea that a process starts afresh at certain random times, so that renewal theory applies to it.
• Renewal process: a point process on [0, ∞) whose gaps between events are iid random variables.

Chapters 4–5: Point Processes; Continuous Time Markov Chains

• Poisson process: a renewal process with exponential gaps, and the basic example of a point process.
• Holding times: the exponentially distributed times a continuous-time chain spends in each state.

Chapters 6–7: Brownian Motion; The General Random Walk

• Brownian motion (Wiener process): the continuous-time model of random motion that underlies diffusions and much of large-sample theory.
• Step distribution: the common distribution F of the iid increments of a random walk.

Critical Analysis

Strengths

• Modular design for different readers. The publisher describes the discussion as carefully modularised, with a sharp distinction between rigorous proof and plausibility argument. A reader can follow the ideas first and return to the proofs later. The author also marks material that beginners may skip (the sigma-field discussion in Chapter 1) or should avoid (Chapter 7).
• No measure theory required. The stated prerequisite is a graduate-level probability course that is not measure-theoretic, and advanced notions of conditioning are postponed to the later chapters.
• Construction as well as analysis. The author includes constructions of each class of process studied, on the view that people who build models should know how to build them. The listed applications are queuing, storage, risk analysis, genetics, inventory, choice, economics and sociology.
• Reception. Reviewers quoted by the publisher single out the lucid writing and the choice of exercises (Gian-Carlo Rota), and the clear motivation of the deeper theory, the examples and the exercises (Zentralblatt MATH). Choice and Internationale Mathematische Nachrichten also recommend it as a textbook.

Limitations

• Not a first look at probability. The book assumes a graduate-level course in non-measure-theoretic probability, so students without it will need a more elementary text first.
• No stochastic calculus. The seven chapters stop at Brownian motion as a process. There is no chapter on stochastic integration or stochastic differential equations, so readers heading toward finance or diffusions need a second book.
• Heavy front-loading of discrete theory. Chapters 1–3 occupy nearly half of the roughly 610 pages of main text, so a reader mainly interested in Brownian motion has a long road before Chapter 6.

Where It Fits in the Course

Within the stochastic processes list, this is the general graduate-level text on the classical toolkit. Sheldon Ross's Introduction to Probability Models covers Markov chains, Poisson processes and their applications at an undergraduate level and is the easier route into the same territory. Dobrow's Introduction to Stochastic Processes with R pairs a similar syllabus with simulation and finishes with a gentle first look at stochastic calculus. Resnick goes further in rigour and in renewal and point-process theory, and is a natural next step after either.

For continuous-time finance the sequence is different. Shreve's Stochastic Calculus for Finance I is a discrete-time introduction, and Øksendal's Stochastic Differential Equations, Lamberton and Lapeyre's Introduction to Stochastic Calculus Applied to Finance and Björk's Arbitrage Theory in Continuous Time all build on Brownian motion and stochastic integrals. Resnick supplies the Brownian motion and random walk background for these books, but not the integration theory.

How to Use This Book

The book suits first-year graduate students in statistics, operations research, applied mathematics and engineering who have already taken a non-measure-theoretic probability course. Strong undergraduates can use it for a second course in stochastic processes.

A workable route is Chapter 1 (skipping the sigma-field material at first), then Chapters 2, 3 and 5 in order, since Chapter 5 reuses the dissection argument of Chapter 2 and the regeneration ideas of Chapter 3. Chapter 4 can be read after Chapter 3, because it builds on renewal processes and the Poisson process. Readers mainly interested in Brownian motion can look at Chapter 6 early and go back as needed. Chapter 7 is for a second pass or for readers who want the general theory of random walks.

On the first reading, follow the plausibility arguments and leave the rigorous proofs for later. Work the exercises, because the book relies on them to build the models it describes.