Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit

Citation

Bird, S., Klein, E., & Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O'Reilly Media. Free official edition (online version updated for Python 3 and NLTK 3): https://www.nltk.org/book/

Chapter Summary

Chapter 0: Preface

The preface defines natural language processing broadly—from counting word frequencies to "understanding" utterances—and positions the book as a practical, introductory-to-intermediate tutorial built on Python and the open-source Natural Language Toolkit (NLTK). It addresses readers new to programming, new to Python and experienced in Python, explains that chapters alternate between language-driven and programming-driven presentation, describes graded exercises (☼ easy, ◑ intermediate, ★ difficult), and offers two course plans, one for arts and humanities students and one for science and engineering students.

Chapter 1: Language Processing and Python

The first chapter gets readers computing with text immediately: loading NLTK's sample texts, searching with concordances, counting vocabulary, and treating texts as Python lists. It introduces frequency distributions, collocations and bigrams, conditionals and loops, and ends with a non-technical tour of what "understanding" language would require—word sense disambiguation, pronoun resolution, generation, machine translation, dialogue systems and textual entailment—and the limits of NLP.

Chapter 2: Accessing Text Corpora and Lexical Resources

This chapter surveys corpora shipped with NLTK (Gutenberg, web and chat text, Brown, Reuters, inaugural addresses, annotated and multilingual corpora) and shows how to load one's own corpus. Conditional frequency distributions are used to compare genres and to generate random text from bigrams. On the Python side it introduces text editors, functions and modules. It then covers lexical resources—wordlists, the CMU pronouncing dictionary, comparative wordlists, Toolbox lexicons—and WordNet's synsets, hierarchy, lexical relations and semantic similarity.

Chapter 3: Processing Raw Text

Here readers learn to work with text from the web and from disk: strings, Unicode, regular expressions for finding word patterns, normalisation (stemming and lemmatisation), tokenisation with regular expressions, sentence and word segmentation, and formatting output. The chapter makes the step from ready-made corpora to messy real-world text.

Chapter 4: Writing Structured Programs

This chapter consolidates the Python scattered through Chapters 1–3: assignment and object references, sequences, programming style, functions (parameters, scope, docstrings), program development and debugging, algorithm design including recursion and dynamic programming, and a sample of Python libraries (plotting, NetworkX, CSV, NumPy).

Chapter 5: Categorizing and Tagging Words

Part-of-speech tagging is introduced through tagged corpora and a universal tagset, with Python dictionaries as the data structure for mapping words to properties. The chapter builds and evaluates automatic taggers—default, regular-expression, lookup, unigram and n-gram taggers combined by backoff—discusses the sparse data problem and performance limits, introduces transformation-based (Brill) tagging, and asks how linguists decide a word's category from morphological, syntactic and semantic clues.

Chapter 6: Learning to Classify Text

The chapter introduces supervised classification through examples such as name-gender identification, document classification, part-of-speech tagging, sentence segmentation, dialogue act types and textual entailment, with attention to feature selection and sequence classification. It covers evaluation (test sets, accuracy, precision and recall, confusion matrices, cross-validation) and explains decision trees, naive Bayes and maximum entropy classifiers, ending with what such models can tell us about language.

Chapter 7: Extracting Information from Text

Information extraction is presented as a pipeline from segmentation and tagging to entity detection and relation extraction. The core of the chapter is chunking: noun-phrase chunking with tag patterns and regular expressions, chinking, IOB representation and the CoNLL-2000 corpus, evaluating and training classifier-based chunkers, and cascaded chunkers that build nested trees. It closes with named entity recognition and relation extraction.

Chapter 8: Analyzing Sentence Structure

The chapter motivates syntax through unlimited sentence possibilities and pervasive ambiguity, then introduces context-free grammars, writing one's own grammars, and recursion. It compares parsing algorithms (recursive descent, shift-reduce, left-corner, well-formed substring tables/chart parsing), presents dependency grammar and valency, and discusses grammar development with treebanks, "pernicious" ambiguity and weighted (probabilistic) grammars.

Chapter 9: Building Feature Based Grammars

Feature structures are introduced to handle agreement and other fine-grained distinctions without multiplying categories. The chapter explains attribute–value matrices, re-entrancy, subsumption and unification, and extends feature-based grammars to subcategorisation, heads, auxiliary inversion, unbounded dependency constructions, and case and gender in German.

Chapter 10: Analyzing the Meaning of Sentences

Starting from querying a database in English, the chapter introduces propositional and first-order logic, theorem proving, truth in a model, variable assignments, quantification and quantifier scope ambiguity, and model building. It then shows how to build meaning representations compositionally with feature-based grammars and the λ-calculus (quantified noun phrases, transitive verbs) and ends with Discourse Representation Theory and discourse processing.

Chapter 11: Managing Linguistic Data

The final technical chapter treats corpora as designed objects: a case study of the TIMIT corpus, the life cycle of a corpus (creation scenarios, quality control, curation), acquiring and converting data from the web, word processors, spreadsheets and databases, choosing annotation layers and standards, special considerations for endangered languages, working with XML and ElementTree, processing Toolbox lexicons, and describing resources with OLAC metadata.

Chapter 12: Afterword: The Language Challenge

The afterword reflects on the field: the contrast between language processing and symbol processing, contemporary divides between rationalist and empiricist and between realist and idealist approaches, the role of linguistic knowledge versus statistical methods, and a roadmap of NLTK's future development.

Key Concepts

Chapter 0: Preface

• NLTK: an open-source Python library with software, corpora and documentation for teaching and research in NLP.
• Graded exercises: easy, intermediate and difficult exercises at the end of every chapter.

Chapter 1: Language Processing and Python

• Token versus type: a particular occurrence of a word versus the distinct word form.
• Frequency distribution: a count of how often each item (such as each word) occurs.
• Collocation: a sequence of words that occurs together unusually often.

Chapter 2: Accessing Text Corpora and Lexical Resources

• Corpus: a large, structured collection of texts, often categorised by genre or topic.
• Conditional frequency distribution: a set of frequency distributions, one per condition such as genre.
• WordNet synset: a set of synonymous words representing one sense, linked into a semantic hierarchy.

Chapter 3: Processing Raw Text

• Tokenisation: segmenting a text into words, punctuation and other units.
• Regular expression: a pattern language for finding and manipulating strings.
• Stemming and lemmatisation: reducing word forms to a stem or dictionary form.

Chapter 4: Writing Structured Programs

• Object references: assignment in Python shares objects rather than copying them.
• Dynamic programming: storing results of subproblems to avoid recomputation, widely used in NLP.

Chapter 5: Categorizing and Tagging Words

• Part-of-speech tagging: assigning word classes such as noun or verb to tokens.
• Backoff: falling back from a specialised model (bigram tagger) to a more general one (unigram tagger) when needed.
• Sparse data problem: most possible contexts are unseen even in large training data.

Chapter 6: Learning to Classify Text

• Supervised classification: learning to predict labels from features using labelled training data.
• Training, dev-test and test sets: separate data for building, tuning and finally evaluating a model.
• Precision and recall: measures of how many predicted items are correct and how many correct items are found.

Chapter 7: Extracting Information from Text

• Chunking: segmenting and labelling multi-token sequences such as noun phrases.
• IOB tags: Inside/Outside/Begin labels for representing chunks.
• Named entity recognition: identifying mentions of people, organisations, locations and similar entities.

Chapter 8: Analyzing Sentence Structure

• Context-free grammar: productions that say how phrases of each category decompose into parts.
• Parsing algorithms: recursive descent (top-down), shift-reduce (bottom-up) and chart methods.
• Dependency grammar: analysing structure as relations between heads and dependents.

Chapter 9: Building Feature Based Grammars

• Feature structure: a set of attribute–value pairs describing grammatical properties.
• Unification: combining two feature structures into one containing the information of both, if compatible.

Chapter 10: Analyzing the Meaning of Sentences

• Truth in a model: evaluating logical formulas against a domain and a valuation.
• λ-calculus and β-reduction: tools for building meaning representations compositionally.
• Discourse Representation Theory: representing meaning across sentences, including anaphora.

Chapter 11: Managing Linguistic Data

• Corpus life cycle: collection, annotation, quality control, publication and later enrichment.
• Metadata (OLAC): standard descriptions that let others discover language resources.

Chapter 12: Afterword

• Rationalism versus empiricism: competing views on the role of innate knowledge versus data in language processing.

Critical Analysis

Strengths

• Learning by doing. As the preface promises, almost every idea is accompanied by runnable code using NLTK's bundled corpora, so readers can test claims immediately; the chapters contain hundreds of worked examples and graded exercises.
• Accessible to humanities students. The early chapters teach Python from scratch through language examples, Chapter 4 consolidates programming later, and the preface offers a separate course plan for arts and humanities students.
• Linguistically grounded. Unlike many programming-first NLP books, it devotes full chapters to tagset design and word categories, syntax and parsing, feature-based grammar, formal semantics and corpus management, including endangered-language documentation and Toolbox data.
• Clear teaching of evaluation. Chapters 5–7 repeatedly separate training and test data, introduce baselines, accuracy, precision, recall and cross-validation, and discuss why results can be over-optimistic.

Limitations

• Pre-neural content. The material reflects NLP up to about 2009: classifiers are decision trees, naive Bayes and maximum entropy, and there is no coverage of word embeddings, neural networks or large language models, which dominate current practice.
• Online version partly unfinished. The book's index page marks Chapters 5, 10 and 11 as needing "minor fixes", and the online text was updated for Python 3 and NLTK 3.0 (assuming Python 3.2 or later); some code may still need adjustment for current NLTK releases.
• Tied to one toolkit. Much of the text teaches NLTK's own classes and corpora, so skills transfer less directly to other libraries used in industry and research today.
• Uneven difficulty. The preface notes that the pace picks up after Chapter 4; Chapters 9 and 10 (feature grammars, λ-calculus) are considerably harder than the opening chapters.

Where It Fits in the Course

In the Computational and Corpus Linguistics list, this book sits between the programming-free introductions and the more technical titles. Language and Computers explains what language technology does without programming, and Doing Corpus Linguistics introduces corpus methods; either is a good first step, after which this book teaches students to carry out such analyses themselves in Python. McEnery and Hardie's Corpus Linguistics: Method, Theory and Practice is generally described as a methodological and theoretical treatment of corpus linguistics and complements Chapters 1–3 and 11 here, while Gries's Quantitative Corpus Linguistics with R takes a statistical, R-based route through similar data.

Jurafsky and Martin's Speech and Language Processing is the natural next step after this book: it covers algorithms and models in greater depth and, in its current draft, the neural methods this book omits. Students can use Bird, Klein and Loper to gain hands-on skill with tagging, parsing and classification before turning to Jurafsky and Martin for theory and modern approaches.

How to Use This Book

The book is designed for readers with no programming background as well as for experienced programmers, and for courses in NLP, computational linguistics, corpus linguistics or text mining. Install Python 3 and NLTK, download the NLTK data, and type the examples as you read. The preface's table of suggested lectures gives an arts and humanities course two to four lectures on each of Chapters 1–5, 8 and 9 and little or no time on Chapter 6, while a science and engineering course moves faster through Chapters 1–4 and spends more time on Chapters 6–11. Another option the preface suggests is to cover Chapters 1–5 and then specialise in classification (6–7), syntax (8–9), semantics (10) or data management (11).

Work through at least the easy (☼) and intermediate (◑) exercises in each chapter, and use the further reading sections to connect the code to the research literature. Because the field has moved on, pair Chapters 5–7 with a modern account of machine learning for NLP if your course covers current methods.

Edition and Sources

This guide is based on the online edition of Natural Language Processing with Python at https://www.nltk.org/book/, which the authors have updated for Python 3 and NLTK 3 (the first edition was published by O'Reilly in 2009; the site states there are no plans for a second edition). It draws on the preface (audience, emphasis, organisation, exercises, Python 3 notes and course plans), the full table of contents and every section heading of Chapters 1–11 and the Afterword, together with each chapter's opening questions and end-of-chapter summary.