Blueprints for Text Analytics Using Python

Machine Learning Based Solutions for Common Real World (NLP) Applications

Jens Albrecht, Sidharth Ramachandran, Christian Winkler

Published by O'Reilly, 2020

Find the book at
O'Reilly
Amazon.com
Amazon.de
Amazon.co.uk
Amazon.fr
Amazon.in

If you like the book or the code examples here, please leave a friendly comment on Amazon!

Download your free chapter now!

Free download of Chapter 7 "How to Explain a Classifier".

Content of this Repository

This repository is currently in preparation. Please do not yet send any comments.

This repository contains the code examples of our O'Reilly book. You will find a subdirectory for each chapter containing a Jupyter notebook and additional files for the setup.

Below you find the links to view the notebooks here on Github or execute them directly on Google Colab. In the section thereafter you will find instructions to setup the environment on your local computer.

Problems and errors

If you discover any problems or have recommendations on how to improve the code, do not hesitate to create an issue here in the repository.

For errors in the book text, please use O'Reilly's errata page.

spaCy 3.0 and Gensim 4.0

The book uses spaCy 2.3.2 and gensim 3.8.3. spaCy 3.0 is now officially release with several new features and a few API changes (https://spacy.io/usage/v3). Gensim 4.0 is in beta (https://github.com/RaRe-Technologies/gensim/releases).

We are already updating our notebooks. But currently textacy is not yet supporting spaCy 3.0, although work is already in progress (see this pull request from us). Until textacy for spaCy 3.0 is released, you can use our own fork for the installation (see blueprints.yaml in this directory).

View or Run the Notebooks

For each chapter of the book we provide three links:

• "git" opens the notebook for viewing here on Github (sometimes not working because of Github issue)
• "nbviewer" opens the notebook for viewing on nbviewer.ipython.org
• "colab" opens a runnable copy on Google's Colab service

If you run the notebook locally or on Colab, you can execute each cell separately by hitting "Shift-enter". Do not skip cells and don't forget to run the first code cells for the setup.

• Chapter 1: Gaining First Insights from Textual Data [git] [nbviewer] [colab]
• Chapter 2: Extracting Textual Insights with APIs [git] [nbviewer] [colab]
• Chapter 3: Scraping Websites and Extracting Data [git] [nbviewer] [Colab]
• Chapter 4: Preparing Textual Data For Statistics and Machine Learning [git] [nbviewer] [colab]
• Chapter 5: Feature Engineering and Syntactic Similarity [git] [nbviewer] [Colab]
• Chapter 6: Text Classification Algorithms [git] [nbviewer] [Colab]
• Chapter 7: How to Explain a Text Classifier [git] [nbviewer] [Colab]
• Chapter 8: Unsupervised Methods: Topic Modeling and Clustering [git] [nbviewer] [Colab]
• Chapter 9: Text Summarization [git] [nbviewer] [Colab]
• Chapter 10: Exploring Semantic Relationships with Word Embeddings [git] [nbviewer] [colab]
• Chapter 11: Performing Sentiment Analysis on Text Data [git] [nbviewer] [colab]
• Chapter 12: Building a Knowledge Graph [git] [nbviewer] [colab]
• Chapter 13: Using Text Analytics in Production [git] [app]

Local Setup

The following instructions should work on Linux, Windows and MacOS. If you are a Windows user familiar with Linux, you should check out the Windows Subsystem for Linux, Version 2 (WSL2). This allows to use a Linux system on the Windows machine. However, using native Windows should also be no problem.

It is helpful to install git on your machine, but you can also download the full repository from Github as a zip file. If you use git, run the following commands from the command line:

git clone https://github.com/blueprints-for-text-analytics-python/blueprints-text.git
cd blueprints-text

Otherwise download the zip file, unpack it to a location convenient to you, and open a command line terminal in the project directory blueprints-text.

For local setup, we recommend to use Miniconda, a minimal version of the popular Anaconda distribution that contains only the package manager conda and Python. Follow the installation instructions on the Miniconda Homepage. If you already have Anaconda or Miniconda installed on your system: That's fine. We will create a separate virtual environment for the blueprints book so that our installation will not interfere with your previous setup.

After installation of Anaconda/Miniconda, run the following command(s) from the project directory:

conda env create --name blueprints --file blueprints.yml
conda activate blueprints

The prompt should change after activation and indicate that you are working in the blueprints environment. Our installation includes the Jupyter notebook extensions. We suggest to enable the extensions "table of contents" (toc2), "execute time", and "variable inspector" (varInspector):

jupyter nbextension enable toc2/main
jupyter nbextension enable execute_time/ExecuteTime
jupyter nbextension enable varInspector/main

Now you can start the Jupyter notebook server:

jupyter notebook

If working on WSL under Windows, add --no-browser.

Browse to the respective chapter and open the notebook file (suffix .ipynb)