All Projects â†’ computationalmodelling â†’ Nbval

computationalmodelling / Nbval

Licence: other
A py.test plugin to validate Jupyter notebooks

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Nbval

Ipytest
Pytest in IPython notebooks.
Stars: ✭ 139 (-59.94%)
Mutual labels:  jupyter-notebook, ipython-notebook, pytest
Testbook
🧪 📗 Unit test your Jupyter Notebooks the right way
Stars: ✭ 146 (-57.93%)
Mutual labels:  jupyter-notebook, pytest
Sqlcell
SQLCell is a magic function for the Jupyter Notebook that executes raw, parallel, parameterized SQL queries with the ability to accept Python values as parameters and assign output data to Python variables while concurrently running Python code. And *much* more.
Stars: ✭ 145 (-58.21%)
Mutual labels:  jupyter-notebook, ipython-notebook
Ipywebrtc
WebRTC for Jupyter notebook/lab
Stars: ✭ 171 (-50.72%)
Mutual labels:  jupyter-notebook, ipython-notebook
Dive Into Machine Learning
Dive into Machine Learning with Python Jupyter notebook and scikit-learn! First posted in 2016, maintained as of 2021. Pull requests welcome.
Stars: ✭ 10,810 (+3015.27%)
Mutual labels:  jupyter-notebook, ipython-notebook
Ipytracer
📊 Algorithm Visualizer for IPython/Jupyter Notebook
Stars: ✭ 138 (-60.23%)
Mutual labels:  jupyter-notebook, ipython-notebook
Rfm Analysis
Python script (and IPython notebook) to perform RFM analysis from customer purchase history data
Stars: ✭ 165 (-52.45%)
Mutual labels:  jupyter-notebook, ipython-notebook
Notebooks
A collection of Jupyter/IPython notebooks
Stars: ✭ 78 (-77.52%)
Mutual labels:  jupyter-notebook, ipython-notebook
Relevant Search Book
Code and Examples for Relevant Search
Stars: ✭ 231 (-33.43%)
Mutual labels:  jupyter-notebook, ipython-notebook
pytest-notebook
A pytest plugin for regression testing and regenerating Jupyter Notebooks
Stars: ✭ 35 (-89.91%)
Mutual labels:  ipython-notebook, pytest
Notebooks Statistics And Machinelearning
Jupyter Notebooks from the old UnsupervisedLearning.com (RIP) machine learning and statistics blog
Stars: ✭ 270 (-22.19%)
Mutual labels:  jupyter-notebook, ipython-notebook
Prototypical Networks Tensorflow
Tensorflow implementation of NIPS 2017 Paper "Prototypical Networks for Few-shot Learning"
Stars: ✭ 122 (-64.84%)
Mutual labels:  jupyter-notebook, ipython-notebook
Spark Py Notebooks
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
Stars: ✭ 1,338 (+285.59%)
Mutual labels:  jupyter-notebook, ipython-notebook
Vscodejupyter
Jupyter for Visual Studio Code
Stars: ✭ 337 (-2.88%)
Mutual labels:  jupyter-notebook, ipython-notebook
Sci Pype
A Machine Learning API with native redis caching and export + import using S3. Analyze entire datasets using an API for building, training, testing, analyzing, extracting, importing, and archiving. This repository can run from a docker container or from the repository.
Stars: ✭ 90 (-74.06%)
Mutual labels:  jupyter-notebook, ipython-notebook
Machine Learning With Python
Practice and tutorial-style notebooks covering wide variety of machine learning techniques
Stars: ✭ 2,197 (+533.14%)
Mutual labels:  jupyter-notebook, pytest
Pandas Tutorial
Tutorial on Using Pandas
Stars: ✭ 66 (-80.98%)
Mutual labels:  jupyter-notebook, ipython-notebook
Show ast
An IPython notebook plugin for visualizing ASTs.
Stars: ✭ 76 (-78.1%)
Mutual labels:  jupyter-notebook, ipython-notebook
Evolutionary Computation Course
Jupyter/IPython notebooks about evolutionary computation.
Stars: ✭ 173 (-50.14%)
Mutual labels:  jupyter-notebook, ipython-notebook
Scipy Cookbook
Scipy Cookbook
Stars: ✭ 326 (-6.05%)
Mutual labels:  jupyter-notebook, ipython-notebook

Py.test plugin for validating Jupyter notebooks

Build Status PyPI Version Documentation Status

The plugin adds functionality to py.test to recognise and collect Jupyter notebooks. The intended purpose of the tests is to determine whether execution of the stored inputs match the stored outputs of the .ipynb file. Whilst also ensuring that the notebooks are running without errors.

The tests were designed to ensure that Jupyter notebooks (especially those for reference and documentation), are executing consistently.

Each cell is taken as a test, a cell that doesn't reproduce the expected output will fail.

See docs/source/index.ipynb for the full documentation.

Installation

Available on PyPi:

pip install nbval

or install the latest version from cloning the repository and running:

pip install .

from the main directory. To uninstall:

pip uninstall nbval

How it works

The extension looks through every cell that contains code in an IPython notebook and then the py.test system compares the outputs stored in the notebook with the outputs of the cells when they are executed. Thus, the notebook itself is used as a testing function. The output lines when executing the notebook can be sanitized passing an extra option and file, when calling the py.test command. This file is a usual configuration file for the ConfigParser library.

Regarding the execution, roughly, the script initiates an IPython Kernel with a shell and an iopub sockets. The shell is needed to execute the cells in the notebook (it sends requests to the Kernel) and the iopub provides an interface to get the messages from the outputs. The contents of the messages obtained from the Kernel are organised in dictionaries with different information, such as time stamps of executions, cell data types, cell types, the status of the Kernel, username, etc.

In general, the functionality of the IPython notebook system is quite complex, but a detailed explanation of the messages and how the system works, can be found here

https://jupyter-client.readthedocs.io/en/latest/messaging.html#messaging

Execution

To execute this plugin, you need to execute py.test with the nbval flag to differentiate the testing from the usual python files:

py.test --nbval

You can also specify --nbval-lax, which runs notebooks and checks for errors, but only compares the outputs of cells with a #NBVAL_CHECK_OUTPUT marker comment.

py.test --nbval-lax

The commands above will execute all the .ipynb files and 'pytest' tests in the current folder. Specify -p no:python if you would like to execute notebooks only. Alternatively, you can execute a specific notebook:

py.test --nbval my_notebook.ipynb

By default, each .ipynb file will be executed using the kernel specified in its metadata. You can override this behavior by passing either --nbval-kernel-name mykernel to run all the notebooks using mykernel, or --current-env to use a kernel in the same environment in which pytest itself was launched.

If the output lines are going to be sanitized, an extra flag, --nbval-sanitize-with together with the path to a confguration file with regex expressions, must be passed, i.e.

py.test --nbval my_notebook.ipynb --nbval-sanitize-with path/to/my_sanitize_file

where my_sanitize_file has the following structure.

[Section1]
regex: [a-z]*
replace: abcd

regex: [1-9]*
replace: 0000

[Section2]
regex: foo
replace: bar

The regex option contains the expression that is going to be matched in the outputs, and replace is the string that will replace the regex match. Currently, the section names do not have any meaning or influence in the testing system, it will take all the sections and replace the corresponding options.

Coverage

To use notebooks to generate coverage for imported code, use the pytest-cov plugin. nbval should automatically detect the relevant options and configure itself with it.

Parallel execution

nbval is compatible with the pytest-xdist plugin for parallel running of tests. It does however require the use of the --dist loadscope flag to ensure that all cells of one notebook are run on the same kernel.

Help

The py.test system help can be obtained with py.test -h, which will show all the flags that can be passed to the command, such as the verbose -v option. Nbval's options can be found under the Jupyter Notebook validation section.

Acknowledgements

This plugin was inspired by Andrea Zonca's py.test plugin for collecting unit tests in the IPython notebooks (https://github.com/zonca/pytest-ipynb).

The original prototype was based on the template in https://gist.github.com/timo/2621679 and the code of a testing system for notebooks https://gist.github.com/minrk/2620735 which we integrated and mixed with the py.test system.

We acknowledge financial support from

  • OpenDreamKit Horizon 2020 European Research Infrastructures project (#676541), http://opendreamkit.org

  • EPSRC’s Centre for Doctoral Training in Next Generation Computational Modelling, http://ngcm.soton.ac.uk (#EP/L015382/1) and EPSRC’s Doctoral Training Centre in Complex System Simulation ((EP/G03690X/1),

  • The Gordon and Betty Moore Foundation through Grant GBMF #4856,by the Alfred P. Sloan Foundation and by the Helmsley Trust.

Authors

2014 - 2017 David Cortes-Ortuno, Oliver Laslett, T. Kluyver, Vidar Fauske, Maximilian Albert, MinRK, Ondrej Hovorka, Hans Fangohr

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].