akabe/ocaml-jupyter-datascience

Travis CI	MicroBadger

A ready-to-use environment of Jupyter (IPython notebook) and OCaml Jupyter (OCaml kernel) with libraries for data science and machine learning.

Getting started

First, launch a Jupyter server as follows.

$ docker run -it -p 8888:8888 akabe/ocaml-jupyter-datascience
[I 15:38:04.170 NotebookApp] Writing notebook server cookie secret to /home/opam/.local/share/jupyter/runtime/notebook_cookie_secret
[W 15:38:04.190 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 15:38:04.197 NotebookApp] Serving notebooks from local directory: /notebooks
[I 15:38:04.197 NotebookApp] 0 active kernels
[I 15:38:04.197 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/?token=4df0fee0719115f474c8dd9f9281abed28db140d25f933e9
[I 15:38:04.197 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 15:38:04.198 NotebookApp] No web browser found: could not locate runnable browser.
[C 15:38:04.198 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=4df0fee0719115f474c8dd9f9281abed28db140d25f933e9

Second, access to the URL at the above last line to your web browser, then

You can create OCaml notebooks!

Notebooks on your host machine can be mounted to a Docker container like

docker run -it -p 8888:8888 -v $PWD:/notebooks akabe/ocaml-jupyter-datascience

Distributions

The default images are built on Debian 8:

CentOS

Debian

Pre-installed packages

Standard libraries

The OCaml standard library is too small in practical use. The following packages provide popular data structures, a lot of frequently-used functions such as string operations, various iteration on collections, etc.

• Jane Street Core (GitHub, API) — A huge extended standard library developed by Jane Street Capital. The library is actively maintained and reliable due to industrial use of Jane Street. Its interface is designed differently from the OCaml standard library.
• Batteries Included (GitHub, API) — A famous extended standard library compatible with the OCaml standard library. It is smaller than Jane Street Core, but commonly-used functions are implemented.

Numerical computation

• Lacaml (GitHub, API) — A binding to BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra PACKage), traditional linear algebra libraries written in Fortran. This library supplies basic operations on vectors and matrices (e.g., addition, dot product, multiplication), LU, QR, SVD, least-square fitting, etc.
• SLAP (GitHub, API) — A high-level wrapper of Lacaml with type-based static size checks for vectors and matrices.
  • notebooks/slap_two_layer_neural_network.ipynb
• GSL (GitHub, API) — A binding to GNU Scientific Library (GSL), a rich numerical analysis library containing interface to BLAS. This library contains eigenproblem solvers, least square fitting, pseudo-random number generators (such as Mersenne Twister), FFT (fast Fourier transform), Monte-Carlo simulation, etc.
  • notebooks/gaussian_random_walk.ipynb
  • notebooks/random_dataset_generation.ipynb
• Owl (GitHub, API) — a general-purpose numerical library: dense and sparse matrix, linear algebra, regressions, maths and stats functions.
  • notebooks/owl.ipynb
• FFTW3 (GitHub) — A binding to FFTW3, a major fast Fourier transform library.
  • notebooks/fftw3_example.ipynb
  • notebooks/formant_estimation_by_AR.ipynb
• libsvm (BitBucket, API) — Support vector machine (SVM) is a powerful model in machine learning. This is a binding to libsvm, a library for SVMs.
• TensorFlow (GitHub) — A binding to TensorFlow, a popular open-source neural network library developed by Google.
• L-BFGS (API) — A binding to L-BFGS-B, a quasi-Newton library for bound-constrained optimization.
• Ocephes (GitHub, API) — A binding to Ocephes, a library of special math functions like Binominal, Gaussian, Gamma distributions, incomplete Beta integral.
• Oml (GitHub, API) — A small library for numerical computing on OCaml.
• GPR (GitHub, API, PDF) — Efficient and scalable Gaussian Process Regression in OCaml.
• MeCab (GitHub, API) — an OCaml binding of MeCab, a part-of-speech and morphological analyzer. The following dictionaries are available:
  • ipadic at /usr/lib/mecab/dic/mecab-ipadic, and
  • mecab-ipadic-NEologd at /usr/lib/mecab/dic/mecab-ipadic-neologd.

Visualization

• Archimedes (API) — A 2D plot library like matplotlib in Python. You can embed PNG images in Jupyter notebooks.
  • notebooks/introduction.ipynb
• Cairo2 (GitHub, Tutorial, API) — A binding to Cairo, a 2D vector graphics library. This library is useful as a backend of Archimedes.
  • notebooks/simple_image_filtering.ipynb
• PLplot (GitHub) — a binding to PLplot, a practical library for creating scientific charts.
  • notebooks/plplot.ipynb

Data sources

• MySQL (GitHub, API) — A well-benchmarked MySQL client library providing blocking API.
• MariaDB (GitHub) — MariaDB is a rich relational database management system compatible with MySQL. This client library provides blocking and nonblocking (Lwt and Async) interfaces that can connect both of MySQL and MariaDB.
• Postgresql (GitHub, API) — PostgreSQL is a popular relational database with many aggregate and window functions useful for data analysis.
• SQLite3 (GitHub, API) — A client library for SQLite3, a easy-to-use lightweight database.
• Cohttp (GitHub, API) — A very lightweight HTTP(s) client/server library using Async and Lwt. This library is useful to access to API servers or Web scraping. The API documentation of ocaml-uri is also useful.
  • notebooks/cohttp_lwt_DuckDuckGoAPI.ipynb
  • notebooks/cohttp_async_DuckDuckGoAPI.ipynb

Concurrent programming

• Jane Street Async (GitHub, API) — Jane Street Capital's library for concurrent programming.
• Lwt (GitHub, API) — A library of cooperative light-weight threads.

Other packages

• Re (GitHub) — A fast and easy-to-use regular expression library for OCaml. This library supports Glob, POSIX, Perl, PCRE, and OCaml-Str-style syntaxes.
• Camomile (GitHub, API) — Camomile is a library for character encoding conversion and unicode utilities.
• LambdaSoup (GitHub, API) — Lambda Soup is a functional HTML scraping and manipulation library for OCaml aimed at being easy to use.
• OCaml CSV (GitHub, API) — A library to read and write comma-separated-values (CSV) format files.
• ppx_sexp_conv — Automatic generation of converters between OCaml datatypes and S-expressions.
• ppx_deriving_yojson — Automatic generation of converters between OCaml datatypes and JSONs.
• ppx_regexp — Pattern matching by PCRE-style regular expressions.

CUI tools

• ImageMagick — ImageMagick is a program to create, edit, compose, or convert bitmap images. This supports many formats, e.g., PNG, JPEG, GIF, TIFF, PDF, etc.
• FFmpeg — FFmpeg is a powerful tool for converting audio and video files.
• PhantomJS — PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

Examples

$ git clone https://github.com/akabe/docker-ocaml-jupyter-datascience.git
$ docker run -it -p 8888:8888 -v $PWD/docker-ocaml-jupyter-datascience/notebooks:/notebooks akabe/ocaml-jupyter-datascience

Contribution

If you know a widely-used numerical library in OCaml, find a bug, or have an idea to improve this environment, please create an issue or pull-request your changes.