HTMpandaVis

UPDATE 9/2020 - major change - using solely NetworkAPI of HTM.core - fully AUTOMATIC baking process ! UPDATE 6/2020 - major change - SQLITE3 instead of TCP sockets + Dash plots visualization in web browser !

See video tutorial or presentation video!

Screenshots for visualization of the 2D recognition project

This project aspires to create tool that helps visualize HTM systems in 3D by using opensource framework for 3D rendering https://www.panda3d.org/

It allows to see architecture of the system in 3D space, e.g. connection of several layers and inputs and to see input representation, activity of columns and even individual cells in each simulation step. User can observe vast scalable space by moving as "ghost" and interact with objects. It is supposed as tool for educational purpose or as an inspect tool.

I was inspired by following:

• HTM school Episode 10 visualization - Topology
• Highbrow
• Sanity

The visualization is application written purely in Python3.

How it works

• Data for visualization are generated by so called "baking". This process generates sqlite3 database file, optionally folder with binary dump files.
• User can open these data in HTMpandaVis and explore them

You can also browse SQLite3 dabase file (.db) directly with ordinary browser such as Sqlite Browser.

Baking

To bake your simulation, you must use htm.core networkAPI. Only difference is, that you need to import instead of from htm.bindings.engine_internal import Network this from pandaBaker.pandaNetwork import Network

That is all. PandaVis will generate automatically all data.

If you don't want to change imports and keep using panda, you can modify flag Network.bakePandaData = True/False

See baking readme.

Dash plots visualization

HTMpandaVis can be used also to record custom dataStreams with pandaBaker and then visualize it in web browser. Dash plotly library is used. These plots are interactive!. For creating layout arrangement, axis and plot labels, there are JSON layout configuration files. They are located in HTMpandaVis\dashVis\layouts. See hotgym example for more informations.

Usage with databaseRegion from htm.core networkAPI

Within htm.core, there is databaseRegion in networkAPI that allows to output scalar values into sqliteDatabase. Data can be then plot in the same way as above. See htm.core examples/napi_hello_database.

How to install on Linux

Python >3.6 is recommended. Also using one of the python environment managers is recommended, like Anaconda

Install htm.core (here building from source, see repo readme if you need other installation instructions)

sudo apt-get install cmake
git clone https://github.com/htm-community/htm.core.git
python setup.py install --user --force

Install prerequisities & clone pandaVis

sudo apt-get install python3-tk

git clone https://github.com/htm-community/HTMpandaVis.git

python -m pip install -r requirements.txt

 #this installs pandaBaker package for baking process
python setup.py install

Run example

There is NetworkAPI hotgym example in the htm.core repository.

All you have to do to bake it, is changing one line!

1. For baking change : from htm.bindings.engine_internal import Network to from pandaBaker.pandaNetwork import Network

when you run the script, database and dump files will be created inside script folder in /bakedDatabase

2. Run client - pandaVis tool

cd HTMpandaVis
python run.py

And choose run 3D explorer or "run both" if you want to run dash visualization also.