Visual Search

A visual search engine based on Elasticsearch and Tensorflow (now fully dockerized to run it in up-to-date development environments).

Description

This repository contains code in Python 2.7 and utilizes Faster-RCNN (with VGG-16 as backbone) implemented in Tensorflow 0.12.1 to extract features from images. An Elasticsearch instance is used to store feature vectors of the corresponding images, along with a plugin to compute distance between them.

TODO: Replace the outdated Faster-RCNN with a faster and more accurate model (suggestions or any collaboration is welcomed).

Requirements

The setup assumes you have a running installation of nvidia-docker and driver version 367.48 or above.

Setup Elasticsearch

First, we need to build the Elasticsearch plugin to compute distance between feature vectors. Make sure that you have Maven installed.

cd elasticsearch/es-plugin
mvn install

Next, we need to create a docker network so that all other containers can resolve the IP address of our elasticsearch instance.

docker network create vs_es_net

Finally, start the elasticsearch container. It will automatically add the plugin, create a named docker volume for persistent storage and connect the container to the network we just created:

cd ../ && docker-compose up -d

Index images

In order to populate the elasticsearch db with images, we need to first process them with a feature extractor (Faster-RCNN). The indexer services can do this for any image we place inside visual_search/images.

First we build a dockerized environment for the object detection model to run in:

cd visual_search && docker build --tag visual_search_env .

Here we use an earlier version implemented by @Endernewton. To get pre-trained model, you can visit release section, download and extract file model.tar.gz to visual_search/models/ folder. Optionally, you can run:

mkdir models && cd models
curl https://github.com/tuan3w/visual_search/releases/download/v0.0.1/model.tar.gz
tar -xvf model.tar.gz

To index the desired images, copy the corresponding compose file to the proper directory and start the indexing service:

cd ../ && cp indexer/docker-compose.yml .
docker-compose up

Start server

Before starting the server, again copy the corresponding compose file (overwrite the one used for indexing data) into the proper directory and start the containerized flask server:

cp server/docker-compose.yml .
docker-compose up -d

Now, you can access the link http://localhost:5000/static/index.html to test the search engine.

LICENSE

MIT