joehoeller / Object Detection On Thermal Images

Object Detection on Thermal Images

Robust Object Classification of Occluded Objects in Forward Looking Infrared (FLIR) Cameras using Ultralytics YOLOv3 and Dark Chocolate. Medium Article that compliments code repo: Article on Medium

Production Grade Results

1. mAP: 0.961
2. Recall: 0.922
3. F1: 0.857

Downloads needed to run codebase

1. Download pre-trained weights here: link

2. FLIR Thermal Images Dataset: Download

3. Go into /data folder and unzip labels.zip

4. Addt'l instructions on how to run Ultralytics Yolov3

Instructions

• Must have NVIDIA GPUs with Turing Architecture, Ubuntu and CUDA X installed if you want to reproduce results.

• Add the data provided by FLIR to a folder path called /coco/FLIR_Dataset.

• Place the custom pre-trained weights you downloaded from above into: /weights/*.pt

• Converted labels from Dark Chocolate are located in data/labels, which you unzipped above.

• The custom *.cfg with modified hyperparams is located in /cfg/yolov3-spp-r.cfg.

• Class names and custom data is in /data/custom.names and custom.data.

Install & Run Code:

After download is complete run pip install requirements, or click into the requriements.txt file for the Anaconda commands. Install COCO: bash yolov3/data/get_coco_dataset.sh, then add FLIR images to: /coco/images/FLIR_Dataset. Select any random grouping of non-annotated images, (ctrl-click any random sample of 5 to 10, or 20 if you like), copy them, and them paste them into data/samples folder.

• Go back to the root of the project where the requirements.txt file is and open a command prompt, run the following:

$ python3 detect.py --data data/custom.data --cfg cfg/yolov3-spp-r.cfg --weights weights/custom.pt

Modified config in yolov3-spp-r.cfg file; Leverages Spatial Pyramid Pooling with Ultralytics YOLOv3 for better feature extraction and higher precision on thermal images.

See Convolutional Neural Network Architecture below:

• At the root of the project, you will then see a folder named output get generated with annotated images and bounding boxes around the objects within the images you chose for the data/samples folder.

• To get metrics, go back into command line at root of project and run:

  $ python (python cmd prompt)

  $ from utils import utils

  $ utils.plot_results()

You will then see an image of charts get generated at root of project called results.png

• To get class-wise scores run ( * Note that -r /yolov3-spp-r.cfg is the altered CNN architecture):

$ python3 test.py --cfg cfg/yolov3-spp-r.cfg --weights weights/custom.pt --data data/custom.data

Need consulting to better understand computer vision implmentation for better business outcomes?

If Artifical Intelligence Applications are important to you or your business, please get in touch or email [email protected].

Consulting ideas on which this production-grade project could be forked to for real-word use cases:

• Search and Rescue for Public Safety: Object Classification in Thermal Images using Convolutional Neural Networks for Search and Rescue Missions (SAR) with Unmanned Aerial Systems (UAS).

• Defense and Aerospace: Detecting IEDs (Improvised Explosive Devices): in live combat war zones. Self-Driving Cars: Autonomous vehicles, personal or commercial.

• Industrial Applications: Electrical grid monitoring, wind power, and oil refinery monitoring.

• Improved Breast Cancer Screening & Detection: Automated analysis of tumor segmentation in thermal images using artificial intelligence increases the accuracy of detecting breast cancer, and enables use in breast cancer screening programs.

• Segmentation of Industrial Material types for automated assembly lines: Deep Thermal Imaging for material type recognition of Spatial Surface Temperature Patterns (SSTP).

• Sense and Detect Active School Shooters: Ensemble with Doppler signal processing methods for Concealed Weapon Detection in a Human Body by Infrared Imaging.

• NASA/ESA Land Rovers (e.g.; Mars Exploration Rovers (MER) Spirit and Opportunity): Fork GitHub repo and customize to measure heat signatures from various extraterrestrial objects. This will allow one to determine what materials/composites are in these objects. Also allows (martian) rover to further explore extra-terrestrial planets while identifying objects we cannot see in normal spectrum.