Indi Allsky

indi-allsky is software used to manage a Linux-based All Sky Camera using the INDI framework. Theoretically, any INDI supported CCD/CMOS camera can be functional.

Pictured: SpaceX Cargo Dragon (over Georgia) headed for splashdown off the coast of Florida

Features

• Multiple camera vendor support
  • ZWO
  • Svbony
  • QHY
  • Altair
  • ToupTek
  • Starlight Xpress
  • Player One Astronomy
  • Raspberry Pi HQ Camera
  • libcamera support (imx477, imx378, etc)
  • Canon DSLRs
  • Generic web cameras
  • More to come
• Dark frames to remove hot pixels
• Timelapse video generation
• Network file transfers - Upload images and videos to remote site
• Publish data to an MQTT service for monitoring
• Keograms
• Star Trails
• Automatic meteor/plane/satellite detection
• Images display local hour angle of sun and moon
• Moon mode - reduced gain when the moon is overhead
• Remote INDI server operation - operate camera remotely over the network
• Pseudo-Sky Quality Meter - Use your all sky camera to measure sky brightness/quality
• SQLite database stores image metadata

Requirements

• A computer running a modern Linux distribution, such as a Raspberry Pi
  • Multicore is recommended
    • ARM
    • x86_64
  • 2GB RAM recommended, 1GB minimum
    • 512MB is adequate for image acquisition, but not enough to generate timelapse videos with ffmpeg
  • 64GB of storage minimum to store 2 months of videos and 30 days of JPEG images.
  • (Optional) Internet connectivity for image uploading
• An INDI supported camera
  • CPU architecture support varies between camera manufacturers
• NEW: A libcamera supported camera

Distibution support

MacOS support is theoretically possible, but not tested.

libcamera support

libcamera is a new camera interface designed to replace the legacy camera interfaces such as V4L2.

Proper libcamera support is only working on Raspbian 11 64-bit on Raspberry Pi 3 & 4.

Note: Genererating and processing raw (dng) files on a system with less than 1GB of memory and libcamera will cause out-of-memory conditions. There is an option to generate JPEG or PNG files with libcamera which solves this issue.

Installation

1. Install git

apt-get install git

2. Clone the indi-allsky git repository

git clone https://github.com/aaronwmorris/indi-allsky.git

3. Navigate to the indi-allky sub-directory

cd indi-allsky/

4. Run setup.sh to install the relevant software

./setup.sh

• Note: You may be prompted for a password for sudo

5. Start the software

systemctl --user start indiserver
systemctl --user start indi-allsky

6. Login to the indi-allsky web application https://raspberrypi.local/

• Note: The web server is configured with a self-signed certificate.

Manual operation

1. Stop indi-allsky service

systemctl --user stop indi-allsky

2. Activate the indi-allsky python virtual environment

source virtualenv/indi-allsky/bin/activate

3. Start indi-allsky

./allsky.py run

Logs

• When indi-allsky is run from the command line, logs are sent to STDERR by default.
• When the indi-allsky service is started, logs are sent to syslog via facility local6. Logs are stored in /var/log/indi-allsky/indi-allsky.log and rotated daily.

Dark frames

indi-allsky fully automates the capture and processing of master dark calibration frames. Currently, sigma clipping and average methods are supported.

https://github.com/aaronwmorris/indi-allsky/wiki/Dark-Calibration-Frames

Moon mode

This is a special night time operating mode intended to reduce gain when the moon is more illuminated and above the horizon

Keograms

Keograms are a visual representation of the entire timelapse video in a single frame. Every image is rotated so that the vertical aligns to the meridian and then the center-vertical column is extraced from each frame and compiled into the keogram. The rotation parameter in the config is KEOGRAM_ANGLE

https://github.com/aaronwmorris/indi-allsky/wiki/Keogram-Rotation

Below you can see perodic clouds passed over between 8-9pm and again between 4-5am. If you look closely enough, you can see the Pleiades star cluster and the Orion constellation as it passed through the meridian in this example keogram.

Note: The horizontal lines are just hot pixels that were subtracted by the dark frame.

Star Trails

Star trail images stack the stars from each frame to show their progression across the sky.

Star Trails Timelapse

Video of the star trails being stacked in real-time!

YouTube

Star Detection

indi-allsky utilizes OpenCV pattern matching to detect and count the number of stars in the view. Star counts are a good objective measurement of sky conditions.

Star and meteor detection support using detection masks to customize your Region of Interest if there are obstructions in your view. https://github.com/aaronwmorris/indi-allsky/wiki/Detection-Masks

Meteor Detection

Using OpenCV Canny edge detection and Hough Line Transform, indi-allsky is able to perform basic line detection to detect meteor and fireball trails in the view. Airplane and satellite trails are also detected using this method. Images are tagged with an asterisk in the image viewer if a trail has been detected.

Focus Mode

Focus mode is a special setting that generates images more often and implements a Variance of Laplacian scoring algorithm on the image to assist with focusing the camera. Images are not saved when focus mode is enabled.

Web Interface

The indi-allsky web interface is built on the Flask MVC framework. It is designed to be a dashboard for your sky. Included is the ability to fully manage the camera configuration without having to manually edit from the command line.

The default credentials for accessing the web interface are user admin and password secret

An unauthenticated public access URL is also available at https://raspberrypi.local/indi-allsky/public (actual hostname may vary)

Home Page

Charts

Early evening, the sun was still going down, but a cloud passed by, increasing the average brightness and lowering the star count.

A large cloud passed over significantly increasing the brightness of the sky and blocking out almost all of the stars.

Image viewer

Historical images browsing. Pictured: A small satellite flare.

Timelapse viewer

Historical Star trails and Keograms. The Keogram image is linked directly to the timelapse video for the night.

System Info

Database

All media generated are logged in a local SQLite database stored in /var/lib/indi-allsky/indi-allsky.sqlite This database is used as the source of images for timelapse and keogram generation, as well as, for displaying images via the web interfaces.

The database is managed via the python modules SQLAlchemy and alembic to provide migrations (schema upgrades) automatically in the setup.sh script.

Performance

indi-allsky itself is written in python, but python is just the glue between the different libraries, most of which are C based which makes indi-allsky extremely fast. A 1920 x 1080 image can be dark frame calibrated, debayered, histogram processed, text applied, and compressed to a JPG in less than 0.5 seconds on Raspberry Pi 3 class hardware. PNG processing is a little more taxing, but usually only takes a few seconds.

ffmpeg video processing is considerably more expensive. A 2 minute 1920x1080 h.264 encoded video compiled from 3,000 frames requires ~20 minutes on Raspberry Pi 3 (4 core) hardware. Encoding takes place in a separate process from image aqcuisition and image processing and is run at the lowest CPU priority so image acquision is not impacted.

Software Dependencies

Architecture

indi-allsky utilizes python's multiprocessing library to enable parallelizing tasks so that image processing does not interfere with image aquisition, etc.

Configuration

All configuration is read from /etc/indi-allsky/config.json . You can find configuration examples in the examples/ folder.

Tested Hardware

I have extensively tested the ZWO ASI290MM and the Svbony SV305. 3-4 weeks of steady runtime with no intervention are common. The only reason I restart my cameras are code updates (or power failures).

The hardware below has at least been plugged in and tested for correct detection and CFA decoding.

If you have an INDI supported camera from a vendor not listed, open an enhancement request and I can work with you to support the camera.

Gotchas

Common problems you might run into.

• The indi-allsky python processes consume ~500MB of RAM.
  • 1K (1920x1080) h.264 encoding with ffmpeg requires an additional ~500MB of RAM. 1GB of RAM should be the bare minimum system memory. You should also have 100-500MB of additional swap space to prevent running out of memory during encoding. 2GB of RAM recommended.
  • 4K (3840x2160) h.264 encoding requires an additional 2+GB of RAM. 4GB of RAM recommended.
  • Above 4K resolution requires greater than 4GB of RAM.
• In Raspbian 10 (legacy), the h.264 codec in ffmpeg has a maximum frame size of 4096×2304 (AVC level 5.1). If your camera generates higher resolution images, you will need to scale the frames or use the Region of Interest (RoI) options to reduce the frame size.
  • NEW: indi-allsky now has the ability to scale the native resolution images during the ffmpeg encoding phase, so you do not need to pre-scale your images.
  • The RaspberryPi HQ camera has a bin1 image size of 4056x3040. Setting IMAGE_SCALE to 75 in the config results in a image size of 3042x2280. Alternatively, you can center crop the image using IMAGE_CROP_ROI set to [0, 368, 4056, 2672] for an image size of 4056×2304.
• ffmpeg in Raspbian 11 enables AVC level 6.0+ which permits h.264 resolutions up to 8192×4320 (you must have sufficient system memory)
  • https://en.wikipedia.org/wiki/Advanced_Video_Coding

File Transfer

indi-allsky supports several file transfer methods. Additional file transfer methods are planned such as direct to YouTube uploads.

https://github.com/aaronwmorris/indi-allsky/wiki/File-transfers

MQTT Publishing

indi-allsky supports publishing all sky data to an MQTT service for monitoring.

For more info, see the wiki page: https://github.com/aaronwmorris/indi-allsky/wiki/MQTT-Broker-Publishing

Alternatives

• Thomas Jacquin's Allsky
  • Free, Open source
  • Linux, SoC
  • https://github.com/thomasjacquin/allsky
• AllSkEye
  • Free version, Commercial option
  • Windows
  • https://allskeye.com/
• RPi Meteor Station
  • Free, Open source
  • Linux, SoC
  • https://github.com/CroatianMeteorNetwork/RMS
• Meteotux Pi
  • Free version, Commercial option
  • Linux, SoC
  • https://www.meteotuxpi.com/
• UFOCapture
  • Shareware, Commercial option
  • Windows
  • https://sonotaco.com/soft/e_index.html

Commercial hardware

• Oculus All-Sky Cameras
  • https://www.sxccd.com/cameras/oculus-all-sky-cameras/
• Alcor System
  • https://www.alcor-system.com/new/AllSky/Alphea_camera.html

Acknowledgements

• Thomas Jacquin - indi-allsky is heavily inspired by his allsky software.
• Marco Gulino - His examples from indi-lite-tools were key to understanding how to work with pyindi-client
• PixInsight - Various algorithms were used that are in the PixInsight documentation