Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.
How does it work
- An energy map (edge detection) is generated from the provided image.
- The algorithm tries to find the least important parts of the image taking into account the lowest energy values.
- Using a dynamic programming approach the algorithm will generate individual seams across the image from top to down, or from left to right (depending on the horizontal or vertical resizing) and will allocate for each seam a custom value, the least important pixels having the lowest energy cost and the most important ones having the highest cost.
- We traverse the image from the second row to the last row and compute the cumulative minimum energy for all possible connected seams for each entry.
- The minimum energy level is calculated by summing up the current pixel value with the lowest value of the neighboring pixels obtained from the previous row.
- We traverse the image from top to bottom and compute the minimum energy level. For each pixel in a row we compute the energy of the current pixel plus the energy of one of the three possible pixels above it.
- Find the lowest cost seam from the energy matrix starting from the last row and remove it.
- Repeat the process.
The process illustrated:
Original image | Energy map | Seams applied |
---|---|---|
Features
Key features which differentiates this library from the other existing open source solutions:
- GUI progress indicator
- Customizable command line support
- Support for both shrinking or enlarging the image
- Resize image both vertically and horizontally
- Face detection to avoid face deformation
- Support for multiple output image type (jpg, jpeg, png, bmp, gif)
- Support for
stdin
andstdout
pipe commands - Can process whole directories recursively and concurrently
- Use of sobel threshold for fine tuning
- Use of blur filter for increased edge detection
- Support for squaring the image with a single command
- Support for proportional scaling
Face detection
The library is capable of detecting human faces prior resizing the images by using the lightweight Pigo (https://github.com/esimov/pigo) face detection library. The library does not require to already have OpenCV installed.
The image below illustrates the application capabilities for human face detection prior resizing. It's clearly visible from the image that with face detection activated the algorithm will avoid cropping pixels inside the detected faces, retaining the face zone unaltered.
Original image | With face detection | Without face detection |
---|---|---|
Install
First, install Go, set your GOPATH
, and make sure $GOPATH/bin
is on your PATH
.
$ export GOPATH="$HOME/go"
$ export PATH="$PATH:$GOPATH/bin"
Next download the project and build the binary file.
$ go get -u -f github.com/esimov/caire/cmd/caire
$ go install
MacOS (Brew) install
The library can also be installed via Homebrew.
$ brew install caire
Usage
$ caire -in input.jpg -out output.jpg
Supported commands:
$ caire --help
The following flags are supported:
Flag | Default | Description |
---|---|---|
in |
- | Input file |
out |
- | Output file |
width |
n/a | New width |
height |
n/a | New height |
preview |
true | Show GUI window |
perc |
false | Reduce image by percentage |
square |
false | Reduce image to square dimensions |
blur |
1 | Blur radius |
sobel |
10 | Sobel filter threshold |
debug |
false | Use debugger |
face |
false | Use face detection |
angle |
float | Plane rotated faces angle |
GUI progress indicator
A GUI preview window is also supported for showing the resizing process. For the GUI part I've opted of using the Gio library for its robustness and modern architecture. But in order to use it you have to install all of its dependencies. So please check the installation section here: https://gioui.org/#installation.
The preview window is activated by default but you can disable it with by setting the -preview
flag to false. When the images are processed concurrently from a directory the preview mode is disabled.
Face detection to avoid face deformation
In order to detect faces prior rescaling use the -face
flag. There is no need to provide a face classification cascade file, since it's already embedded into the generated binary file. The sample code below will rescale the provided image with 20%, but will run the face detection prior rescaling in order tot avoid face deformations.
For face detection related settings please check the Pigo documentation.
$ caire -in input.jpg -out output.jpg -face=1 -perc=1 -width=20
stdin
and stdout
pipe commands
Support for You can also use stdin
and stdout
with -
:
$ cat input/source.jpg | caire -in - -out - >out.jpg
in
and out
default to -
so you can also use:
$ cat input/source.jpg | caire >out.jpg
$ caire -out out.jpg < input/source.jpg
You can provide also an image URL for the -in
flag or even use cURL as a pipe command in which case there is no need to use the -in
flag.
$ caire -in <image_url> -out <output-folder>
$ curl -s <image_url> | caire > out.jpg
Process multiple images from a directory concurrently
The library can also process multiple images from a directory concurrently. You only need to provide the source and the destination folder by using the -in
and -out
flags.
$ caire -in <input_folder> -out <output-folder>
Support for multiple output image type
There is no need to define the output file type. The library is capable of detecting the output image type directly by the file extension and encodes the image to that specific type. You can export the resized image even to a Gif file, in which case the generated file shows the resizing process interactively.
Other options
In case you wish to scale down the image by a specific percentage, it can be used the -perc
boolean flag. In this case the values provided for the width
and height
are expressed in percentage and not pixel values. For example to reduce the image dimension by 20% both horizontally and vertically you can use the following command:
$ caire -in input/source.jpg -out ./out.jpg -perc=1 -width=20 -height=20 -debug=false
Also the library supports the -square
option. When this option is used the image will be resized to a square, based on the shortest edge.
When an image is resized on both the X and Y axis, the algorithm first try to rescale it prior resizing, but also preserves the image aspect ratio. Afterwards the seam carving algorithm is applied only to the remaining points. Ex. : given an image of dimensions 2048x1536 if we want to resize to the 1024x500, the tool first rescale the image to 1024x768 and then will remove only the remaining 268px.
Caire integrations
- Caire can be used as a serverless function via OpenFaaS: https://github.com/esimov/caire-openfaas
- Caire can also be used as a
snap
function (https://snapcraft.io/caire):$ snap run caire --h
Results
Shrunk images
Original | Shrunk |
---|---|
Enlarged images
Original | Extended |
---|---|
Useful resources
- https://en.wikipedia.org/wiki/Seam_carving
- https://inst.eecs.berkeley.edu/~cs194-26/fa16/hw/proj4-seamcarving/imret.pdf
- http://pages.cs.wisc.edu/~moayad/cs766/download_files/alnammi_cs_766_final_report.pdf
- https://stacks.stanford.edu/file/druid:my512gb2187/Zargham_Nassirpour_Content_aware_image_resizing.pdf
Author
- Endre Simo (@simo_endre)
License
Copyright © 2018 Endre Simo
This project is under the MIT License. See the LICENSE file for the full license text.