awesome-image-alignment-and-stitching

A curated list of awesome resources for image alignment and stitching, etc. The list will be updated continuously.

Table of Content

[TOC]

Tutorials

• Repo: Image registration

  A resource list about image registration related to natural/remote sensing/medical image and point cloud.

• MICCAI2019 Tutorial: Learn2Reg: Tutorial on Deep Learning in Medical Image Registration [Homepage]

  Three hands-on sessions guiding participants to understand and implement published algorithms using clinical imaging data, including unsupervised registration, label-supervised registration, and discrete deep learning registration.

• A Practical Review on Medical Image Registration: from Rigid to Deep Learning based Approaches [PDF&Slides]

  A tutorial for anyone who wants to learn Medical Image Registration, by Natan Andrade, Fabio Augusto Faria, Fábio Augusto Menocci Cappabianco, SIBGRAPI2018

• [Blog] Image Registration: From SIFT to Deep Learning [Homepage] [Notes/Chinese]

  It presents OpenCV feature-based methods before diving into Deep Learning, including example codes.

• [Book] Image Descriptor: Modern Approaches, Bin Fan, Zhenhua Wang, and Fuchao Wang. Springer, 2015.

• [Report] 刘帅成：图像对齐技术及其应用 [Video]

Review

Matching/Registration/Alignment

• [2017CVPR] HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors [Homepage]

Stitching/Mosaicking

• [2019GRSM] Remote Sensing Image Mosaicking: Achievements and Challenges
• [2019IVC] Image mosaicing: A deeper insight
• [ 2006Szeliski] Image Alignment and Stitching: A Tutorial

Remote Sensing

• [2018ISPRSJ] A deep learning framework for remote sensing image registration

Medical Registration

• Deep Learning in Medical Image Registration: A Survey, 2019
• Learning image-based spatial transformations via convolutional neural networks: A review, Magnetic Resonance Imaging, December 2019
• Deformable Medical Image Registration: A Survey, TMI2013

Geometric Pre-processing

• [2019TCSVT] DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time
• [2019CVPR] Learning to Calibrate Straight Lines for Fisheye Image Rectification
• [2019CVPR] Deep Single Image Camera Calibration with Radial Distortion [Proj]
• [2015CVPR] Line-Based Multi-Label Energy Optimization for Fisheye Image Rectification and Calibration [Proj]
• [2019IJCV] Superpixel-Guided Two-View Deterministic Geometric Model Fitting [Guobao Xiao]

Image Matching

Feature Detection

- Key Point

• GLAMpoints: Greedily Learned Accurate Match points, arXiv2019.8

• [2018NeurIPS] LF-Net: Learning Local Features from Images [github]

• [2017CVPR] Comparative Evaluation of Hand-Crafted and Learned Local Features

• [2016ECCV] LIFT: Learned Invariant Feature Transform [Theano] [TF] [Eduard Trulls]

- Line Detection

• Line as object: datasets and framework for semantic line segment detection, arXiv2019.9

• [2020PAMI] Learning Regional Attraction for Line Segment Detection

• [2019CVPR] Learning Attraction Field Representation for Robust Line Segment Detection [github]

• [2019CVPR] PPGNet: Learning Point-Pair Graph for Line Segment Detection [github]

• [2018CVPR] [Wireframe] Learning to Parse Wireframes in Images of Man-Made Environments [github]

• [2017ICCV] Semantic Line Detection and Its Applications

• [2017PAMI] A Novel Linelet-based Representation for Line Segment Detection [Code]

• [2017CVPR] MCMLSD: A Dynamic Programming Approach to Line Segment Detection [Code]

• Edge Drawing/EDLines/LBD Descriptors/LSD [Code] [Code2]

- Edge Detection

• Semantic Edge Detection with Diverse Deep Supervision, arXiv2018.12
• [2017CVPR] CASENet: Deep Category-Aware Semantic Edge Detection

- Ellipse detection

• Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection, arXiv2019.9

• [2018TIP] Arc-support Line Segments Revisited: An Efficient and High-quality Ellipse Detection [github]

• [2017IJCV] Holistically-Nested Edge Detection

• [2017TIP] A Fast Ellipse Detector Using Projective Invariant Pruning [Code]

• [2012ECCV] [ELSD] A parameterless line segment and elliptical arc detector with enhanced ellipse fitting [Homepage&Code]

Feature Description

• Single and Cross-Dimensional Feature Detection and Description: An Evaluation, arXiv2019.10

• [2020AAAI] LCD: Learned Cross-domain Descriptors for 2D-3D Matching

• [2019ICCV] Beyond Cartesian Representations for Local Descriptors

• [2016CVPR] Accumulated Stability Voting: A Robust Descriptor from Descriptors of Multiple Scales [Proj] [Code]

• [2015ICCV] Discriminative learning of deep convolutional feature point descriptors

Image Matching

- Feature Matching

• GLMNet: Graph Learning-Matching Networks for Feature Matching, arXiv2019.11

• SuperGlue: Learning Feature Matching with Graph Neural Networks, arXiv2019.11

• [2019IJCV] Locality Preserving Matching

• [2019TIP] Robust Feature Matching Using Spatial Clustering With Heavy Outliers

• [2019CVPR] RF-Net: An End-to-End Image Matching Network based on Receptive Field [Code]

• [2019ICCV] Learning Two-View Correspondences and Geometry Using Order-Aware Network [Codes]

• [2018PAMI] CODE: Coherence based Decision Boundaries for Feature Matching [Code]

• [2018PAMI] Best-buddies similarity-robust template matching using mutual nearest neighbors

  [2015CVPR] Best-Buddies Similarity for Robust Template Matching [Proj&Code]

• [2018CVIU] Deep Spectral Correspondence for Matching Disparate Image Pairs

• [2017MM] Deep Matching and Validation Network: An End-to-End Solution to Constrained Image Splicing Localization and Detection [Codes]

• [2017ICMR] Panorama to panorama matching for location recognition

• [2016IJCV] DeepMatching: Hierarchical Deformable Dense Matching [Code]

• [2016BMVC] Fully-Trainable Deep Matching

• [2016NeurIPS] Universal Correspondence Network

• [2015CVPR] MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching

• [2013CVPR] Joint Spectral Correspondence for Disparate Image Matching [Code]

• [2013HPCS] Gpu-asift: A fast fully affine-invariant feature extraction algorithm [Homepage] [Code]

• [2012CVPR] Aligning Images in the Wild [Homepage]

• [2012CVPR] Image Matching using Local Symmetry Features [Proj&Code]

- Line Matching

• [2012PR] [LPI] Robust Line Matching through Line-point Invariants, [Code]

  [2010CVPR] Line Matching Leveraged By Point Correspondences [Proj]

- OutlierRemoval

• [2019ICCV] Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses [github] [Proj]

• [2019CVPR] MAGSAC: marginalizing sample consensus (no in-outlier threshold) [Code]

• [2018CVPR] Latent RANSAC [Code]

• [2017CVPR] GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence [Code] [Talk]

- Cross-View Matching

• [2019ICCV] Bridging the Domain Gap for Ground-to-Aerial Image Matching [Code]
• [2019ICCV] Ground-to-Aerial Image Geo-Localization With a Hard Exemplar Reweighting Triplet Loss
• [2018CVPR] CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization
• [2017CVPR] Cross-View Image Matching for Geo-Localization in Urban Environments
• [2016CVPR] Learning to Match Aerial Images With Deep Attentive Architectures
• [2016CVPR] Regularity-Driven Facade Matching Between Aerial and Street Views
• [2015ICCV] Wide-Area Image Geolocalization With Aerial Reference Imagery
• [2015CVPR] Learning Deep Representations for Ground-to-Aerial Geolocalization
• [2013CVPR] Cross-View Image Geolocalization

Image Registration

• Report: Deep Learning: Image Registration, Steven Chen and Ty Nguyen, 2017 [Slides]

Homography

• Homography-Estimation-List_pwc

• Content-Aware Unsupervised Deep Homography Estimation, arXiv2019.9 [Code]

• [2019Acce] Combining Convolutional Neural Network and Photometric Refinement for Accurate Homography Estimation

• [2019ICCV] Homography from two orientation- and scale-covariant features [Code]

• [2019AppSci] STN-Homography: Direct Estimation of Homography Parameters for Image Pairs, AppliedScience

• [2018ACCV] Rethinking Planar Homography Estimation Using Perspective Fields [Code]

• [2018RAL] Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model, IEEE RAL [Code]

• [2018ECCV] GridFace: Face Rectification via Learning Local Homography Transformations

• [2017CVPR] CLKN: Cascaded Lucas-Kanade Networks for Image Alignment

• [2017ICCVW] Homography Estimation from Image Pairs with Hierarchical Convolutional Networks [Code]

• [2016RSSW] [HomographyNet] Deep Image Homography Estimation, RSSW2016 [Note] [Note2] [unofficial1-tf] [unofficial2-tf] [unofficial3] [unofficial4-pytorch] [unofficial5-keras] [unofficial6-keras]

• Auxiliary

• Convolutional Neural Network Optimization for Homography Estimation, thesis2018

• Homography Estimation using Deep Learning for Registering All-22 Football Video Frames, TechReport2017 [PDF]

Registration

- Natural Scene

• [2017PAMI] Transformations Based on Continuous Piecewise-Affine Velocity Fields

  [2015ICCV] Highly-Expressive Spaces of Well-Behaved Transformations: Keeping It Simple [Code-Python+CUDA] [Code-CPU+Julia] [Proj]

• [2015CVPR] Robust Image Alignment with Multiple Feature Descriptors and Matching-Guided Neighborhoods [Proj/Code]

• [2014ECCV] Multi-modal and Multi-spectral Registration for Natural Images [Proj]

- Remote Sensing

• Coarse to fine non-rigid registration: a chain of scale-specific neural networks for multimodal image alignment with application to remote sensing, arXiv2018.2

• [2018ECCV] Multimodal Image Alignment Through a Multiscale Chain of Neural Networks with Application to Remote Sensing [Homepage] [Code]

• [2018Acce] Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features [github]

• CVPR 2019

• Metric Learning for Image Registration [github]

• Networks for Joint Affine and Non-Parametric Image Registration

• Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation

• SDRSAC: Semi-definite-Based Randomized Approach for Robust Point Cloud Registration Without Correspondences [Codes]

• 3D Local Features for Direct Pairwise Registration

• PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet [Codes]

• FilterReg: Robust and Efficient Probabilistic Point-Set Registration Using Gaussian Filter and Twist Parameterization [Codes]

• NeuIPS2019

• Arbicon-Net: Arbitrary Continuous Geometric Transformation Networks for Image Registration, NeurIPS2019
  Jianchun Chen (New York University), Lingjing Wang (New York University), Xiang Li (New York University), Yi Fang (New York University)

• Recurrent Registration Neural Networks for Deformable Image Registration, NeuIPS2019
  Robin Sandkühler, Simon Andermatt, ...

• PRNet: Self-Supervised Learning for Partial-to-Partial Registration, NeuIPS2019
  Yue Wang (MIT) · Justin M Solomon (MIT)

• Region-specific Diffeomorphic Metric Mapping, NeurIPS 2019

• Automatic and Robust Skull Registration Based on Discrete Uniformization, ICCV2019 Junli Zhao, Xin Qi, Chengfeng Wen, Na Lei, Xianfeng Gu

• Linearly Converging Quasi Branch and Bound Algorithms for Global Rigid Registration, ICCV2019 Nadav Dym, Shahar Ziv Kovalsky

• Deep Closest Point: Learning Representations for Point Cloud Registration, ICCV2019 Yue Wang, Justin M. Solomon

• A Deep Step Pattern Representation for Multimodal Retinal Image Registration, ICCV2019 Jimmy Addison Lee, Peng Liu, Jun Cheng, Huazhu Fu

• Efficient and Robust Registration on the 3D Special Euclidean Group, ICCV2019 Uttaran Bhattacharya, Venu Madhav Govindu

• Robust Variational Bayesian Point Set Registration, ICCV2019 Jie Zhou, Xinke Ma, Li Liang, Yang Yang, Shijin Xu, Yuhe Liu, Sim-Heng Ong

• Recursive Cascaded Networks for Unsupervised Medical Image Registration, ICCV2019 [github] Shengyu Zhao, Yue Dong, Eric I-Chao Chang, Yan Xu

- Medical Registration

• MICCAI2019

• Image-and-Spatial Transformer Networks for Structure-Guided Image Registration,

• Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration

• A Deep Learning Approach to MR-less Spatial Normalization for Tau PET Images

• TopAwaRe: Topology-Aware Registration

• Multimodal Data Registration for Brain Structural Association Networks

• Dual-Stream Pyramid Registration Network

• A Cooperative Autoencoder for Population-Based Regularization of CNN Image Registration

• Conditional Segmentation in Lieu of Image Registration

• DeepAtlas: Joint Semi-supervised Learning of Image Registration and Segmentation

• Linear Time Invariant Model Based Motion Correction (LiMo-MoCo) of Dynamic Radial Contrast Enhanced MRI

• Incompressible Image Registration Using Divergence-Conforming B-Splines

• Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks [Codes]

- Point-cloud Registration

• [2019ICCV] DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration
• CMRNet: Camera to LiDAR-Map Registration, IEEE ITSC2019

Image Stitching

CNN Stitching

• [2020] Attentive Deep Stitching and Quality Assessment for 360° Omnidirectional Images, JSTSP
• Secondary
• [2019] StitchNet: Image Stitching using Autoencoders and Deep Convolutional Neural Networks, thesis
• [2018] Learning-based Natural Geometric Matching with Homography Prior, ElectronicsLetter
• [2016] Image stitching with single-hidden layer feedforward neural networks, IJCNN

Spatially-Varying Warps

• [2020TIP] Local-Adaptive Image Alignment Based on Triangular Facet Approximation
• [2020JINS] Shape-optimizing mesh warping method for stereoscopic panorama stitching
• [2019TVCG] Content-Preserving Image Stitching with Regular Boundary Constraints
• [2019CGI] Stereoscopic Image Stitching with Rectangular Boundaries
• [2019TMM] A Novel Projective-Consistent Plane based Image Stitching Method
• [2018TMM] Parallax-Tolerant Image Stitching Based on Robust Elastic Warping [Code]
• [2018TMM] Quasi-Homography Warps in Image Stitching
• [2018ECCV] Image Stitching with Multiple Registrations [Homepage]
• [2018ECCV] Object-Centered Image Stitching [Homepage]
• [2018ECCVW] GreenWarps: A Two-Stage Warping Model for Stitching Images using Diffeomorphic Meshes and Green Coordinates
• [2018Acce] An Accurate Multi-Row Panorama Generation Using Multi-Point Joint Stitching
• [2018Acce] Generalized Content-Preserving Warps for Image Stitching
• [2018Conf] Large Parallax Image Stitching Using an Edge-Preserving Diffeomorphic Warping Process
• [2017] Robust Multi-homography Method for Image Stitching under Large Viewpoint Changes, conf
• [2014TMM] Image Alignment by Piecewise Planar Region Matching

Seam-based Stitching

• [2016ECCV] SEAGULL: Seam-guided Local Alignment for Parallax-tolerant Image Stitching [Note]

• [2014CVPR] Parallax-tolerant Image Stitching [Proj/Data]

• [2013EG] Seam-Driven Image Stitching

Dsitortion Preserving

• [2019TIP] Single-Perspective Warps in Natural Image Stitching [Code] [Code-multi]

• [2016ECCV] Natural Image Stitching with the Global Similarity Prior [Proj] [MacOS] [CCC] [VS]

• [2005TOG] As-Rigid-As-Possible Shape Manipulation; [JoG2009] Implementing As-Rigid-As-Possible Shape Manipulation and Surface Flattening [Proj]

Warping/Projection

• [2009CGF] A Shape-Preserving Approach to Image Resizing [Proj/Code]

• [2005ICCV] Squaring the Circle in Panoramas

• [2005TOG] As-rigid-as-possible shape manipulation [Proj]

Stitching/Panoramas

• [2016TOG] Jump: Virtual reality video

• [2016ACCV] Pano2Vid: Automatic Cinematography for Watching 360° Videos [Proj] [Code]

• [2014ECCV] PanoContext: A Whole-room 3D Context Model for Panoramic Scene Understanding, oral [Proj/Code]

• [2014ECCV] Photo Uncrop [Proj]

• [2010ECCV] Real-Time Spherical Mosaicing Using Whole Image Alignment

• [2010CVPR] Generating Sharp Panoramas from Motion-blurred Videos [Proj]

• [2008MM] Discovering panoramas in web videos [Proj]

• [2007EG] Scene Collages and Flexible Camera Arrays [Proj] [Sofeware]

• [2007CVPR] Layered Depth Panoramas [Proj]

• [2007Project] Automating Joiners [Proj] [Code] [PDF]

• [2006TOG] Photographing long scenes with multi-viewpoint panoramas [Proj]

Stitching Application

• [2016] Introducing facebook surround 360: An open, high-quality 3d-360 video capture system [Page]

• [2012CVPR] Recognizing Scene Viewpoint using Panoramic Place Representation [Proj/Code]

Stereo Stitching

• [2015CVPR] Casual Stereoscopic Panorama Stitching [Proj/Data]

Image Composition

Seamline

• Coarse-to-fine Seam Estimation for Image Stitching, arXiv2018.5

• [2019SIVP] Quality evaluation-based iterative seam estimation for image stitching [Code]

• [2018SIVP] Perception-based seam cutting for image stitching [Code]

• [2017IPOL] Efros and Freeman Image Quilting Algorithm for Texture Synthesis [2001Dynamic Seamline] [Code]

• [2012TOG] Panorama weaving: fast and flexible seam processing [Proj]

Blending

- Gradient-based

• [2019MM] GP-GAN: Towards Realistic High-Resolution Image Blending [Proj] [github] ⭐️
• [2012TOG] Image Melding: Combining Inconsistent Images using Patch-based Synthesis [Proj/Code]
• [2010ECCV] Error-tolerant Image Compositing [Code]
• [2007TOG] Efficient gradient-domain compositing using quadtrees [in PS CS3] [Proj]
• [2004SIGG] Interactive Digital Photomontage [Proj&Code]
• [2004ECCV] Seamless image stitching in the gradient domain
• [2003SIGG] Poisson image editing

Dodging

• [2019ISPRSJ] A Closed-Form Solution for Multi-View Color Correction with Gradient Preservation [Code]

  [2017ICCVW] Color Consistency Correction Based on Remapping Optimization for Image Stitching

• [2018TIP] Dissecting and Reassembling Color Correction Algorithms for Image Stitching [Code&Supp]

• [2016CVPR] Efficient and Robust Color Consistency for Community Photo Collections [Homepage] [Code]

• [2011ICCV] Revisiting radiometric calibration for color computer vision

Stitching Assessment

• [2019MM] Cross-Reference Stitching Quality Assessment for 360° Omnidirectional Images [Slides] [Dataset]

• [2017ICCVW] A content-aware metric for stitched panoramic image quality assessment

• Secondary

• Quality Evaluation for Stitched Panoramic Videos, Thesis2018.11

Video Stitching

• [2019BMVC] [CNNStitch] Video Stitching for Linear Camera Arrays [CNN-based] [Proj] ⭐️

• [2018TIP] Dynamic Video Stitching via Shakiness Removing [github]

• [2017CGF] High-resolution 360 video foveated stitching for realtime VR

• [2016TIP] Joint video stitching and stabilization from moving cameras [Proj] [Shuaicheng Liu]

• [2016TOG] Rich360: Optimized spherical representation from structured panoramic camera arrays

• [2016CGF] Seamless video stitching from hand-held camera inputs [Proj]

• [2015CVPRW] Video Stitching with Spatial-Temporal Content-Preserving Warping

• [2015EG] Panoramic Video from Unstructured Camera Arrays [Homepage]

• [2015TOC] Efficient video stitching based on fast structure deformation

• Secondary

• [2018] Robust spatial–temporal bayesian view synthesis for video stitching with occlusion handling, MVA

Video Stabilization

• StableNet: Semi-Online, Multi-Scale Deep Video Stabilization, arXiv2019.7

• [2019SIGGAsia] Deep Iterative Frame Interpolation for Full-frame Video Stabilization

• [2019TIP] Deep Online Video Stabilization With Multi-Grid Warping Transformation Learning [Code] [DebugNote] [Note] Deep Online Video Stabilization, arXiv2018

• [2019CVPR] Robust Video Stabilization by Optimization in CNN Weight Space [Video]

• [2018TOG] Joint Stabilization and Direction of 360◦ Videos

• [2018CGF] Deep Video Stabilization Using Adversarial Networks [Note]

• [2018] Deep Learning in Video Stabilization Homography Estimation, ConfNNA2018

• [2015TIP] Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping

• [2013CVPR] Plane-Based Content Preserving Warps for Video Stabilization

• [2013TOG] Bundled camera Paths for Video Stabilization [Proj/Code] [As-similar-as-possible warping]

• [2013TVCG] Spatially and Temporally Optimized Video Stabilization [Proj]

• [2009TOG] Content-Preserving Warps for 3D Video Stabilization [Proj/Data]

Libs

• SmartBlend

  It is an application for seamless image blending by Michael Norel. Smartblend allows stitching of many problematic shots (parallaxed, with moving objects or with differing exposure).

• GCO3.0

  Matlab code for Max-flow/min-cut

Stitching Docs/Examples

Blogs

• 环形影像拼接的视差问题, 2011

• Real-time panorama and image stitching with OpenCV, 2016

Examples

• Panorama stitching Project (Python) [Code] ⭐️

• OpenPano: Automatic Panorama Stitching From Scratch [Code] ⭐️⭐️

• Image-Alignment-and-Panoramas [Code]