All Projects → sniklaus → Softmax Splatting

sniklaus / Softmax Splatting

an implementation of softmax splatting for differentiable forward warping using PyTorch

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Softmax Splatting

Pytorch Pwc
a reimplementation of PWC-Net in PyTorch that matches the official Caffe version
Stars: ✭ 402 (+84.4%)
Mutual labels:  cupy, cuda
Speedtorch
Library for faster pinned CPU <-> GPU transfer in Pytorch
Stars: ✭ 615 (+182.11%)
Mutual labels:  cupy, cuda
Pytorch Liteflownet
a reimplementation of LiteFlowNet in PyTorch that matches the official Caffe version
Stars: ✭ 281 (+28.9%)
Mutual labels:  cupy, cuda
revisiting-sepconv
an implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation using PyTorch
Stars: ✭ 43 (-80.28%)
Mutual labels:  cuda, cupy
Pynvvl
A Python wrapper of NVIDIA Video Loader (NVVL) with CuPy for fast video loading with Python
Stars: ✭ 95 (-56.42%)
Mutual labels:  cupy, cuda
Chainer
A flexible framework of neural networks for deep learning
Stars: ✭ 5,656 (+2494.5%)
Mutual labels:  cupy, cuda
Cupy
NumPy & SciPy for GPU
Stars: ✭ 5,625 (+2480.28%)
Mutual labels:  cupy, cuda
Pytorch Unflow
a reimplementation of UnFlow in PyTorch that matches the official TensorFlow version
Stars: ✭ 113 (-48.17%)
Mutual labels:  cupy, cuda
3d Ken Burns
an implementation of 3D Ken Burns Effect from a Single Image using PyTorch
Stars: ✭ 1,073 (+392.2%)
Mutual labels:  cupy, cuda
Sepconv Slomo
an implementation of Video Frame Interpolation via Adaptive Separable Convolution using PyTorch
Stars: ✭ 918 (+321.1%)
Mutual labels:  cupy, cuda
Adacof Pytorch
Official source code for our paper "AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation" (CVPR 2020)
Stars: ✭ 110 (-49.54%)
Mutual labels:  cupy, cuda
Spanet
Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset (CVPR'19)
Stars: ✭ 136 (-37.61%)
Mutual labels:  cupy, cuda
Ck Caffe
Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
Stars: ✭ 192 (-11.93%)
Mutual labels:  cuda
Amgx
Distributed multigrid linear solver library on GPU
Stars: ✭ 207 (-5.05%)
Mutual labels:  cuda
Pytorch Spynet
a reimplementation of Optical Flow Estimation using a Spatial Pyramid Network in PyTorch
Stars: ✭ 190 (-12.84%)
Mutual labels:  cuda
Macos Egpu Cuda Guide
Set up CUDA for machine learning (and gaming) on macOS using a NVIDIA eGPU
Stars: ✭ 187 (-14.22%)
Mutual labels:  cuda
Genomeworks
SDK for GPU accelerated genome assembly and analysis
Stars: ✭ 215 (-1.38%)
Mutual labels:  cuda
Cunn
Stars: ✭ 205 (-5.96%)
Mutual labels:  cuda
Nvidia Docker
Build and run Docker containers leveraging NVIDIA GPUs
Stars: ✭ 13,961 (+6304.13%)
Mutual labels:  cuda
Hybridizer Basic Samples
Examples of C# code compiled to GPU by hybridizer
Stars: ✭ 186 (-14.68%)
Mutual labels:  cuda

softmax-splatting

This is a reference implementation of the softmax splatting operator, which has been proposed in Softmax Splatting for Video Frame Interpolation [1], using PyTorch. Softmax splatting is a well-motivated approach for differentiable forward warping. It uses a translational invariant importance metric to disambiguate cases where multiple source pixels map to the same target pixel. Should you be making use of our work, please cite our paper [1].

Paper

setup

The softmax splatting is implemented in CUDA using CuPy, which is why CuPy is a required dependency. It can be installed using pip install cupy or alternatively using one of the provided binary packages as outlined in the CuPy repository.

The provided example script is using OpenCV to load and display images, as well as to read the provided optical flow file. An easy way to install OpenCV for Python is using the pip install opencv-contrib-python package.

usage

We provide a small script to replicate the third figure of our paper [1]. You can simply run python run.py to obtain the comparison between summation splatting, average splatting, linear splatting, and softmax splatting. Please see this exemplatory run.py for additional information on how to use the provided reference implementation of our proposed softmax splatting operator for differentiable forward warping.

xiph

In our paper, we propose to use 4K video clips from Xiph to evaluate video frame interpolation on high-resolution footage. Please see the supplementary benchmark.py on how to reproduce the shown metrics.

video

Video

license

The provided implementation is strictly for academic purposes only. Should you be interested in using our technology for any commercial use, please feel free to contact us.

references

[1]  @inproceedings{Niklaus_CVPR_2020,
         author = {Simon Niklaus and Feng Liu},
         title = {Softmax Splatting for Video Frame Interpolation},
         booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
         year = {2020}
     }

acknowledgment

The video above uses materials under a Creative Common license as detailed at the end.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].