GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → royshil → obs-backgroundremoval

royshil / obs-backgroundremoval

Licence: MIT license
An OBS plugin for removing background in portrait images (video), making it easy to replace the background when screen recording.

Programming Languages

C++
36643 projects - #6 most used programming language
shell
77523 projects
CMake
9771 projects
ruby
36898 projects - #4 most used programming language
Batchfile
5799 projects

Labels

mac-osxobsobs-pluginlibobs

Projects that are alternatives of or similar to obs-backgroundremoval

tally pi
TallyPi allows you to build network controlled tally lights for live cameras, intended for control by OBS but extensible enough to use for whatever purposes you like.
Stars: ✭ 35 (-97.32%)
Mutual labels:  obs
streamlabs-chat
Free customizable theme, template, custom coded CSS for Twitch, Streamlabs, Streamelemens chat box widget & overlay.
Stars: ✭ 59 (-95.48%)
Mutual labels:  obs
XION-ChaseCam
This is a free-to-use HTML/javascript based overlay for roleplay streamers. Basically it mimics the overlay of the AXON bodycam, but since most folks play in 3rd person, it's a ChaseCam. I've included a logo, and the html file. The html file has the css, html, and javascript all in one file for ease of editing. Goto line 81 of the html file to c…
Stars: ✭ 27 (-97.93%)
Mutual labels:  obs
obs-websocket-gd
Control OBS from inside a Godot game or app.
Stars: ✭ 42 (-96.78%)
Mutual labels:  obs
terraform-otc
Terraform integration modules for Open Telekom Cloud
Stars: ✭ 20 (-98.47%)
Mutual labels:  obs
obs-studio
This is a community-supported modified build of OBS Studio.
Stars: ✭ 86 (-93.4%)
Mutual labels:  obs
obs-service-go modules
OBS Source Service to download, verify, and vendor Go module dependency sources
Stars: ✭ 18 (-98.62%)
Mutual labels:  obs
BeatRecorder
Easily record your BeatSaber gameplay!
Stars: ✭ 20 (-98.47%)
Mutual labels:  obs
obs-golang-plugin
OBS Studio Golang Plugin
Stars: ✭ 50 (-96.17%)
Mutual labels:  obs
character-overlay
Web App for adding an OBS overlay with character information such as name, picture, and health for your favorite role-playing game.
Stars: ✭ 17 (-98.7%)
Mutual labels:  obs
OBS
No description or website provided.
Stars: ✭ 42 (-96.78%)
Mutual labels:  obs
gh-notifier
Desktop notifier for all your "social" GitHub notifications
Stars: ✭ 21 (-98.39%)
Mutual labels:  mac-osx
obs-text-slideshow
OBS plugin inspired by the built in image slideshow, except for text sources instead. Both Free Type 2 and GDI+ are supported.
Stars: ✭ 45 (-96.55%)
Mutual labels:  obs
OBSliveTally
Display the stream status or tally information for a scene or source on any device in the network.
Stars: ✭ 20 (-98.47%)
Mutual labels:  obs
awesome-script
常用的 shell 脚本,包括 centos, Ubuntu, Mac osx, 以及 Docker, Python 等常用软件的安装, 配置, 开发环境搭建的脚本.
Stars: ✭ 50 (-96.17%)
Mutual labels:  mac-osx
HEIF
Mac OS X: Convert any image to HEIF/HEIC format
Stars: ✭ 58 (-95.55%)
Mutual labels:  mac-osx
obs-websocket-js
Consumes https://github.com/obsproject/obs-websocket
Stars: ✭ 521 (-60.05%)
Mutual labels:  obs
file-upload
koa2 middleware support upload to cos/oss/obs/aws/local
Stars: ✭ 28 (-97.85%)
Mutual labels:  obs
obs blade
Make use of the OBS WebSocket Plugin (https://github.com/obsproject/obs-websocket) and control your stream
Stars: ✭ 182 (-86.04%)
Mutual labels:  obs
simple-obs-stt
Speech-to-text and keyboard input captions for OBS.
Stars: ✭ 89 (-93.17%)
Mutual labels:  obs
View All Similar Projects ➔

OBS Plugin: Background Removal

Download

Check out the latest releases for downloads and install instructions.

Introduction

This plugin is meant to make it easy to replace the background in portrait images and video. It is using a neural network to predict the mask of the portrait and remove the background pixels. It's easily composable with other OBS plugins to replace the background with e.g. an image or a transparent color.

The models used for background detection are SINet: https://arxiv.org/abs/1911.09099 and MODNet: https://arxiv.org/pdf/2011.11961.pdf The pre-trained model weights were taken from:

Some more information about how I built it: https://www.morethantechnical.com/2021/04/15/obs-plugin-for-portrait-background-removal-with-onnx-sinet-model/

Building

The plugin was built and tested on Mac OSX, Windows and Ubuntu Linux. Help is appreciated in building on other OSs and formalizing the one-click installers.

Prerequisites for building

Mac OSX

Install dependencies

You may use homebrew:

$ brew install opencv onnxruntime

Or - you may also build a (very minimal) version of OpenCV and ONNX Runtime for static-linking, instead of the homebrew ones:

<root>/build/ $ ../scripts/makeOpenCV_osx.sh
<root>/build/ $ ../scripts/makeOnnxruntime_osx.sh

Static linking should be more robust across versions of OSX, as well as building for 10.13.

Finding libobs

If you install the desktop OBS app (https://obsproject.com/download) you already have the binaries for libobs (e.g. /Applications/OBS.app/Contents/Frameworks/libobs.0.dylib) But you don't have the headers - so clone the main obs repo e.g. git clone --single-branch -b 27.2.4 [email protected]:obsproject/obs-studio.git (match the version number to your OBS install. Right now on OSX it's 27.2.4)

Build

$ mkdir -p build && cd build
$ cmake .. -DobsLibPath=/Applications/OBS.app/Contents/Frameworks -DobsIncludePath=~/Downloads/obs-studio/libobs
$ cmake --build . --target dist
$ cpack

Install

Unpack the package to the plugins directory of the system's Library folder (which is Apple's preferred way but requires admin privileges)

sudo mkdir -p "/Library/Application Support/obs-studio/plugins/obs-backgroundremoval"
sudo unzip -j obs-backgroundremoval-macosx.zip "Plugins/obs-backgroundremoval.so" -d "/Library/Application Support/obs-studio/plugins/obs-backgroundremoval/bin"
sudo unzip -j obs-backgroundremoval-macosx.zip "Resources/data/obs-plugins/obs-backgroundremoval/*" -d "/Library/Application Support/obs-studio/plugins/obs-backgroundremoval/data"

or directly to your OBS install directory, e.g.

$ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/

The first is recommended as it preserves the plugins over the parallel installation of OBS versions (i.e. running the latest productive version and a release candidate), or operations like brew upgrade obs, whereas the latter will also remove the plugin if you decide to delete the OBS application.

Linux

Ubuntu

$ sudo apt install -y libobs-dev libopencv-dev language-pack-en wget git build-essential cmake libsimde-dev
$ wget https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz
$ sudo tar xzvf onnxruntime-linux-x64-1.7.0.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"

Clone this repo into a directory of your choice, then build and install, e.g.:

$ cd obs-backgroundremoval
$ mkdir build && cd build
$ cmake .. && cmake --build . && sudo cmake --install .

To build with CUDA support, you will need the CUDA libraries installed as well as the GPU version of the ONNX Runtime library.

Use the instructions at https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu-installation to set up the NVIDIA Cuda repository, then:

These instructions use the 1.9.0 version of onnxruntime. If you wish to use a different version, change the export line to the version number you require.

$ export ONNX_VERSION=1.9.0
$ sudo apt install cuda libcudnn8
$ wget https://github.com/microsoft/onnxruntime/releases/download/v${ONNX_VERSION}/onnxruntime-linux-x64-gpu-${ONNX_VERSION}.tgz
$ sudo tar xzvf onnxruntime-linux-x64-gpu-${ONNX_VERSION}.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"

Then build and install:

$ sudo ldconfig  # required if you have previously had a different version of onnxruntime installed
$ mkdir build && cd build
$ cmake -DWITH_CUDA=ON .. && cmake --build . && sudo cmake --install .

If after installing, the plugin fails to load and cannot be found in the interface, there are additional steps to carry out.

$ mkdir -p ~/.config/obs-studio/plugins/obs-backgroundremoval/bin/64bit
$ ln -s /usr/local/lib/obs-plugins/obs-backgroundremoval.so ~/.config/obs-studio/plugins/obs-backgroundremoval/bin/64bit/
$ ln -s /usr/local/share/obs/obs-plugins/obs-backgroundremoval ~/.config/obs-studio/plugins/obs-backgroundremoval/data

If you wish to install this system-wide, change obs-studio path from ~/.config/obs-studio/plugins to /usr/share/obs/obs-plugins.

Archlinux

A PKGBUILD file is provided for making the plugin package

$ cd scripts
$ makepkg -s

Building for Arch in Docker (host OS e.g. MacOSX):

$ docker pull archlinux:latest
$ docker run -it -v $(pwd):/src archlinux:latest /bin/bash
# pacman -Sy --needed --noconfirm sudo fakeroot binutils gcc make
# useradd builduser -m
# passwd -d builduser
# printf 'builduser ALL=(ALL) ALL\n' | tee -a /etc/sudoers
# sudo -u builduser bash -c 'cd /src/scripts && makepkg -s'

Windows

We will use static linking (as much as possible) to aviod having to lug around .DLLs with the plugin.

Install Prerequisites

Install OpenCV via vcpkg:

$ mkdir build
$ cd build
$ git clone https://github.com/microsoft/vcpkg
$ cd vcpkg
$ .\bootstrap-vcpkg.bat
$ .\vcpkg.exe install opencv[core]:x64-windows-static

Install Onnxruntime with NuGet:

$ cd build
$ mkdir nuget
$ Invoke-WebRequest https://dist.nuget.org/win-x86-commandline/latest/nuget.exe -UseBasicParsing -OutFile nuget.exe
$ .\nuget.exe install Microsoft.ML.OnnxRuntime.DirectML -Version 1.11.0
$ .\nuget.exe install Microsoft.ML.OnnxRuntime.Gpu -Version 1.11.0
Build OBS Studio from source

Clone the OBS repo, $HOME\Downloads\ $ git clone --recursive [email protected]:obsproject/obs-studio.git, to e.g. Downloads. Use OBS scripts to build: $HOME\Downloads\obs-studio $ .\CI\build-windows.ps1

Find the complete instructions here: https://obsproject.com/wiki/build-instructions-for-windows

Build and install the plugin

$ cmake .. -Dlibobs_DIR="$HOME\Downloads\obs-studio\build64\libobs"
$ cmake --build . --config Release
$ cpack
$ Expand-Archive .\obs-backgroundremoval-0.4.0-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force

To build with CUDA support, tell cmake to use the CUDA version of OnnxRuntime

$ cmake .. -Dlibobs_DIR="$HOME\Downloads\obs-studio\build64\libobs" -DWITH_CUDA=ON

The rest of the build process is similar, but the result archive will be obs-backgroundremoval-win64-cuda.zip.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].