Veue
Background
This is a fully featured rails application, built for a startup called Veue that folded in July of 2021. The goal of the startup was to build a low-latency, realtime interactive streaming service. "Twitch for everyone!"
While the service was live, we hosted over 100 live shows, with 11,000 chat messages, and 1,100 HOURS of audiences watching shows.
1,760 commits by primarily 3 main developers into this Rails application over ~10 months: @hcatlin, @ParamagicDev, and @Sirbuland
As we are shutting down the company, we decided that since we weren't going to sell off the assets that we may as well just share what we built with the world– as long as you know how to read Ruby.
There aren't many "mature" Rails applications out there by experienced dev teams that people can look at and so we thought we'd share what we had!
As the project is, it requires access to the Phenix Live Streaming platform, which at release requires an agreement with them. In this version, you can stream with low-latency right from a modern browser.
Previous, there was code that used MUX + an Electron app called the Broadcaster to send an RTMP feed to Mux, which has self-service sign up. Ideally, if this were to be used by other people, we should go with a service that allows quick signup and credit card payments, but for now we're leaving the main branch as-is.
It's maybe-but-probably not worth it to run this yourself, but who knows.... better to put it out there than have it just blink out of existence!
Also, don't judge us too hard! We did our best to code a great product as quickly as we could!
Introduction
This is the primary Rails application for Veue. It's designed to be as standard of a Rails app as is possible, using the best practices of the Rails community and the included tooling around the ecosystem.
We use the following technologies:
- Postgres
- Haml
- Sass CSS
- Webpacker
- Stimulus.js
- Typescript
We use the following Third-Party services:
- Phenix (Video Streaming platform)
- Render (Hosting)
- AppSignal (APM)
- LogDNA (Logging)
- Google Perpsective (Content Filtering)
- IFTTT (Triggered Events)
- Twilio (SMS)
- AWS S3 (Image Storage)
Developer Setup
1. Setup Phenix Account
In your local copy of this repo, generate a file config/application.yml
with values that another team member
can share with you.
# config/application.yml
PHENIX_APP_ID: ?????????
PHENIX_APP_SECRET: ?????
This file will be ignored by git... and leave it that way. Take this secret to your graaaaavvvveeeeeeeee.
2. Postgres
For Postgres installation, visit this page: https://www.postgresql.org/download/
3. Ruby, Rails & Database Setup
First, lets pull in our dependencies:
bundle install && yarn install --check-files
Great, now let's setup the app & database. Run the following in your terminal:
rake db:setup
This will create the database, migrate the database, and seed the database for you with a couple users. However, these users aren't streamers yet... we need to set them up to be streamers!
Run the Rails server (from root of the project in a new shell)
rails s
4. Webhook With Ngrok Setup
This tool runs a magic TSL tunnel back to your machine as long as you keep it open in shell.
First download the executable from https://ngrok.com/download
You can test it from the examples on their download page. It creates a tunnel at a public URL back to an app running on your machine, only for the specified port. (Exit with Ctrl-C and the tunnel will be broken.)
To install, use brew install --cask ngrok
or yarn global add ngrok
depending on your preferences.
In a web browser, go to https://dashboard.ngrok.com/ and login with your Veue Google Account. This will give you a free pro ngrok account. Follow Step 2 which sets up your auth token.
To run ngrok:
$> ngrok http 3000
Look for two "Forwarding" URLs with random subdomain strings for .ngrok.io (The second one will be https)
This will open a public endpoint that we can use for our webhook. Do not close this tab! Copy the https url in there.
It will look something like https://username.ngrok.io
In your config/application.yml
file add the following line replacing with your custom ngrok URL.
HOSTNAME: https://username.ngrok.io
6. Webpack Dev Server
To speed up development of TS files (from root of the project in a new shell)
./bin/webpack-dev-server
Your Own Data?
To make the site work locally, you should follow the directions for how to do a Broadcast.
Troubleshooting
bundle install
issues
If you get a crash when running gem install puma -v '4.3.5' --source 'https://rubygems.org/'
, this strange Puma build error has been reported on macOS Catalina and Big Sur
puma_http11.c:203:22: error: implicitly declaring library function 'isspace' with type 'int (int)' [-Werror,-Wimplicit-function-declaration]
to fix run
gem install puma:4.3.5 -- --with-cflags="-Wno-error=implicit-function-declaration"
Then re-run bundle install
and it should run through cleanly now.
for more info see https://thoughtfulapps.com/articles/rails/puma-implicitly-declaring-library-function-error
Checks
We use both testing and linters and formatters to ensure consistency in our codebase. On every PR, the following are checked:
Rspec
To run the test suite run:
rails spec
Prettier — FOR TYPESCRIPT/JAVASCRIPT
To run on the entire codebase, run
yarn prettier app/ spec/
To run on only one file, run
yarn prettier app/javascript/controllers/my_controller.ts
However, running prettier this way doesn't actually change or prettify your files. All it does is give you warnings.
To have prettier actually re-write your files, you'll want to run it with the --write
flag.
yarn prettier --write app/ spec/
You'll want to do this for any Typescript file or spec you add or modify.
Rubocop — For Ruby
Make sure any Ruby code you add or modify passes Rubocop.
You do this by running rubocop
on your console. To tell Rubocop to correct your syntax for you use rubocop -A
The full docs are here: https://docs.rubocop.org/rubocop/1.3/index.html
Streaming Architecture
There are several goals of our Streaming Architecture:
- Low Latency
- Replay Enabled
- Sycnchronized Events
- Browser-Based
There are three main components to how a stream works.
- Publishing / Ingest - The "streamer" is uploading a video stream to our system and we both record it and send it to live audience members.
- Live Viewing - When the video is live, we need to get the video to the audience member as quickly as possible
- Replay Viewing / VOD - After the broadcast is completed, audience members can re-watch the recording with synchronized events.
The source for this image can be found in the docs
folder. Please update whenever you can!
Environment Variables
Flags
These may need to be changed on occasion
VELVET_ROPE: true/false - Is the landing page on the root or not?
PERSPECTIVE_API_ENABLED: true/false - Used to turn on or off the Perspective API
PERSPECTIVE_API_SCORE_THRESHOLD: 0.0...1.0 - Cutoff values for not publishing chat messages, 1.0 being the worst and 0.0 being innocent
Third Party Configurations
AppSignal
APPSIGNAL_FRONTEND_KEY: string - Needed for AppSignal Javascript Reporting
APPSIGNAL_PUSH_API_KEY: string - AppSignal's Backend Push Key
APPSIGNAL_APP_ENV: string - AppSignal "environment" tag, used to structure reporting in Appsignal
GOOGLE_CLOUD_KEY: string - Access to the Google Cloud Perspective API
Mux
MUX_TOKEN_ID;
MUX_TOKEN_SECRET;
AWS
Used for S3 storage
AWS_ACCESS_KEY_ID;
AWS_BUCKET;
AWS_SECRET_ACCESS_KEY;
LogDNA
LOG_DNA_KEY;
Twilio
TWILIO_ACCOUNT_SID;
TWILIO_AUTH_TOKEN;
TWILIO_PHONE_NUMBER; //- The actual phone number we send from
IFTTT
IFTTT_PUSH_KEY;
Fanout.io
GRIP_REALM_ID;
GRIP_REALM_KEY;
GRIP_URL; //- optional if you want to override