Introduction
I always wondered how the backend of internet radio stations worked. I recently came across the nginx-rtmp-module and decided to try it out. This module allows you to ingest a live stream using the RTMP protocol and serve it to multiple clients using HLS. HLS is a protocol that allows you to serve a live stream to multiple clients using HTTP.
What is RTMP?
RTMP stands for Real Time Messaging Protocol. It is a protocol that allows you to stream audio and video over the internet. It is used by many streaming services such as Twitch, YouTube and Facebook. RTMP is a TCP based protocol and uses port 1935 by default. It is used for ingestion and is not suitable for playback because it is not supported by most browsers.
Set up project files
For this project we require a few files in the root directory. These are:
- Dockerfile
- Nginx config file
- Makefile
Dockerfile
Pretty self explanatory, this is the file to create the Docker image. We manually install Nginx and the RTMP module. Then set up permissions and overwrite the default config file with the one shown below. Then for the CMD we start Nginx in the foreground.
FROM buildpack-deps:bullseye
# Versions of Nginx and nginx-rtmp-module to use
ENV NGINX_VERSION nginx-1.17.3
# Install dependencies
RUN apt-get update && \
apt-get install -y build-essential libpcre3 libpcre3-dev libssl-dev zlib1g zlib1g-dev certbot python3-certbot-nginx
# Download and decompress Nginx
RUN wget -O ${NGINX_VERSION}.tar.gz https://nginx.org/download/${NGINX_VERSION}.tar.gz && \
tar -zxf ${NGINX_VERSION}.tar.gz
# Download and decompress RTMP module
RUN git clone https://github.com/sergey-dryabzhinsky/nginx-rtmp-module.git
# Build and install Nginx
# The default puts everything under /usr/local/nginx, so it's needed to change
# it explicitly. Not just for order but to have it in the PATH
RUN cd ${NGINX_VERSION} && \
./configure \
--with-http_ssl_module \
--add-module=../nginx-rtmp-module && \
make && \
make install
# Create /nginx/hls folders to hold .m3u8 snippets
RUN mkdir /nginx && \
mkdir /nginx/hls
RUN chown -R www-data:www-data /nginx
# Overwrite default config file
COPY nginx.conf /usr/local/nginx/conf/nginx.conf
EXPOSE 1935 80
CMD ["/usr/local/nginx/sbin/nginx", "-g", "daemon off;"]
Nginx config file
We will overwrite the default config file with this one. We listen on port 1935 for RTMP streams and port 80 for HLS streams. We also set up CORS to allow the HLS stream to be played in the browser.
worker_processes auto;
events {
worker_connections 1024;
}
# RTMP configuration
rtmp {
server {
listen 1935; # Listen on standard RTMP port
chunk_size 4000;
application show {
live on;
# Turn on HLS
hls on;
hls_path /nginx/hls/;
hls_fragment 3;
hls_playlist_length 60;
# disable consuming the stream from nginx as rtmp
deny play all;
}
}
}
http {
sendfile off;
tcp_nopush on;
# aio on;
directio 512;
default_type application/octet-stream;
server {
listen 80;
location / {
# Disable cache
add_header 'Cache-Control' 'no-cache';
# CORS setup
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length';
# allow CORS preflight requests
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204;
}
types {
application/dash+xml mpd;
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
root /nginx/;
}
}
}
Makefile
This is not strictly required but makes things easier when you are running lots of commands. I have included commands to build the image locally, run it locally, build it for linux and push it to Dockerhub.
The build-linux command is required as I am working on a Mac which uses the ARM architecture, and the regular docker build command builds an image that runs on this architecture. We will be deploying the container on a Linux machine in a cloud provider which uses the AMD64 architecture, so we need the extra --platform linux/amd6 to build a compatible image.
There is no private information stored on the image so it is safe to push to Dockerhub. If you are using a private repository you can use the docker login command to log in to your account before pushing.
build-local:
docker build -t rtmp .
run-local:
docker run -d -p 1935:1935 -p 80:80 --name nginx-rtmp rtmp
build-linux:
docker build --platform linux/amd64 -t rtmp .
push-dockerhub:
docker tag rtmp <your-dockerhub-username>/rtmp && \
docker push <your-dockerhub-username>/rtmp
Testing locally
For this step you will need to install OBS. This is a free and open source software for video recording and live streaming. It is available for Windows, Mac and Linux.
Run the container locally using the run-local command. Then open OBS and go to Settings > Stream. Set the service to Custom and the server to rtmp://localhost/show. The stream key can be anything you want, I used show. Click Apply and then OK.
Then click on Start Streaming and you should see the stream appear in the browser at http://localhost/hls/show.m3u8.
Deploy to Digital Ocean
This step can be automated (queue follow-up blog post) but I will outline the manual setup here to make it easier to understand. If you don't already have a Digital Ocean account you can sign up here.
Create a droplet
Log in to your Digital Ocean account and click on Create > Droplets. Select the Docker image (comes pre-installed with docker and related tools) - the cheapest plan will be fine. I chose the Frankfurt region but you can choose whichever is closest to you.
Configure the droplet
Once the droplet is created, ssh into it and run the follow commands to pull the image from Dockerhub and run it.
docker pull <your-dockerhub-username>/rtmp
docker run -d -p 1935:1935 -p 80:80 --name nginx-rtmp <your-dockerhub-username>/rtmp
Once this is up and running, you can test it by streaming from OBS as before. You will need to use the IP address of the droplet instead of localhost in the OBS settings.
Set up HTTPS
Update the Dockerfile:
FROM debian:bullseye
LABEL maintainer="Charles Harris <charles.harris.de@gmail.com>"
# Versions of Nginx and nginx-rtmp-module to use
ENV NGINX_VERSION nginx-1.17.3
# Install dependencies
RUN apt-get update && \
apt-get install -y build-essential libpcre3 libpcre3-dev libssl-dev zlib1g zlib1g-dev wget git
# Download and decompress Nginx
RUN wget -O ${NGINX_VERSION}.tar.gz https://nginx.org/download/${NGINX_VERSION}.tar.gz && \
tar -zxf ${NGINX_VERSION}.tar.gz
# Download and decompress RTMP module
RUN git clone https://github.com/sergey-dryabzhinsky/nginx-rtmp-module.git
# Build and install Nginx
RUN cd ${NGINX_VERSION} && \
./configure \
--with-http_ssl_module \
--add-module=../nginx-rtmp-module && \
make && \
make install
# Create /nginx/hls folders to hold .m3u8 snippets
RUN mkdir /nginx && \
mkdir /nginx/hls
RUN chown -R www-data:www-data /nginx
# Overwrite default config file
COPY nginx.conf /usr/local/nginx/conf/nginx.conf
EXPOSE 1935 80 443
CMD ["/usr/local/nginx/sbin/nginx", "-g", "daemon off;"]
and the nginx.conf file:
worker_processes auto;
events {
worker_connections 1024;
}
# RTMP configuration
rtmp {
server {
listen 1935; # Listen on standard RTMP port
chunk_size 4000;
application show {
live on;
# Turn on HLS
hls on;
hls_path /nginx/hls/;
hls_fragment 3;
hls_playlist_length 60;
# disable consuming the stream from nginx as rtmp
deny play all;
}
}
}
http {
sendfile off;
tcp_nopush on;
# aio on;
directio 512;
default_type application/octet-stream;
server {
listen 443 ssl;
server_name livestream.nkfunky.com;
ssl_certificate /etc/letsencrypt/live/livestream.nkfunky.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/livestream.nkfunky.com/privkey.pem;
location / {
# Disable cache
add_header 'Cache-Control' 'no-cache';
# CORS setup
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length';
# allow CORS preflight requests
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204;
}
types {
application/dash+xml mpd;
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
root /nginx/;
}
}
}
then rebuild the image and push it to Dockerhub. Then ssh into the droplet and run the following commands to pull the new image and run it.
# Set up commands
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
docker pull cookershades/rtmp
# Certbot setup certificate
sudo docker run -it --rm -p 443:443 -p 80:80 --name certbot -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot certonly --non-interactive --agree-tos --email "charles.harris.de@gmail.com" --standalone -d livestream.nkfunky.com
# Start server
docker run -p 1935:1935 -p 80:80 -p 443:443 -v "/etc/letsencrypt:/etc/letsencrypt" cookershades/rtmp
LetsEncrypt SSL certificates have a validity of 90 days, so you will need to run the following command to request a certificate renewal within this time period:
# To renew the certbot certificate
sudo docker run -it --rm -p 443:443 -p 80:80 --name certbot -v "/etc/letsencrypt:/etc/letsencrypt" -v "/var/lib/letsencrypt:/var/lib/letsencrypt" certbot/certbot renew
You can easily run this in a cron job on Vercel.