Streaming HLS video (redux)

In a previous post, I suggested a method for streaming HLS video using ffmpeg. Unfortunately, although I could have sworn it worked at the time, based on what I now know, this seems unlikely.

As far as I can tell, there’s no way you can use the hls muxer, and pipe the output; certainly not in the way I needed to use it, as the playlist is updated over time.

All is not lost though, the solution is to output the necessary files to disk:

app.get('/hls/video', function(req, res) {
    var output = "videos/out.m3u8";
    var stats;
    try {
        stats = fs.statSync(output);
    } catch (e) {
    }
    if (stats) {
        return res.redirect(output);
    }
    var redirected = false;
    var watcher = fs.watch("videos/", (e, f) => {
        if (e === "rename" && f === file) {
            if (!res.finished) {
                res.redirect(output);
                redirected = true;
            }
            watcher.close();
        }
    });
    var proc = ffmpeg(output);
    proc.on("exit", code => {
        if (code === 0) {
            if (!redirected && !res.finished) {
                res.redirect(output);
            }
            watcher.close();
        } else {
            res.status(500).end();
        }
    });
});
  
function ffmpeg(output) {
    var cmd = "ffmpeg";
    var filter = "some complex filter expr";
    var args = ["-i", "video1.mp4"];
    ...
    args.push(
        "-vcodec", "libx264",
        "-pix_fmt", "yuv420p",
        "-f", "hls",
        "-hls_playlist_type", "event",
        "-profile:v", "baseline",
        "-level", "3.0"
    );
    args.push(output);
    return spawn(cmd, args);
}

When a video is requested, we check if the playlist already exists, if so we redirect immediately. If not, we run the ffmpeg command, and add a filewatcher.

Once the playlist (m3u8) is created, we redirect. As this is a “live” stream (playlist type: event), the playlist will be updated as the new chunks become ready.

We need to take care not to leak the filewatchers though, in the event of an error. The final piece is to serve the output artifacts, as static files:

app.use("/videos", express.static("videos", {
    setHeaders: function(res, path, stat) {
        if (path.indexOf(".ts") > -1) {
            res.set("cache-control", "public, max-age=300");
        }
    }
}));

The ts chunks can be cached (upstream, by varnish & cloudfront), but we need to ensure the m3u8 isn’t, as it will be changing until the video is complete.

Streaming video to iOS devices

UPDATE: this is fake news, see my newer post for more info.

It seems that neither iOS devices, nor Safari on OS X, support mp4; so if you’re trying to stream video, you need to provide another format.

The recommendation is to use HLS, which fortunately is supported by ffmpeg, you merely need to adjust your incantation:

app.get('/hls/video', function(req, res) {
    res.contentType('application/vnd.apple.mpegurl');
    var proc = ffmpeg();
    proc.stdout.pipe(res);
});
 
function ffmpeg() {
    var cmd = "ffmpeg";
    var filter = "some complex filter expr";
    var args = ["-i", "video1.mp4"];
    ...
    args.push(
        "-vcodec", "libx264",
        "-f", "hls",
        "-hls_time", "9",
        "-hls_list_size", "0",
        "-profile:v", "baseline",
        "-level", "3.0",
        "pipe:1"
    );
    return spawn(cmd, args);
}

I could probably use conneg to decide which format to return, rather than the uri, but I’m not convinced that my caching infrastructure (varnish and cloudfront now!) would handle that correctly.

Streaming video from ffmpeg

ffmpeg is a fantastic tool for converting, concatenating, or otherwise fiddling with video content. If you can generate what you need in advance, then you can upload it to s3 (or some other CDN like option); but sometimes you need to stream video on demand.

First, you need to get the ffmpeg incantation right:

ffmpeg -i video1.mp4 -i video2.mp4 ... \
       -filter_complex "something something" \
       -movflags "frag_keyframe+empty_moov"
       -f mp4 pipe:1

Using pipe:1 sends the output to stdout (as documented here), and the movflags are needed to allow streaming in mp4 format.

With this in place, it’s easy to pipe the output data to a browser using expressjs:

#!/usr/bin/env node
"use strict";

const express = require('express');
const { spawn } = require('child_process');

var app = express();

app.get('/video', function(req, res) {
    res.contentType('video/mp4');
    var proc = ffmpeg();
    proc.stdout.pipe(res);

    res.on("close", () => {
        proc.kill("SIGKILL");
    });
});

app.listen(4000);

function ffmpeg() {
    var cmd = "ffmpeg";
    var filter = "some complex filter expr";
    var args = ["-i", "video1.mp4"];
    ...
    args.push(
        "-filter_complex", filter,
        "-s", "1280x720",
        "-acodec", "aac",
        "-vcodec", "h264",
        "-movflags", "frag_keyframe+empty_moov",
        "-f", "mp4",
        "pipe:1"
    );
    return spawn(cmd, args);
}

I found I ended up with “zombie” ffmpeg processes, if the client connection had closed before the video ended (because it’s still trying to write data to the pipe?). There’s probably a neater way to solve that, but kill -9 is pretty effective!

At this point, you can stream some video; but this solution won’t scale, every request runs the ffmpeg command, which is pretty cpu intensive. It would be relatively simple to cache the output, either in memory or on disk; or you could put the node process behind a caching proxy.

However, the best solution for our needs seemed to be using this as an “origin server” for Cloudfront. This means that only the first request (of each type) hits our server, and the response is cached for as long as necessary.

If you need to render different videos for every request, then you’ll just have to throw money at it!