HTTP Streaming in Varnish

Since April this year we've been working with a few customers implementing proper support for streaming of objects in Varnish. The code was feature complete in August and has matured since, now reaching a stage where we are comfortable releasing it.

A short history of Video on the Web

Streaming of Video on the web has had a long and somewhat painfull history. In the early days of the web, the techonlogy was dominated by propriotary protocols all requiring browser plugins to work properly. Since you loose most of your audience when you are requiring them to install a plugin, it didn't really catch on. The situation wasn't helped by various networks blocking ports, making the transport impossible.

Allmost any network out there will allow HTTP to pass through it. The protocol can be proxied, passes through NAT and is generelly well understood. If you have a IP address chances are you have access to HTTP. This makes HTTP a good candidate for transporting Video.

How HTTP streaming works with Varnish

All of the implementations we've seen have the following in common. You have a H264 video stream, either linear (live) or a big file, usually acompanied by AAC audio. You have a stream fragmentor, which chops the stream into segments, giving each segment a separate URL. The client then requests each segment in a separate requests. The notion of the stream is superimposed through pointers in each segment or by maintaining some sort of playlist. It is up to the client to join the segments toghether in a smooth manner. For linear (live) video it looks like this:

Streaming over HTTP with Varnish 2.x

You could of course use earlier versions of Varnish to distribute video as well. The problem is that Varnish won't deliver the object to the client until it has downloaded the complete object itself. This add latency and if you have a cache hiarchy, the latency will accumulate. For users switching from one stream (channel) to another the latency will be clearly felt unless the segment is already present in the cache closest to the client.

So, basically what we needed was the ability to start delivering the object before it is completly downloaded to the Varnish.

Limitations in the implementation in Varnish 3.0

Varnish 3.0 has some support for streaming objects, but I can't say that I'm very happy with the semantics. It only allows one client to stream the object and others are placed on hold while the object is being streamed. If this client is a 3G phone downloading a 100 megabyte object other clients are likely to give up waiting and go and do something else.

The new and improved streaming implementation

Martin, the developer who implemented the new and improved streaming, gave a presentation on the user group meeting in Baltimore in September. Basically, his code lifts the limitations of the 3.0 release and allows Varnish to deliver the objects, while they are being fetched, to multiple clients.

The last month he's been working with PHK to push the code into trunk. He is also merging the code into the current 3.0 codebase and Tollef, our release manager is preparing a 3.0.2s, which consists of the 3.0.2 release with the streaming support patched in. The release was ran through the fryer, Kristians stress testing harness, last night and has passed all the tests. Release is immient.

Want to know more?

We've written a white paper where we describe the implementation and our experiences in more detail. If you are interested in our assistance implementing this, we'd be happy to share our experience and help you get up and running. Please contact us and we'll talk some more.