Cold Hard Code

For those not familiar in what's happened since Rails has taken off, and Django has become a very robust and popular framework, there has been a movement towards something coined WSGI, Web Server Gateway Interface. This is a specification that is very prevalent in the Python community, and is generally accepted as a Good Idea. In a nutshell, it tries to normalize and enhance communication from an application to the frontend webserver (so that lighttpd, apache and nginx differences are nullfied)

Ruby has a reference implementation called Rack, which takes and shares many ideas of WSGI, but makes it Ruby-ish.

Perl now has Plack, an astounding effort by Tatsuhiko Miyagawa. Plack has been awarded the "Module of the Year" award at London Perl Workshop, and in my opinion is very well deserved.

Within Plack, you can write very lightweight applications just using what Plack offers. You can also use Plack to handle deployment of several other frameworks, including Catalyst (via Catalyst::Engine::PSGI).

As an example of what can be done, I first saw Tatsumaki. Tatsumaki is an event-driven HTTP server that handles streaming (as well as long-poll comet services).

I built an application using Tatsumaki to handle streaming live race results to people who couldn't make it to the track.

It worked out very well, except for a small memory leak (that has been fixed). I had, at peak, 615 concurrent clients with an average of about 380 concurrent clients.

I was using the Plack server directly (Tatsumaki::Server), and having clients connect directly to it. I had a peak throughput of 10Mbps, with an average of 1.6Mbps of data going through Tatsumaki.

I call that a fantastic success, and thank Miyagawa-san for his hard work in getting this software to where it is. Without Tatsumaki, and Plack, I would have been doing a hacky long-poll solution that would have been much slower and less featured.

I also built a YUI3 based Multipart XmlHttpRequest library so that I can stream using YUI3 (I prefer this over jQuery) but IE failed to work, so I had to fall back to long-poll clients which worked more similarly across all browsers.