etsy

Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.

The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll, it can handle thousands of simultaneous standing connections, which means it is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features — every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)

See the Tornado documentation for a detailed walkthrough of the framework.

The Web Sockets API enables web browsers to maintain a bi-directional communication channel to a server, which in turn makes implementing real-time web sites about 1000% easier than it is today.

Currently, the only reasonable technical facility available to browsers to communicate to web servers is XMLHttpRequest. Sites that update in real-time like FriendFeed use a number of horrible hacks on top of XMLHttpRequest like long-polling to get data in real-time. (If you are interested, Tornado ships with a chat demo application that uses this long-polling technique - here is the JavaScript in all its hacky glory).

Web Sockets support a much simpler interface that enables both the client and the server send messages to each other asynchronously

otherinbox

on scala

Lift is an expressive and elegant framework for writing web applications. Lift stresses the importance of security, maintainability, scalability and performance, while allowing for high levels of developer productivity. Lift is open source software licensed under an Apache 2.0 license.

Cassandra is a distributed storage system for managing structured data while providing reliability at a massive scale.

Deploy a Ruby on Rails app on EC2 in five minutes

mapreduce implementation

The part I found interesting was the first detailed description of using the MapReduce model to run large-scale Expectation Maximization (EM) computations in parallel. An implementation of this on Hadoop and Amazon EC2 will let you tackle some large scale

etech keynote