Hybrid language for getting high-performance inner loops in your code, called from python or javascript, JIT-compiled on the machine. dependcy graph/mapreduce primitives drive the whole thing. Nice. "KL is the custom language used to define operators in Fabric Engine. KL combines the benefits of high performance languages like C++ with the benefits of dynamic languages like JavaScript and Python. ‘KL’ stands for ‘Kernel Language’ and refers to the scope of the language: writing operators. Kernels are small stateless blocks of code with an entry function. The syntax of KL is similar to the syntax of JavaScript and C."

GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job is can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input and pipe it into commands in parallel.

The bestest introduction to the easiest parallelisation doohickey I know

Jit-compiled scientific computing language that promises the performance of c, but has a REPL and such. intriguing.

lite concurrency for python for science

GoldenOrb is a cloud-based open source project for massive-scale graph analysis, built upon best-of-breed software from the Apache Hadoop project modeled after Google’s Pregel architecture.

Our goal is to foster solutions to complex data problems, remove limits to innovation and contribute to the emerging ecosystem that spans all aspects of big data analysis.

This is a hacked together script to connect to all of your running EC2 instances, and to provide a single SSH prompt amongst all of them, and to assist in uploading, splitting, and downloading files. email me at winniningham at and email server called gmail.com :P

mapreduce alternative thingy for distributed nodes

Looks like Haskell has autoparallelising decorators

neat strong timing for ruby (based on chuck+ green threads)

groovy is to the JVM as Reia is to the Erlang VM

a framework of timing primitives for distributing code across unruly servers, processses, algorithms, implementations. In ruby ("for now", they say)

handy educational tutes on various concurrency models; a bit of a hadoop emphasis

monads explained using javacript

3d visualization for javascript by sencha labs. uses webgl and workers, handles dynamic importing from various sources

wiki comparison of python concurrent workflow managers

it's possibly to call out from an event loop into a green thread in a different OS thread in python, using Fibra. (other concurrency frameworks, maybe?). Concurrency and UI toolkits, wikkid.

calling conventiosn in a continuation passing style for node.js.

updated Beazely coroutines - pure python coroutine and scheduler implementation, showing how to built upa a concurrent python event architecture without callbacks, including, for bonus points, spinning off evented subprocesses to let io and cpu-bound processes live in peace

streaming JSON parser at last!

bigger, enterprisier, buzzwordier python evolutionary library. seems pure GA as opp GP, but with tasty operators, visualisation, and parallelisation support

my preferred way of doing websocket programming from python. concurrency though non-blocking IO, none of google's reference implementation thread idiocy, higher performance... good times.

non-blocking communication with non-python subprocess from python

this is now the stdlib python concurrency doohickey. I wonder if it could support concurrency apart from proc/threads? RMI anyone?

asynchronous RMI including websocket support.

old-skool c++ bulk network'd computation

hadoopish network bulk synchronous library

parallel graph computation for boost

SMP and network process launching for python. look straight up -lotsa-stuff lob oriented instead of mapreduce or anything trendy like that.

class-based python concurrency with transparent distribution across network and process

max/msp style flow control programming for python, with concurrency and weirdness. Seems to be an open-source yahoo pipes re-implementation.

greenlet-based network library with recently added epoll support.

Gevent started as Eventlet with a few bugs fixed and a few features dropped.

1. gevent is built on top of libevent

2. gevent’s interface follows the conventions set by the standard library

hi performance minimal queu protocol

this visualization of how the GIL works - and doesn't is a good visualization of how the GIL works - and doesn't

firefox's yield keyword from the source

firefox's yield keyword

holy crap - did you know that python supported full coroutines? I didn't.

trampoliing styles as a scheduler in javascript

"A trampolined program is organised as a single loop in which computations are scheduled and their execution allowed to proceed in discrete steps."

google's compiled high-performance language with erlang-like concurrency. Emphasis on simplicty of lang implementation and programmer productivity. OO, but no type hierarchy. standard library already supports enough infrastructure to run web servers.

run a javascript IO (HTTP?) server with event'd io. built on Chrome's v8 engine. To consider: implication of this.

diesel's in the running for most elegant co-routine system

generator based pyton keep-the-connection-alive thingy. rumoured to have poor test coverage, though.

an OO/wsgi style of python web framework - light, purportedly, but focusses on providing the server in python. no recommended data storage layer.

parallel requests in Google App Engine!

Via simon willison... live collaboration awesomeness built in python/JS.

crazy - a distributed, high security language for computation over crappy, slow, compromised or untrusted networks.

another twisted-lite? Whatever- anything to feed my coroutine fetish. "Weightless presents a way to implement data-processing programs, such as web-servers, with coroutines in Python. The results are lightweight, efficient and readable programs without call-backs, threads and buffering. It is designed to neatly fit with asynchronous I/O or coroutine-based scheduling."

minimalist python concurrency along a message-passing actor line. libraries for games, evented io and such