Or should that headline have been A/B Testing Comes to Google App Engine API? Google’s PaaS has quietly been rolling out a release every month. In its second release of the year, the App Engine team has focussed not just on its usual fixes, but a series of changes for the Administration Console and a platform features that are bound to make other PaaS vendors sit up and take note: support for A/B Testing of your web application.

App Engine 1.6.3 was announced at the App Engine blog and the A/B Testing support is a valuable addition for existing clients. A/B Testing, also known as split testing or bucket testing, is a popular technique in which you serve different versions of your web application to a different set of customers. It is particularly useful when you want to release some new features but want to test out the feedback via a selective set of users. For more information on A/B Testing, refer to the Smashing Magazine article.

The A/B Testing support is called Traffic Splitting in App Engine and can be configured in a simple way. You need to simply identify a non-default version of your application, specify the percentage of traffic it should receive and choose the type of Splitting i.e. by IP Address or Cookie. Keep in mind that this feature is experimental and is bound to change moving forward.

The other platform change announced is the support for DKIM (Domain Keys Identified Mail) signature, which will be added to every email going out from a user of a Google Apps domain from a request originating on that domain, or from an app administrator with an account on a Google Apps domain.

In addition to the above platform changes, a bunch of changes have been added to the Administration Console, key among them being ability to shutdown an instance from the Instances View and specifying the amount of storage and duration of time for the logs. The first GB of Logs storage is free but additional storage will be at $0.24 per Gig per month.

The release has also seen several bug fixes for both the Python and Java runtimes. App Engine also supports the Go language runtime but there is no mention about it in this version. For App Engine 1.6.3 Release Notes, refer to the Python and Java sections respectively.

(Image Credits: Smashing Magazine)

Sponsored by

This guest post comes from Amber Case, co-founder of Geoloqi, a private, real-time platform for location sharing. She also speaks frequently on Cyborg Anthropology, the study of humans and computers.

In this post I’ll describe how we planned, built and tested a truly real-time location-based game with Socket.io, Redis, Node.js, and what we learned along the way. Over the past few months, we’ve spent the majority our free time building a real-time game as a test for our location platform, Geoloqi. We call the game MapAttack! due to its map-based nature. Two teams compete to capture the most points on the gameboard. The gameboard, in this case, is the city streets of the neighborhood the players are in.

We set each geofence up with a point value that would give players points for entering geofences. The idea was that a virtual map would be set up on top of the real world, and players on red and blue teams would try to capture all of the geofence points in the game before the other team. To capture a point, the phone would have to detect when the player entered the fence, determine the point value of the fence, notify the player that he received the point, turn the geofence the color of the team, and then add the point to the player score and the overall team score.

Why Build a Real-Time Geofencing Game?
We wanted to create a game that allowed people to physically interact with the real world instead of a computer console like a first person shooter or a real-time strategy game. We were inspired by playing a real-life version of Pac-Man called Pacmanhattan, invented by graduate students at the Interactive Telecommunications Program at NYU in 2004. We played it at Portland’s WhereCamp conference in 2008, and we wanted to see if we could make a GPS version of the game, as Pacmanhattan relied entirely on phone calls and physical maps. We also needed a good demo of Geoloqi’s streaming API.

Technical Challenges
Here is an overview of the problems we had to focus on in order to build the game.

Handling the detection of users entering and leaving 200+ geofences concurrently.
Handling the volume of location-updates from all the phones in a given game (20 or more users per game).
Allowing each phone and web browser watching the game to be able to see the movements of players and the geofences changing color in real time. Every phone in the game sends its location to the server, which broadcasts that data to every other phone and browser watching the game.
Handling errors and differences in GPS technology on different smart phone models in order to ensure a fair gameplay experience.

Differences in GPS Hardware
GPS signals are known for reflecting off of tall buildings in urban settings. This causes inaccuracy and inconsistency in location data. It is less-pronounced in newer phones, but it greatly shows in older ones.

Before our Streaming API
Before we finished the Geoloqi streaming API and before we started using Node.js and Socket.io, everything was based on polling for new updates. Phones reported their location at 5 second intervals and the browsers would update the game board in 5 second intervals.

Using Socket.io, Node.js, Redis, and Sinatra Synchrony

Socket.io

Socket.io is a cross-browser web socket implementation allowing us to do real-time data updates on the browser and also supports older browsers. We can use the latest technology without requiring all of our users to update to the newest browsers, thanks to Socket.io falling back to older technologies in older browsers. This allow us to do instant updates across browsers and the phones in the game.

Node.js

Node.js is Evented I/O for V8 Google’s Javascript implementation for Chrome, implemented with a reactor pattern, that enables for large amounts of asynchronous data traffic.

We use a Node.js server to stream the location data from the phones to the Redis pub/sub channel. It publishes to Redis, and another Node server subscribes to that redis channel. Our Node.js server receives updates from the phones using a custom protocol similar to Google’s Protocol Buffers, which is essentially a very compact binary JSON.

When a browser wants to start streaming data, it connects to the Socket.io server and that server then subscribes to the Redis pub/sub channel. The Socket.io server sends that data via Websockets to the browser, falling back to Flash or long-polling if Websockets is not available.

In essence, Socket.io allows us to use Websockets, which are completely new, but also allows this to work on older browsers thanks to the fallback tricks.

Redis

Redis is an open source, advanced key-value store that has support for message queues using something called publish/subscribe, or pub/sub (not to be confused with PubSubHubbub).

From the higher level what this lets us do is handle the difficulty of sending data to all of the phones in the game and the browser in real-time. Every phone in the game sends its location to the server, which broadcasts that data to every other phone and browser watching the game.

One of the interesting things about the publish/subscribe system is that with a traditional system you have to maintain connections and iterate through each in order to pass data through them. The alternative would be that if you had 10,000 users you’d have to iterate through an array of 10,000 connections, which would be very slow and prone to locking up on socket problems.

Using Redis pub/sub is like starting a radio station. Once it is turned on, people (in this case, browsers) can just listen in. This allows us to do real-time data updates to clients (browsers and phones) at a massive scale.

Sinatra Synchrony

Sinatra::Synchrony is a small extension for Sinatra that dramatically improves the concurrency of Sinatra web applications. Powered by EventMachine and EM-Synchrony, it increases the number of clients your application can serve per process when you have a lot of traffic and slow IO calls (like HTTP calls to external APIs). Because it uses Fibers internally to handle blocking IO, no callback gymnastics are required! This means we can just develop as if we were writing a normal Sinatra web application.

Sinatra::Synchrony allows us to do asynchronous programming (ala Node.js), except that it wraps the callbacks in Fibers (which are basically co-routines in Ruby). This allows you to do synchronous programming while taking advantage of asynchronous code. Aside from being easier to program this way, it also allows us to switch to a different concurrency/parallelism strategy if we need to. Kyle Drake developed Sinatra Synchrony specifically for MapAttack. Drake’s work became popular after he made a presentation on Sinatra::Synchrony at PDX Ruby.

The MapAttack Game Server
Finally, there is the MapAttack Game Server. In this case the Game Server is a simple database that takes care of storing the player point data that is displayed on the map and on the phones as players grab points in real-time.

Source Code
We made the source code for MapAttack available for download. You can download or fork the source code for the MapAttack website, iPhone application and Android application. If you build anything interesting with it, please let us know.

Upcoming Games
We’ll be bringing MapAttack! to WhereCamp Portland on October 7-9, 2011. We’ll give an overview of the technology there as well. If you plan to be in the area, please join us.

Sponsored by

Related ProgrammableWeb Resources Geoloqi API Profile, 2 mashups

Clustrix, a Y Combinator graduate from 2006, launched today with the claim that it’s built a transaction database with MySQL-like functionality and reliability that can scale to billions of entries. Clustrix plans to sell its appliance (which consists of more than a terabyte of memory and its proprietary software) to web firms that don’t want to take on the complicated task of sharding their data (replicating it across multiple databases), or moving to less robust database options like Cassandra or a key value store such as what’s provided by Twitter.

This is big stuff. Indeed, Paul Mikesell — CEO of Clustrix and the former co-founder of storage system success story Isilon — said the goal is to use its appliance to solve a growing problem for companies managing large amounts of data, such as big travel, e-commerce and social websites. As the web grows more social, companies are trying to keep track of more pieces of data about users and their relationships to other users. This creates complicated and large databases that can slow down access to user information, and thus the end user experience.

We’ve written about myriad attempts to solve these data scalability problems, attempts that have spawned appliance startups and whole branches of code designed to help sites scale their data, from Hadoop to Cassandra to Twitter’s Gizzard. Mikesell said the product could replace the need for caching appliances such as those offered by Schooner or Northscale, but could also work in conjunction with them.

As for some of the open source options, new programming languages like Bloom, or cloud-based scalable databases such as Microsoft’s SQL Azure or Rackspace’s partnership with FathomDB, Mikesell is confident that the ability to replicate the functionality of a relational database at webscale without sharding or tweaking the existing code is powerful enough that customers would pay $80,000 for a 3-node machine containing the software. There are plenty of companies reluctant to trust the open-source spin-outs from companies like Twitter and Facebook.

The market is clearly there for scalable relational database products (GigaOM Pro, sub req’d), so if Clustrix can take the $18 million invested in it from Sequoia, ATA Ventures and US Venture Partners and turn it into an Isilon-like exit, more power to it.

Building webscale or cloud applications is hampered by figuring out ways to spread tasks out over thousands of computers without slowing things down, or requiring too many people to keep things running. Virtualization and faster storage helps, as do new databases (GigaOM Pro sub req’d) and caching techniques, but right now folks are trying to adapt how they program computers to reflect that one has now become many.

Bloom, a programming language created at the University of California, Berkeley by a group led by Joseph Hellerstein, is one such effort. Bloom was profiled this week as one of the top 10 emerging technologies by MIT’s Technology Review, because it could help cloud computing continue to scale. Here’s how, according to Technology Review:

The challenge is that these languages process data in static batches. They can’t process data that is constantly changing, such as readings from a network of sensors. The solution, ­Hellerstein explains, is to build into the language the notion that data can be dynamic, changing as it’s being processed. This sense of time enables a program to make provisions for data that might be arriving later — or never.

Hellerstein also gave an extensive interview to HPC in the Cloud this week about what Bloom is and the problem it’s trying to solve. From that interview:

To put it simply, our what our work is trying to do is start with the data itself and get people to talk about what should happen to the data step-by-step through a program without ever having them specify at all how many machines are involved. So, when you ask a query of a database you describe what data you want—not how to get it.

The interview lays out how this programming effort  came about (building network protocols) and who might care most about using Bloom (Amazon, Google or anyone with big data needs), but for me the best part of it was how Hellerstein underscored that the ability to harness a heck of a lot of servers and treat them as a single computer is the next big shift in information technology.

We can call it cloud computing, webscale applications or merely bigger data centers, but the key element here is that the hardware has gone social in ways that require many-to-many ways of communication and delivering instructions to the processors — inside the servers, between the servers, and soon, between data centers. The exciting aspect of this shift is that while larger companies like Google, Yahoo and Amazon are innovating, there is plenty of room for startups with a new appliance, server, networking technology or chunk of code to make waves — and hopefully, money.

For more on the effort, please check out the FAQ’s Hellerstein has posted on his blog.

Image courtesy of Flickr user tibchris

Any debate over open vs. closed systems has to touch on open-source software and the ways in which companies are attempting to build code as a community effort, while still profiting from it in some way. So I chatted with Mark Shuttleworth, CEO of Canonical, the company that supports Ubuntu, about how it walks the line between spending to support open-source software and finding a business model that works.

Canonical’s 330 employees are responsible for maintaining, supporting and selling service for Ubuntu, an open-source version of the Linux operating system for servers, desktops and computer manufacturers. Some 120-150 of the Canonical employees contribute directly to the new releases of the software that come out every six months, and most of the company’s revenue comes from supporting enterprise server customers and makers of computers that want to put Ubuntu on desktops. Consumers also download the software, but few pay Canonical for support. The company is not yet profitable.

Shuttleworth believes that in order to develop a strong business model around an open approach, one has to create an open option early, ideally through a strong standardization process and one also needs to have a lot of different open-source projects fighting it out.  For example, in the operating system world there wasn’t a strong history of open alternatives, which meant that Ubuntu had to out-open its proprietary competition, which has high costs.

In that way it has pushed Canonical perhaps further out toward open on the spectrum. Shuttleworth calculates the direct costs of being so open as bringing people together in ways that empowers them and makes them feel like members of a community, as well as reaching out and putting in place the infrastructure to create a company. However, there are indirect costs as well.

“There is a myth that being open is necessarily more efficient and cheaper, but there are no hordes of people showing up to do the hard stuff,” Shuttleworth says. “Occasionally wonderful, magical things happen — really incredible things do happen, like people show up unexpectedly with brilliant ideas — but it’s still hard and expensive and you still have to be willing to do all the hard and expensive things and do it in an open fashion. And you’re still likely to be accused of being open only when it’s convenient.”

He points to the cloud computing market as one that tends to give a lot of lip service toward openness but where a lack of a big standardization effort and robust open source competition could lead to a relatively closed ecosystem.

“The basic story there is pretty bad at the moment,” Shuttleworth says. He notes that proprietary infrastructure, hypervisors and even the APIs and ways data is stored can lock folks into one cloud for life. “We need real open alternatives early in the process, making it possible for people to build own cloud infrastructure that responds to the same APIs that Amazon’s do.”

He’s accepted that Amazon Web Services’ APIs for its web services, while not created through an open standards group, have become a de facto standard and said that it’s more efficient to build open-source code around Amazon APIs rather than try to develop new ones for accessing the cloud. Canonical has a partnership agreement with Eucalyptus, which offers open-source software to create an AWS-compatible cloud, where people can use Ubuntu  and Eucalyptus to create their own cloud computing platform. But Shuttleworth would like to see more open-source options other than Eucalyptus  for building out a cloud computing service of your own.

At the platform-as-a-service level, the issue around openness will be around moving data from cloud to cloud easily. There’s room there for an open standard or open databases, he said. But at every level, when considering building a business around open source software, he he believes that “you want a common and clear standard with competing open source versions using that standard.”

That keeps proprietary vendors at bay, and gives the companies building a business around the open-source software a chance to decide where they want to be on the open-to-closed spectrum. But it also introduces the prospect of fragmentation, which we’ll leave for a later post.

Related content from GigaOM Pro (sub req’d):

For Open Cloud Computing, Look Inside Your Data Center

SpringSource, a division of VMware, has purchased the open-source cloud messaging company behind the RabbitMQ software. The value of the deal was undisclosed, but the purchase of Rabbit Technologies Ltd. is yet another effort by VMware to become the operating system for enterprise clouds (GigaOM Pro, sub req’d) and add value to its commoditized hypervisor. It’s also the latest example of a company selling proprietary software buying up an open-source software company aimed at the cloud.

Cloud providers use RabbitMQ to create a messaging server allowing them to quickly manage the flow of messages between applications. It can also be used to notify users of a web service when content on the site has changed, such as when someone posts a Facebook photo and the service sends an email out notifying all a user’s friends.

The RabbitMQ code was created by Cohesive FT and LShift based on the relatively young AMQP standards effort backed by major banks, Cisco and a handful of smaller companies. As hardware is virtualized, translating some of the network equipment like load balancers into software allow services running on the virtualized hardware to scale better. Hopefully we’ll learn more about SpringSource, RabbitMQ and VMware’s plans for becoming the cloud OS when VMware CEO Paul Maritz speaks at our Structure conference in June.

Introduction to AMQP Messaging with RabbitMQ
View more presentations from Dmitriy Samovskiy.

Image courtesy of Flickr user Joshua Davis

Amazon Web Services has launched its Simple Notification Service (Amazon SNS), which allows developers to create a push notification system for applications. The service allows companies to deliver messages to customers of their applications or even to other applications in a couple of different formats, among them HTTP and email. Amazon SNS could be used for system administrators in an IT department (notifying clients if they’re hitting a certain limit on storage capacity or that latency on their service is too high), or it could be used to build out notifications for mobile applications, such as letting consumers when friends check into a location, or when they have new email.

Developers using the service pay per instance, as with all Amazon cloud products. The price includes a per-request, notification delivery and data transfer fee, but developers can get started with Amazon SNS for free. Each month, Amazon SNS customers get the first 100,000 Amazon SNS Requests, the first 100,000 notifications over HTTP and the first 1,000 notifications over email free. After that, prices range from 6 cents to $2 per 100,000 messages sent for delivery and 8-15 cents per gigabyte of data transferred.

Related GigaOM Pro content (sub. req’d):Report: Delivering Content in the Cloud

Image courtesy of Flickr user Ed Siacoso (aka SC fiasco)

Cloud computing is big right now, but the sheer number of options, and the lack of interoperability, can be an issue for developers. Vendor lock-in reduces the options available to those working on cloud computing initiatives, while the variability between APIs inevitably leads to many hours spent pouring over documentation when comparing cloud service providers.

There are a number of projects that can reduce, or even eliminate, some of these problems by exposing the functionality of a number of cloud service providers through a consistent interface. Here is a list of 7 such projects.

DeltaCloud is a RehHat initiative that provides a REST API and user tools to manage EC2, RHEV-M, RackSpace and RimuHosting, with VMWare ESX and more coming soon.

libcloud is a pure python client library for interacting with many of the popular cloud server providers. The libcloud web pages lists support for 11 cloud services provides.
The Simple Cloud API brings cloud technologies to PHP and the PHPilosophy to the cloud, starting with common interfaces for File Storage Services, Document Storage Services and Simple Queue Services. The API page shows support for Amazon S3, Windows Azure Blob Storage and Nirvanix, with more to come.
jclouds is an open source framework that helps you get started in the cloud and reuse your java development skills. Jclouds support many clouds including Amazon, VMWare, Azure, and Rackspace.
The Dasein Cloud API provides an abstraction for applications that wish to be written independent of the clouds they are controlling.
OpenNebula is an open and flexible tool that fits into existing data center environments to build any type of Cloud deployment. OpenNebula 1.4 supports Xen, KVM and VMware virtualization platforms and on-demand access to Amazon EC2 and ElasticHosts Cloud providers. It also features new local and Cloud interfaces, such as libvirt, EC2 Query API and OGC OCCI API.

Cloudloop is a new open source Java API and command-line management tool for cloud storage. Unfortunately the cloudloop web site was hacked recently, but you can find more information with this post on TheServerSide.

Just as with the growth of these cloud APIs and platforms, there are more libraries and abstraction initiatives all the time. Feel free to share any others we missed in the comments.

Pat Gelsinger, who moved to EMC late last year after 30 years at Intel, is stirring things up at the storage giant with a plan to virtualize and federate storage so data and compute can truly be linked together (hat tip The Register). The implication of this vision is that organizations will have the ability to keep constantly changing information up to date around the world in real time despite the challenges of moving huge amounts of data over networks that measure data in in gigabytes rather than petabytes.

In a presentation on Thursday, Gelsinger pointed out that compute and storage are rapidly getting better about dealing with more information, while networks  are trying to catch up. “Compute is doubling every two years. Storage doubles every 15 months, and networking is much much much slower, like every four years, so how do you deal with latency bandwidth and consistency?” Gelsinger said.

Gelsinger’s answer is caching. Imagine a two-way content delivery network built on EMC appliances that tracks and replicates changes made to data at one node and then pushes them out to all the other nodes as quickly as possible. Gelsinger calls this freeing the information from physical storage, but it sounds more like making sure your information is in a bunch of different physical storage containers. He mentions EMC’s acquisition of intellectual property from Yotta Yotta as offering the breakthrough required to build this technology.

But at the end of the day, this is all a big if, not an actual product yet.  If EMC can link storage and virtualized machines together, the data center that “follows the sun” — basically moving compute loads around the world where it’s cheapest to run them – or automatic failover for cloud services become possible. However, it will be controlled by a proprietary hardware vendor, which certainly clouds its prospects a bit.

Digg, the San Francisco-based social media company, is dropping MySQL and instead betting its future on Cassandra, an open-source data store. It’s just the latest sign of the growing popularity of the software, which was developed (and open sourced) by Facebook to search through its inbox. While Facebook has since backed off Cassandra, Digg plans to open source all its work on Cassandra and champion the software’s development and adoption.

In a blog post on the Digg blog, John Quinn, Digg’s VP of engineering, writes:

Perhaps our most significant infrastructure change is abandoning MySQL in favor of a NoSQL alternative. To someone like me who’s been building systems almost exclusively on relational databases for almost 20 years, this feels like a bold move.

What’s Wrong with MySQL?

Our primary motivation for moving away from MySQL is the increasing difficulty of building a high performance, write intensive, application on a data set that is growing quickly, with no end in sight. This growth has forced us into horizontal and vertical partitioning strategies that have eliminated most of the value of a relational database, while still incurring all the overhead.

Digg is just the latest high-profile convert to the NoSQL world. Instead of using databases such as MySQL, many of the companies that deal in near-real-time information are opting for new kind of data stores — most of them open source, such as Cassandra and CouchDB.

Cassandra is roughly the open-source equivalent of Google’s Big Table. It was intended by Facebook to solve the problem of inbox search; the company needed something that was fast, reliable and had the ability to handle read and write requests at the same time. Messaging in an environment as heavily used as Facebook requires a system that can not only store data but also provide results for search queries at blazing fast speeds.

Stu Hood, the technical lead for the search team in the Email & Apps division of Rackspace, recently said:

I think that distributed databases solve a problem that a lot of companies with large datasets have had to solve independently in the past…Cassandra has an approach that hybridizes the Bigtable and Dynamo models, where a lot of its competitors chose to take one path or the other. Over the Bigtable clones, Cassandra has huge high-availability advantages, and no single point of failure (possible because of the eventually consistent approach). When compared to the Dynamo adherents, Cassandra has the advantage of a more advanced datamodel, allowing for a single “row” to contain billions of column/value pairs: enough to fill a machine. You also get efficient range queries for the top level key, and even within your values.

Data Presentations Cassandra Sigmod
View more presentations from jhammerb.

In a post last year, contributing writer Gary Orenstein pointed out that thanks to these attributes, Cassandra has potential applications beyond inbox search that include “recommendation engines, targeted advertising, and content search, particularly when you combine many concurrent inputs and output requests to the same data set.”

Digg is a prototypical application. The company tells me that it gets:

40 million visitors a month, who in turn account for roughly 500 million page views a month.
20,000 daily submissions

It also generates:

170,000 daily Diggs
19,000 comments

As these numbers suggest, there is a high amount of interaction between the system and its users. No wonder Digg digs Cassandra!

Related content from GigaOM Pro (sub req’d):

What Cloud Computing Can Learn From NoSQL.