The cloud...it's full of *people*!

This is the story of the development of a small piece of software to deploy to the cloud, developed by a cloud of people, and how nominally unrelated groups interacted along the way.

I'm interested in having a way to easily create new Rails servers in the cloud (specifically AWS) and to deploy my apps to them. I'm familiar with the bits and pieces that are involved but I hadn't stitched them together. I started to write a Rails app to manage my AWS assets, partially as a path to learning extjs, but when the AWS Console came out I ditched that and started back on the original problem. Here are the different bits that I planned to use, and how I became aware of them:

• Sprinkle, by Marcus Crafter (http://github.com/crafterm/sprinkle/tree/master) - Sprinkle handles deployment of resources on Ubuntu boxes in a pretty easy-to-understand way. Unfortunately I have a strong preference for Postgres over MySql (blame Simon Harris) and the default Sprinkle deployment recipes all use MySql. Still, it didn't take me long to make some extensions to install Postgres instead of MySql.



• Passenger-Stack, by Ben Schwarz (http://github.com/benschwarz/passenger-stack/tree/master) - Sprinkles for Apache, Passenger, Mysql, Memcached & Git. I found out about this via a Ryan Allen tweet.



• Amazon Web Services (http://aws.amazon.com/) - well known, but the fairly recent addition of Elastic Block Storage (EBS) gives a way to easily create persistent data that outlives a given instance, say for a Postgres database.

The last piece to this puzzle is a web site called oDesk (http://www.odesk.com/). I was talking to a client last week and she mentioned that she thought Australian Rails developers were the most expensive in the world. When I asked her what she was doing she said that she was using off-shore Rails developers and was finding good people who would work for US$20/hr, from India but also from the USA. It seemed like the recession in the USA was having interesting side affects. I was definitely curious how using oDesk would work from a client perspective.

So I had this small problem that I didn't have time to work on, and my curiousity about oDesk, and it seemed natural to solve them together. I posted my job on oDesk - produce a script that, given an EC2 instance and an ESB volume, would automatically configure the instance to use Rails with Passenger and Postgres, storing the data on the EBS volume. In addition, provide a test application so I can verify the Capistrano deployment to the instance. I posted this as a fixed price job, and my plan was to use a few different people and compare the outcomes. I posted my job last Sunday (it's currently Thursday).

I had my first response in a few hours, from within Australia! Twenty four hours didn't bring any more responses, so I pushed my job in front of a number of Rails developers who had high ratings. Many declined as they were too busy, some thought the suggested price was too low given the need to learn about AWS, and some still haven't responded. I got two more positive responses, one from an experienced developer in St. Petersburg and one from a less experienced developer on the east coast of the USA. The bids were $333.33 (Australia), $250.00 (Russia), and $166.67 (USA). The expected development times were 1 day, 1 day and 1 week.

I accepted the responses and nominated a starting date of Tuesday. On Wednesday I got my solutions from Australia and St Petersburg. I've tested the Russian solution and it works fine. There seem to be couple of issues in the Australian solution but the developer
is being very responsive, and the USA solution isn't due yet.

What's more interesting is what else happened in the 24 hours of development. The Russian developer forked passenger-stack on GitHub and hosted the fork as a public repository on GitHub (which was fine by me). However, I found out about this through an email from Marcus Crafter, who knew I was looking for a Postgres solution and pointed the fork out to me! Then I saw these tweets from Ben Schwarz (@benschwarz) "@kouky Just so you know, someone forked passenger-stack overnight rolling postgres support in, I'll try and pull everything together :)" and "@atnan @kouky, I've just blindly merged the PostgreSQL stuff as I'm up in the hills. I'll check it all out tomorrow :)" The status message for passenger-stack on GitHub now says "Adding support for PostgreSQL. Sprinkle will prompt the user to select MySQL/PostgreSQL server and the relevant Ruby database drivers."

So it looks like my little experiment ended up sponsoring the change that I wanted in the core implementation of passenger-stack. I suppose I could have asked Ben to do it originally, but that seemed like an imposition and I would have only achieved one of my goals. Now I have a script to configure my EC2 instances automatically, a better feeling for oDesk and off-shore development, and I've discovered that you can always sponsor the features that you want in a piece of open source software. Well worth my costs :-)