When Ryan and I started working together, we were both very passionate about continuing some of the work we had started at Linden Lab regarding different ways of building software.  So for the past year, we’ve conducted an interesting experiment, deciding not to hire like a traditional tech startup, but instead building our first software products by paying a team of 101 people distributed worldwide a total of about $360,000 to do the development as 2,200 small jobs, coordinated by software that we built, as described below.  The way we are doing things is working really well, and we think it is a model for a new and better way of building many types of software.  First the big features of this system, and some takeaways:

Develop in very small/parallel contracts: Rather than hiring a core team of developers like a typical startup, we chose instead to break our work into small well-described chunks which were open to public bidding.  We wrote a piece of software called the Worklist to manage this process.  It lets us breaks large pieces of development into small pieces  that would typically take just a few hours to complete (our team has done 2,200 jobs over a years time, meaning an average of 8 per working day) – the average job had about $160 in fees attached to it.   We made the process of reviewing the code (see below), making a bid, getting a code review, and then getting paid for your work fast enough to allow working in these very small units feasible.  We also set up a sandbox system where on bid acceptance a developer could immediately login, start make changes and immediately view the results of their work as well as show them to the entire community of participating developers.

Keep all Source Code in Public:  From day 1, anyone has been able to see all our code as it is being developed.  The license isn’t open source, but the code is open.  This means that if you want to come and do engineering work for us, you can easily examine the code you are about to work on.  This is an important part of making the small jobs system work, because in order to be willing to take on a small job (for relatively less money), you need to be able to see the code before bidding.  Although there are certainly competitive risks to doing startup development with an open codebase, we think the benefits have greatly outweighed the risks.

Connect participants in a live chatroom: Although there are a number of people working from around the world on the software, they are connected by a live chatroom that we built allowing everyone to easily communicate, get help, and see what other people are doing.  This is a critical step – the feeling of working together on a project in this way must be like being in the same room together.  So we created things like notifications via SMS when jobs are ready to review, a searchable chat history allowing a new person to look up everything ever said about a topic, and a way to quickly ‘ping’ other developers to reach them.

Trust participants to set their own prices for common activities: When jobs are added to our system, developers submit bids for the actual development work.  But our process also has smaller fixed-fee items like code reviews or identifying and entering a bug that don’t make sense to bid on.  For these things, we simply suggested that developers set their own fees, choosing a number that was appropriate given their work, bearing in mind that everyone else can see their charges and that (of course) we might not accept new work from them later if we felt the charges were inappropriate.

Lots of interesting takeaways emerge from this experience.  For example, the graph below shows the total amount of money that different developers made during the year, from largest (around $30-40K) to smallest (about $5).  You can see that this has the classical ‘long tail’ seen in many internet distributed collective systems, like Wikipedia or Second Life:  There is a core of substantial contributors and then a larger number of people who do smaller pieces of work.

We were able to get help from a wider number of people with diverse skills sets, while still operating on the limited budget of a startup.  Having 100 people help to build an application is something that typically could only happen in a larger company – so for example we were able to do iPhone development, add real-time sockets for chat and other live data, link to Facebook, and get some very cool javascript data visualizations, without having full-time team members with those skills.  If you look at this curve it suggests that from a competitive perspective a company can perform better if it unlocks value in the tail by supporting this kind of a model – if not for all it’s development, at least for some of it.

Another critical discovery was how well and reasonably people behave and set prices when in a supportive environment with a high degree of transparency.  For example, the process we built for code review invites developers to review other developers work (required prior to check-in) and then simply add a fee for whatever they feel is a reasonable price for their work. Because these prices are public (anyone can review the prices charged by anyone else), they tend to rapidly settle to a fair amount without our needing to haggle over their value with the developers.   As an example, here is a typical job with fees for code review attached.   As an additional incentive, we’ve experimented with a bonus system where every month or so, the developers working together are given an substantial amount of money that they can electively give out to other team members working on the project in any way that they choose.  This sort of system effectively delegates bonus allocation to everyone in a way that further creates a strong self-supporting community.   This isn’t a surprising outcome, given the success of other decentralized peer systems like open source, but it is a novel step to apply it directly to compensation in a work environment.

Clearly there is an obvious and interesting comparison to open source software development, where participants can see and share all code, submit changes in parallel with a variety of different check-in processes, and (typically) aren’t paid.  Our intent was to combine some of the powerful aspects of the open source process with a model in which we actually paid people for their efforts.  It seems that we have been successful in this regard – the experience of live communication with other developers, peer reviews, and a diverse set of contributors feels very similar to the open source experiences we have been involved with.

Although our system is new and definitely will need lots of refinement as we grow (and we have encountered and overcome many obstacles already in building it), it shows great promise.  As an example, the majority of the live chatroom described above was built using our system in a period of about 10 days in February 2010 at a cost of about $5,000.  It was really exciting to watch the core system come online over a period of hours and days as the different pieces were tested, reviewed, and checked in by the contributors.  Having been involved in software development for a couple decades now, I’ve never seen anything built so quickly and effectively.

We are also interested in adding other software projects to the worklist, to see how well this process can build other systems and be used by developers other than it’s original authors.  If you have a project that you would like to host on our system, please contact us about how to get started.  And, of course, if you are a developer interested in joining us and working within this system for pay, please visit the Worklist, get an account setup, and start making bids on jobs!  We’d love to have your help and feedback.

This entry was posted on Tuesday, February 1st, 2011 at 11:06 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.