Here's an email from one of our consultants, and my response (edited for anonymity and to removing the internal cursing!).
So, our product manager asked me today whether we could (just) let quality slide for a while, in the interests of getting stuff done faster.
I gibbered and muttered for a bit, and then gave an answer which amounted to "I don't know how to do that". Which is true; having personally experienced the benefits of practices like pairing, automated tests, and refactoring, and having done things that way exclusively for a quite a while, I'm not sure I know HOW to do it the old way.
I'm not sure that the his concern has gone away, though. He describes his experience as "mostly startup-like stuff", throwing stuff together quickly, and potentially throwing in away a few short years later. He's certainly never worked in an agile environment before, and is doubtful about the value of some of our practices.
Any advice?
As with most companies with a history of software development, there are people who have been burned by non-agile development, and people who haven't had that experience, and they have very different perceptions.
Here are some principles:
1) The customer is in control of the quality sliders for a project. We (IT, not just Cogent) are a service provider who give advice and recommendations, and then act on the instructions/direction of the business.
2) If the customer asks us to do something we (Cogent) fundamentally disagree with, we tell them that we are not the right group to provide that service and we quietly, respectfully disengage. Think of a surgeon who was told to perform operations without sterilisation, and that someone else would deal with the infection later.
3) There certainly are times when "not doing things the right way" is appropriate. I wrote the first cuts of Cogent Times with no test cases, because I didn't know if it would get any traction (i.e. be useful enough for people to use it). I don't regret it, and I would do it that way again. Now I'm adding test cases incrementally. When Runway was only developed by Simon it didn't have tests either - we demonstrated that this didn't scale. Kent's blog on the Flight of a Startup describes this well.
4) I think it's important to understand timeframes though. I believe that letting (internal) quality slide for a while will get stuff done faster for a few weeks. Other people seem to think that it will let them go faster for months or years. I disagree. Put simply, dropping quality won't get things done faster. We have the courage of our convictions here - we use these practices when we spend our own money on our own applications, when we really, truly, want to go as fast as we can.
5) The impact of lower (internal) quality is exacerbated by:
number of people working on the application
quality of the people working on the application
size of the application
level of understanding of the problem domain
level of change in requirements
flexibility of the technology stack
tolerance for defects in production
If your application is being built in a modern language, by one skilled person who is familiar with the product domain, working with people who know what they want and can describe it in detail without hesitation, false starts or iterations, and no one loses any money (or lives) if there are production defects, then what we do is definitely overkill. I've yet to see an environment that matches this description.
Having said all that, in the bigger picture the product manager isn't necessarily the person who makes the decisions on where the quality sliders are. Those are frequently decisions that need to be made at a higher level as they affect the total life cycle cost of the application, and the product manager is only responsible for the development part. This is the classic breakdown mode of process change - the people who bear the cost of the process change are not the people who reap the benefits of the product change. Producing a stable, defect free application that can be extended isn't necessarily in the product manager's interest, though it may very well be in the company's overall interest, especially when funding is allocated per project rather than per product. I present that argument mostly for completeness, since I think that the payback period of our practices is short enough that they're in the product manager's interest as well.
Things get even worse when the organisation funds work as "projects" rather than investing in "products". In these situations the value of feedback is reduced (since success is judged on the delivery of the promised features) and the focus is on the short term (the delivery phase for those features).
However it's also fair for the customer (and I mean the big-picture customer, not just the product manager) to suggest experiments - a good question in those situations is what will be measured to determine if things are better, and better for who? You need all the stakeholders involved in these discussions, not just the product owners.
Also bear in mind that we are not the only people to say these kinds of things. I believe part of what you're seeing is descent into the "easiest" possible implementation of Scrum. Here's what they're implicitly asking for:
"To be able to change their mind on requirements each iteration, and to obtain the benefits of this flexibility, without either performing enough up-front analysis to understand the fundamentals of the problem domain, or making any investment in the technical robustness of the application to support change"
This is very appealing from a business perspective, but it's an illusion. Jeff Sutherland has said he's never seen a hyper-productive Scrum team that didn't use the XP technical practices. Martin Fowler has written on what he calls Flaccid Scrum. The trend is to no longer consider the practices you mention as "agile", but simply "contemporary" - things you should be doing without a second thought.
So my first response to these kinds of suggestions (actually, my second, after I stop cursing) is that if the proposals would make things go faster then we'd happily implement them. But they won't. If I'm challenged further then I would ask them to find someone else on the technical staff who believe that the proposals will make things go faster, and then I'd step aside for them.
Seriously, hospital administrators don't tell surgeons how to maintain hygiene and where to cut. The customer can do whatever they want - we don't have to do it for them.
Not sure if that helps or not.
Steve
Tuesday, December 8, 2009
Monday, October 19, 2009
Roles in a self-organising team
One of the things that I'm asked to do fairly regularly is define the roles for an agile team, be it for Scrum, XP or some other flavour of agile. I usually end up doing this based on required skills, but my first impulse is to refuse, because I don't believe that a self-organising team actually has roles and I think that defined roles limit both the individual and the team. Let me examine that point before I explain how I do go about defining roles and why.
There is one role on a self-organising team - team member. The responsibility of that role is to bring all their skills to bear on the team's problems and organise their contributions to maximise the team's effectiveness.
When something needs to be done on an agile team everyone on the team should be able to see it, and in many cases it will be obvious which individual or individuals are the best suited to getting it done. The simplest approach is to allocate tasks based on skills, but the team might decide to use other criteria as well - they may need to grow their capacity in some area and decide to give the task to someone with less experience to give that person some practice; they may have limited capacity and be forced to give the task to someone who needs to learn the required skill, usually with mentoring from someone on the team. There will also be team-specific and project-specific reasons for not allocating tasks based simply on skill set. The important thing is that the reasons are well-articulated, and that the team member accepts responsibility for the task rather than having it arbitrarily assigned to them.
Although multi-skilled teams are normal and effective, provided the team is dedicated to achieving a result, has sufficient time and is reasonably self-aware you can imagine any arbitrary skills starting point. We could take a team that consisted entirely of testers and give them some work to do. The first tasks they encountered would require some analysis, so someone from the team would need to do this. They wouldn't complete the analysis as quickly or to the same quality as someone with the relevant experience, and they might not enjoy it as much. They might also need to get outside expertise (training and/or mentoring), but they would get through the work eventually. The team might find someone who wanted to grow into this role and make them the primary analyst, or they might decide that no one enjoys it and the role should be rotated through the team (this is probably less efficient, but it explicitly ranks satisfaction and retention ahead of efficiency). The same pattern can be repeated with development and anything else the team needs to get done.
Don't get me wrong - this is clearly a slow path to success and I wouldn't recommend it, but it is a valid approach. It's a contrast to the behaviour I see more often, which is "I can't do X, because I'm a Y". "Can't" is definitely the wrong word - finding the right word will determine how you respond to this assertion. If the person is really saying "I don't like doing X" then keep the assignment short and try to find some other way to make it attractive. If they're saying "I don't know how to do X", then they need training, guidance and support. If they're saying " I won't do X" then they probably shouldn't be part of the team.
That's why I don't like roles - why do I provide descriptions of roles and responsibilities? It's because I would rather start with a team with diverse skills than spend a lot of time developing the same skills, and most organisations understand that as a request for roles. I also appreciate that most people have gravitated towards a role that suits their preferences and skills, and asking people to move into other areas requires both time and motivation that we usually don't have a the beginning of project.
My hope is that companies that go to the trouble of assembling multi-discipline teams will also recognise the benefits of keeping those teams together for long periods of time. Time is the essential ingredient to building role-independent teams - it helps build trust between the team members, it lets people experiment and learn, and it gives the team opportunities to get feedback on different approaches to task assignment.
There is one role on a self-organising team - team member. The responsibility of that role is to bring all their skills to bear on the team's problems and organise their contributions to maximise the team's effectiveness.
When something needs to be done on an agile team everyone on the team should be able to see it, and in many cases it will be obvious which individual or individuals are the best suited to getting it done. The simplest approach is to allocate tasks based on skills, but the team might decide to use other criteria as well - they may need to grow their capacity in some area and decide to give the task to someone with less experience to give that person some practice; they may have limited capacity and be forced to give the task to someone who needs to learn the required skill, usually with mentoring from someone on the team. There will also be team-specific and project-specific reasons for not allocating tasks based simply on skill set. The important thing is that the reasons are well-articulated, and that the team member accepts responsibility for the task rather than having it arbitrarily assigned to them.
Although multi-skilled teams are normal and effective, provided the team is dedicated to achieving a result, has sufficient time and is reasonably self-aware you can imagine any arbitrary skills starting point. We could take a team that consisted entirely of testers and give them some work to do. The first tasks they encountered would require some analysis, so someone from the team would need to do this. They wouldn't complete the analysis as quickly or to the same quality as someone with the relevant experience, and they might not enjoy it as much. They might also need to get outside expertise (training and/or mentoring), but they would get through the work eventually. The team might find someone who wanted to grow into this role and make them the primary analyst, or they might decide that no one enjoys it and the role should be rotated through the team (this is probably less efficient, but it explicitly ranks satisfaction and retention ahead of efficiency). The same pattern can be repeated with development and anything else the team needs to get done.
Don't get me wrong - this is clearly a slow path to success and I wouldn't recommend it, but it is a valid approach. It's a contrast to the behaviour I see more often, which is "I can't do X, because I'm a Y". "Can't" is definitely the wrong word - finding the right word will determine how you respond to this assertion. If the person is really saying "I don't like doing X" then keep the assignment short and try to find some other way to make it attractive. If they're saying "I don't know how to do X", then they need training, guidance and support. If they're saying " I won't do X" then they probably shouldn't be part of the team.
That's why I don't like roles - why do I provide descriptions of roles and responsibilities? It's because I would rather start with a team with diverse skills than spend a lot of time developing the same skills, and most organisations understand that as a request for roles. I also appreciate that most people have gravitated towards a role that suits their preferences and skills, and asking people to move into other areas requires both time and motivation that we usually don't have a the beginning of project.
My hope is that companies that go to the trouble of assembling multi-discipline teams will also recognise the benefits of keeping those teams together for long periods of time. Time is the essential ingredient to building role-independent teams - it helps build trust between the team members, it lets people experiment and learn, and it gives the team opportunities to get feedback on different approaches to task assignment.
Tuesday, October 13, 2009
An interesting exchange
I wrote this in response to a customer email, but it seemed like something that I could put here:
On 14/10/2009, at 9:15 AM, ... wrote:
From: Steve
On 14/10/2009, at 9:15 AM, ... wrote:
Morning Steve,
Saw this article and thought it was interesting. http://www.javaworld.com/community/node/3530
Regards
...
From: Steve
It is interesting. I think it's hard to get the tone right when you're dismissive of agile zealots (who it's hard to agree with, almost by definition), without being dismissive of agile in general. Having been involved with agile for a long time I have a nice clear centre that tells me that agile should be adaptive, but I've also seen the problems that come from being too adaptive too early.
I totally agree that the process for one or two really talented, motivated people is different from the process you use for larger groups/applications. Alistair Cockburn has been very explicit about this in his writing about the Crystal family of methodologies, and Kent Beck has touched on the same subject from a different perspective when he wrote his blogs about "Flight of the Startup". The hard part for most of us is dispassionately looking our our team and ourselves and changing process over time. It's easy to err on the side of too much, too soon or too little, too late, but very hard to judge it just right.
On the "where's the Access of today", that's a question that we've thrown around within Cogent as well. My personal opinion, shared by some others in the company but not everyone, is that we've gone backwards over the last 10 years in productivity and our ability to deliver things quickly. Our capabilitity has increased - we can deliver bigger, more sophisticated applications - but in the process everything seems to have become harder.
I agree with the poster who mentioned the combination of Ruby on Rails and Heroku - that's the combination that I'm using for my personal projects now. I think it's the combination that's the winner - there are other frameworks that to me look more promising than RoR but with each of them I have to manage my production environment myself (for some of them even setting up a production environment is a challenge). Heroku takes care of that and make deployment as simple as possible.
On the OS X desktop MacRuby looks very promising, leveraging Inteface Builder with the power of Ruby. I don't play in the Windows space, so I don't have any idea what the possibilities are there.
...
Steve
Wednesday, October 7, 2009
My Kindle DX and I
I've fielded more questions about my Kindle, and how I use it in Australia, than just about anything else I've mentioned on Twitter so I thought I'd put the information in a blog post.
After October 7 this information is really only useful if you want a Kindle DX - if you're happy with the smaller Kindle then you should buy an internationalised version and avoid all the issues I'll describe.
Books are heavy, and I often want to carry around more than one for reference, especially if I'm doing some writing. At lot of the things I want to read these days are also in PDF and I haven't found a good way to read those - I end up reading a lot either on public transport or just before bed and my laptop isn't convenient in those environments. I tried using my iPhone to read PDFs but that doesn't work for me (other people seem very happy).
The Kindle is great for reading material that's been formatted for the device - I haven't found it to be that good for PDFs (more on that later). It's much better for me than either my laptop or my iPhone. The screen has great contrast, a huge viewing angle, and when I first opened the box I couldn't believe the starting image was being displayed on something electronic.
The first problem is that Amazon won't sell you a Kindle DX if your shipping address is outside this USA. My first solution was to have one shipped to a friend in the USA and then shipped on to me, but it turns out that there are people on eBay who can help. My KIndle DX arrived in the original Amazon packaging, with the original seals intact, with a new shipping wrapper pasted over the original.
The Kindle arrives with a USA style power adapter, but the approach is the same as the iPhone - it's a USB connecter that plugs into the adapter. Since I already have an iPhone charger I can plug into that, or I can charge from my computer.
The Kindle is fairly frugal with power. Since you won't be using WhisperNet you can turn off the wireless connection to reduce battery use. The screen is bi-stable, so it only uses power when you move from one page to another. I charged my Kindle last Friday night - it's currently Thursday morning and I'm still using it happily.
Amazon (who I'll get to later) isn't the only source of content for your Kindle. You can use .mobi files from other sources, or PDFs. For example, I buy combo packs from the Pragmatic Programmers, so I have .mobi versions of all of their books, and their magazine comes out in the same format.
If you want to buy content from Amazon you won't be able to do it with an Australian credit card/address. I've set up a separate Amazon account for my Kindle purchases and given it my old Brooklyn address (you can use that too if you ask me nicely). I buy an Amazon gift voucher on my Australian account and then use the gift card to buy the content on my Amazon Kindle account. It's an extra step, but it's not really that much of a problem. I download the content to my laptop and transfer it to my Kindle via USB (the Kindle looks like any other disk drive).
Yes, it's one more device, but I've put my Kindle in the back pocket of my backpack and I really don't notice the extra weight at all. It's always there if I want to read, and I use it with one hand even if I'm standing on public transport.
There's no economic reason for me to have a Kindle, but I'm really enjoying the experience. The proof of the Kindle pudding will be how it scales to having the majority of my library on a device I carry around with me. I suspect that this will change the way I treat information.
After October 7 this information is really only useful if you want a Kindle DX - if you're happy with the smaller Kindle then you should buy an internationalised version and avoid all the issues I'll describe.
Why a Kindle?
Books are heavy, and I often want to carry around more than one for reference, especially if I'm doing some writing. At lot of the things I want to read these days are also in PDF and I haven't found a good way to read those - I end up reading a lot either on public transport or just before bed and my laptop isn't convenient in those environments. I tried using my iPhone to read PDFs but that doesn't work for me (other people seem very happy).
The Kindle is great for reading material that's been formatted for the device - I haven't found it to be that good for PDFs (more on that later). It's much better for me than either my laptop or my iPhone. The screen has great contrast, a huge viewing angle, and when I first opened the box I couldn't believe the starting image was being displayed on something electronic.
First Problem
The first problem is that Amazon won't sell you a Kindle DX if your shipping address is outside this USA. My first solution was to have one shipped to a friend in the USA and then shipped on to me, but it turns out that there are people on eBay who can help. My KIndle DX arrived in the original Amazon packaging, with the original seals intact, with a new shipping wrapper pasted over the original.
Charging
The Kindle arrives with a USA style power adapter, but the approach is the same as the iPhone - it's a USB connecter that plugs into the adapter. Since I already have an iPhone charger I can plug into that, or I can charge from my computer.
The Kindle is fairly frugal with power. Since you won't be using WhisperNet you can turn off the wireless connection to reduce battery use. The screen is bi-stable, so it only uses power when you move from one page to another. I charged my Kindle last Friday night - it's currently Thursday morning and I'm still using it happily.
Content
Amazon (who I'll get to later) isn't the only source of content for your Kindle. You can use .mobi files from other sources, or PDFs. For example, I buy combo packs from the Pragmatic Programmers, so I have .mobi versions of all of their books, and their magazine comes out in the same format.
If you want to buy content from Amazon you won't be able to do it with an Australian credit card/address. I've set up a separate Amazon account for my Kindle purchases and given it my old Brooklyn address (you can use that too if you ask me nicely). I buy an Amazon gift voucher on my Australian account and then use the gift card to buy the content on my Amazon Kindle account. It's an extra step, but it's not really that much of a problem. I download the content to my laptop and transfer it to my Kindle via USB (the Kindle looks like any other disk drive).
Carrying an extra device
Yes, it's one more device, but I've put my Kindle in the back pocket of my backpack and I really don't notice the extra weight at all. It's always there if I want to read, and I use it with one hand even if I'm standing on public transport.
Summary
There's no economic reason for me to have a Kindle, but I'm really enjoying the experience. The proof of the Kindle pudding will be how it scales to having the majority of my library on a device I carry around with me. I suspect that this will change the way I treat information.
Monday, October 5, 2009
Ubiquitous Language
Business language is a pain in the ass. People are way too smart and capable of dealing with ambiguity, and the more familiarity they have with a particular domain the more ambiguity they can handle. They use multiple words for the same thing (perhaps with subtle differences between the words, perhaps not) and they simultaneously use the same word for different things. If you're not "in the know" it can be very confusing. Worse, it may be confusing the people who *are* in the know - the loose language may be obscuring inconsistencies that only become clear when you try to apply automation (computers aren't as tolerant of ambiguity as people are).
There have been multiple attempts at dealing with this problem. Once upon a time it was common to say that business analysts were translators, responsible for taking things that the business understood and turning them into things that the developers understood. Of course this involves waste, in the form of hand-offs and delays, and inevitably introduces defects.
Extreme Programming introduced the idea of a System Metaphor, but most people find it hard to come up with an appropriate metaphor for their application. When someone mentioned this to Martin Fowler at a conference he said "Oh, we just use the naive metaphor" - that is, the developers used the language of the business.
Other people use different words for the same idea - System of Names for example. Domain Driven Design also has this idea at its core. However, the most common term for this idea is Ubiquitous Language.
Ubiquitous language means that everyone in the team is using the same language for the same things - mainly business concepts, but if there are important technical concepts these should be part of the language as well. The language also needs to be precise - coming up with ubiquitous language often requires disciplined analysis from a number of stakeholders (ubiquitous language does not come from one person working in isolation).
Normally we ask a team to converge on one set of language, but I'm currently working with a project that involves both a vendor and their customer.The vendor already has a well-developed set of language around their product, and the customers have a well-developed set of language around their business processes. Probably neither set of language meets my definition of a ubiquitous language, but we should try move them in this direction. Reducing them to one set of common language would probably be impossible, so we probably have to live with both sets for the duration of the program.
There have been multiple attempts at dealing with this problem. Once upon a time it was common to say that business analysts were translators, responsible for taking things that the business understood and turning them into things that the developers understood. Of course this involves waste, in the form of hand-offs and delays, and inevitably introduces defects.
Extreme Programming introduced the idea of a System Metaphor, but most people find it hard to come up with an appropriate metaphor for their application. When someone mentioned this to Martin Fowler at a conference he said "Oh, we just use the naive metaphor" - that is, the developers used the language of the business.
Other people use different words for the same idea - System of Names for example. Domain Driven Design also has this idea at its core. However, the most common term for this idea is Ubiquitous Language.
Ubiquitous language means that everyone in the team is using the same language for the same things - mainly business concepts, but if there are important technical concepts these should be part of the language as well. The language also needs to be precise - coming up with ubiquitous language often requires disciplined analysis from a number of stakeholders (ubiquitous language does not come from one person working in isolation).
Normally we ask a team to converge on one set of language, but I'm currently working with a project that involves both a vendor and their customer.The vendor already has a well-developed set of language around their product, and the customers have a well-developed set of language around their business processes. Probably neither set of language meets my definition of a ubiquitous language, but we should try move them in this direction. Reducing them to one set of common language would probably be impossible, so we probably have to live with both sets for the duration of the program.
Monday, September 7, 2009
Code Review for Git
I wanted to find a way to contribute to projects that I couldn't pair on, and to be able to scan a project for possible changes without sitting down and taking up someone's time. Sounded a lot like automated code review, so I had a look around at available code review tools. There are a few available - Review Board, Smart Bear and Crucible among them. I know Atlassian and generally like their products so I decided to install Crucible and give it a try.
There were a few confusing configuration problems. Until I read the documentation more closely (at all?) I wasn't aware that the beta Git plugin only worked against local git repositories, and I had to change a configuration file to convince Crucible that Ruby files should be treated as text instead of binaries, but once I got past these problems I clearly had a powerful tool for distributed, multi-party code reviews available to me.
So I created a review, looked at some code, and saw one thing that I would change. I wrote the comment, but things just felt wrong. It had taken me a while to type to comment (I'm not the world's fastest typist, but nor am I the slowest), and it felt like a waste of time. It would have been far faster to make the change and annotate the corresponding commit, but what I had was a code review tool, not an editor. Something wasn't working for me.
When I do a code review I want to achieve two things - I want to fix the code and I want to educate someone (where that someone may be me, if I wrote the original code). What I really wanted was something (an editor) that would let me focus on changes between versions, but let me change the latest version and save it. My VCS could store the comments against the (small) changes and I could point people to the VCS history to see what I'd done, potentially using the same tool.
After looking at a few different tools last night I've settled on Changes. Changes lets you compare directory structures and files, and it comes with some Ruby scripts that will help you retrieve specific revisions from Git, drop them into temp directories and review the differences. I've modified the script to use the current directory if you're comparing to HEAD, so that changes to the head are saved directly to disk and can be committed.
Here's my process:
1) Use GitX to determine the revision I want to use as the base
2) Use changes to view the differences
3) Incrementally work through the files either making changes or excluding the files.
I've created a (silent) video that shows me working through one review. I only make one small change, so it's mostly scrolling and excluding. One small point - the video uses an older version of my Ruby script that created a temp directory even if you were looking at HEAD, so changes were lost. I've fixed that in this version.
I saw some comments that Changes was rather expensive (US$49.99), but it's really cheap compared to many code review tools so I'm pretty happy with this approach.
There were a few confusing configuration problems. Until I read the documentation more closely (at all?) I wasn't aware that the beta Git plugin only worked against local git repositories, and I had to change a configuration file to convince Crucible that Ruby files should be treated as text instead of binaries, but once I got past these problems I clearly had a powerful tool for distributed, multi-party code reviews available to me.
So I created a review, looked at some code, and saw one thing that I would change. I wrote the comment, but things just felt wrong. It had taken me a while to type to comment (I'm not the world's fastest typist, but nor am I the slowest), and it felt like a waste of time. It would have been far faster to make the change and annotate the corresponding commit, but what I had was a code review tool, not an editor. Something wasn't working for me.
When I do a code review I want to achieve two things - I want to fix the code and I want to educate someone (where that someone may be me, if I wrote the original code). What I really wanted was something (an editor) that would let me focus on changes between versions, but let me change the latest version and save it. My VCS could store the comments against the (small) changes and I could point people to the VCS history to see what I'd done, potentially using the same tool.
After looking at a few different tools last night I've settled on Changes. Changes lets you compare directory structures and files, and it comes with some Ruby scripts that will help you retrieve specific revisions from Git, drop them into temp directories and review the differences. I've modified the script to use the current directory if you're comparing to HEAD, so that changes to the head are saved directly to disk and can be committed.
Here's my process:
1) Use GitX to determine the revision I want to use as the base
2) Use changes to view the differences
3) Incrementally work through the files either making changes or excluding the files.
I've created a (silent) video that shows me working through one review. I only make one small change, so it's mostly scrolling and excluding. One small point - the video uses an older version of my Ruby script that created a temp directory even if you were looking at HEAD, so changes were lost. I've fixed that in this version.
I saw some comments that Changes was rather expensive (US$49.99), but it's really cheap compared to many code review tools so I'm pretty happy with this approach.
Thursday, August 27, 2009
How I learned to stop worrying and love Enterprise Agile
I spent Wednesday at the Software Craftsmanship North America Conference and the evening at a cocktail party hosted by Thoughtworks. It was a very interesting day, and in the evening a chance comment by Ward Cunningham helped dispel a misconception I've held for a decade. For the last ten years I've been conflating agile and what I'll loosely call mastery. I think it's fair to say that mastery helps you do agile well, but it's not essential to labelling your project agile.
So I sat down and tried to figure out what "agile" was from my perspective. There are clearly lots of ways that you can attack that problem, but for the moment I've settled on this - a project is agile if it uses scope as the control variable and incorporates feedback mechanisms to facilitate that control.
That's a pretty broad definition, but it helped clear something up for me right away. I've been annoyed by resumes that said "agile experience", because I didn't think the people claiming agile experience met the cut. The problem was that saying "has agile experience" is like saying "has programming experience" - it may be true, but it's not terribly useful. You need a lot more detail before you can form any judgement on the claim - what practices has the person been using, how long have they used them for, can they describe the pros and cons? Almost exactly the kind of information you'd want to have about someone's programming experience.
Mastery, on the hand, is about continual learning and improvement, in this case in the agile context. I try to pursue mastery of agile software development, and I prefer to work with people who aspire to be masters, but that's not the only way to approach agile. Most people in any field are not masters, and many do not aspire to be masters in that particular field, but that doesn't mean they won't benefit from understanding and using agile practices.
So while I will personally continue to pursue mastery of agile software development, and want that to be the focus for Cogent Consulting as well, I hope to spend more time thinking about how to apply agile development in environments that don't have access to master software developers.
So I sat down and tried to figure out what "agile" was from my perspective. There are clearly lots of ways that you can attack that problem, but for the moment I've settled on this - a project is agile if it uses scope as the control variable and incorporates feedback mechanisms to facilitate that control.
That's a pretty broad definition, but it helped clear something up for me right away. I've been annoyed by resumes that said "agile experience", because I didn't think the people claiming agile experience met the cut. The problem was that saying "has agile experience" is like saying "has programming experience" - it may be true, but it's not terribly useful. You need a lot more detail before you can form any judgement on the claim - what practices has the person been using, how long have they used them for, can they describe the pros and cons? Almost exactly the kind of information you'd want to have about someone's programming experience.
Mastery, on the hand, is about continual learning and improvement, in this case in the agile context. I try to pursue mastery of agile software development, and I prefer to work with people who aspire to be masters, but that's not the only way to approach agile. Most people in any field are not masters, and many do not aspire to be masters in that particular field, but that doesn't mean they won't benefit from understanding and using agile practices.
So while I will personally continue to pursue mastery of agile software development, and want that to be the focus for Cogent Consulting as well, I hope to spend more time thinking about how to apply agile development in environments that don't have access to master software developers.
Tuesday, July 28, 2009
Software Cultures
I think every programming language (and frameworks too) generates a dominant culture. It might be implicit in the language/tool itself, or it might happen more gradually through self selection. For example, here's how I distinguish three cultures (the boundaries are inherently fuzzy, and I'm aware that this is totally personal opinion, so people are bound to disagree by either raising counter-examples or saying they don't agree, and that's fine). I also have my tongue firmly in my cheek, but I believe there's an element of truth in each statement.
Java - a fascination with flexibility, generally achieved through complexity (and a healthy(?) does of XML). Ten years ago I would have said Java was focussed on solidity and reliability (in the meeting expectations sense), but I think this has changed over time. New functionality is implemented by adding new options. If it doesn't work, it's because you didn't configure it properly.
Rails - a fascination with bright, shiny new toys. Encourages the image of the developer as a person young enough to live in the code, who can deal with either new tools or tools that fundamentally change every three months or so. New functionality is achieved by implementing a new plugin/gem which needs to be compared to all existing alternatives. No solutions is 100% complete because people lose interest first. If it doesn't work, get in there and change the code yourself. [note - I don't comment on Ruby itself as I don't have enough exposure to the non-Rails community]
Seaside - I'm an old-hand in Smalltalk, just learning Seaside. So far I see a fascination with having something that just frackin' works (though it might be heavily engineered). The documentation sucks though, so get into the code. Changes are generally transparent and backwardly compatible. If it doesn't work, you did something wrong - the community is the documentation.
Each of these cultures has strengths and weaknesses. I'm 46 years old (shit, when did that happen), my raw memory isn't as strong as it was, and I don't think as quickly as I used to. But I have a load of experience in developing design models and applications - I want tools that get out of my way and let me build things. Given my Smalltalk experience, Seaside is a great match for me - much better than Rails. If there was the equivalent of "Agile Web Development Using Rails" then it would rock!
In an age when everyone needs to use portfolio theory to manage their software investments, starting low-cost initiatives by the dozen then abandoning the ones that don't work out, Rails is a clear winner over Java. It lets me experiment at low cost, it lets me get something out-the-door quickly. In that environment Java is a set of leg-irons. But the Rails culture thrives when people have enough time and energy to deal with a firehose of changes and alternatives - as an entrepreneur a lot of this is simply waste. One solution might be to standardise on a particular configuration - that seems like a false.optimisation.
Java - a fascination with flexibility, generally achieved through complexity (and a healthy(?) does of XML). Ten years ago I would have said Java was focussed on solidity and reliability (in the meeting expectations sense), but I think this has changed over time. New functionality is implemented by adding new options. If it doesn't work, it's because you didn't configure it properly.
Rails - a fascination with bright, shiny new toys. Encourages the image of the developer as a person young enough to live in the code, who can deal with either new tools or tools that fundamentally change every three months or so. New functionality is achieved by implementing a new plugin/gem which needs to be compared to all existing alternatives. No solutions is 100% complete because people lose interest first. If it doesn't work, get in there and change the code yourself. [note - I don't comment on Ruby itself as I don't have enough exposure to the non-Rails community]
Seaside - I'm an old-hand in Smalltalk, just learning Seaside. So far I see a fascination with having something that just frackin' works (though it might be heavily engineered). The documentation sucks though, so get into the code. Changes are generally transparent and backwardly compatible. If it doesn't work, you did something wrong - the community is the documentation.
Each of these cultures has strengths and weaknesses. I'm 46 years old (shit, when did that happen), my raw memory isn't as strong as it was, and I don't think as quickly as I used to. But I have a load of experience in developing design models and applications - I want tools that get out of my way and let me build things. Given my Smalltalk experience, Seaside is a great match for me - much better than Rails. If there was the equivalent of "Agile Web Development Using Rails" then it would rock!
In an age when everyone needs to use portfolio theory to manage their software investments, starting low-cost initiatives by the dozen then abandoning the ones that don't work out, Rails is a clear winner over Java. It lets me experiment at low cost, it lets me get something out-the-door quickly. In that environment Java is a set of leg-irons. But the Rails culture thrives when people have enough time and energy to deal with a firehose of changes and alternatives - as an entrepreneur a lot of this is simply waste. One solution might be to standardise on a particular configuration - that seems like a false.optimisation.
Tuesday, July 14, 2009
What would you like in a code quality tool?
Although continuous integration has been a huge step forwards for the development community I think that over time we've tended to treat it as a rather large bucket that we could keep tossing things into. Sure, someone should immediately address that method that's too long, but should it really stop the build and the release of the next version if all the tests pass? My feeling is no, it shouldn't.
I think that the agile community could do more to encourage various parallel continuous processes that test different aspects of the application. In particular, we could separate the static analysis of the code quality from the behavioural analysis that we perform with tests, be they unit, integration or functional tests. If we separate the two aspects then we can look to improve them at different rates and in different dimensions. This separation is the motivation behind an idea that Marty Andrews and Simon Harris have been kicking around for a few years (Marty is the author of Roodi and Complexian, and Simon is the author of Simian, so they have a long-running interest in code quality). Cogent is now planning to turn this into a product - something that you can point towards your source code repository and simply unleash.
We don't have a name for the product yet (suggest one if you like), but it will run static analysis tools (like Checkstyle, Simian, Roodi, Reek, Flay) against each revision of your code base so that you can see trends over time, and it will produce warnings when the values of the metrics exceed certain thresholds. The product will be free for open source projects.
We'll start with a simple, web-based version, probably to run over Ruby projects in public Git repositories (because that's low-hanging fruit for us), but we're going to be very community driven. With this in mind we have a survey that will let you influence the development direction, even before we start. If a tool like this is something that interests you, please fill in our product survey. As a bonus, the first 500 people to complete the survey will get unlimited free access to the product in recognition of their contributions.
I think that the agile community could do more to encourage various parallel continuous processes that test different aspects of the application. In particular, we could separate the static analysis of the code quality from the behavioural analysis that we perform with tests, be they unit, integration or functional tests. If we separate the two aspects then we can look to improve them at different rates and in different dimensions. This separation is the motivation behind an idea that Marty Andrews and Simon Harris have been kicking around for a few years (Marty is the author of Roodi and Complexian, and Simon is the author of Simian, so they have a long-running interest in code quality). Cogent is now planning to turn this into a product - something that you can point towards your source code repository and simply unleash.
We don't have a name for the product yet (suggest one if you like), but it will run static analysis tools (like Checkstyle, Simian, Roodi, Reek, Flay) against each revision of your code base so that you can see trends over time, and it will produce warnings when the values of the metrics exceed certain thresholds. The product will be free for open source projects.
We'll start with a simple, web-based version, probably to run over Ruby projects in public Git repositories (because that's low-hanging fruit for us), but we're going to be very community driven. With this in mind we have a survey that will let you influence the development direction, even before we start. If a tool like this is something that interests you, please fill in our product survey. As a bonus, the first 500 people to complete the survey will get unlimited free access to the product in recognition of their contributions.
Cogent plays dress-ups
This is an email that I sent to everyone at Cogent Consulting this morning, reprinted here because I think it demonstrates the values that we're trying to pursue (and some other people seem to agree). The background is that we're having an offsite this weekend and one of the staff has asked everyone to 'dress nicely' because there will be a professional photographer to take some staff shots. Unlike most of our decisions, this wasn't telegraphed or discussed in advance. There was some resistance.
I broke my transparency rule yesterday, and let what was for me a considered decision look somewhat arbitrary and without explanation. So, why have professional photos of the staff?
First, we're going to be doing more and more work for people who don't know us personally, or even by reputation. The pool of work from people who already know us has shrunk, and the team has grown. Neither of those are bad things - from my perspective they're an inevitable part of time passing. What it also means to me is that we need to be a bit more careful/particular about how we present Cogent as a product/service.
Presenting Cogent has lots of aspects. Peter Styles from RedBubble was talking to some people last night while I was on the phone and pointed to me and remarked "these guys can be the Pivotal Labs of Australia". That assessment appeals to my ego, but it's also an interesting guide to projection, promotion and marketing. The Pivotal guys blog a lot, and they produce open source projects - we do some of each, but not as much as we could. I don't put enough emphasis on it myself.
Another aspect of presenting Cogent is making the website look "polished". It's hard to put my finger on what that really means, but we all have websites that give us a "wow" factor, even if we'd come up with quite different results if we listed those sites. It's not a single thing that makes the difference, but I think that one thing is having well-presented, reasonably consistent, photos of the staff. It literally stops the company from being a faceless entity. Hence the professional photographer on the weekend.
If you stop by the TW office you'll see that they make extensive use of real photos of real staff right in their reception - there's a wall with large scale images, not just snapshots. From memory Evan is reclining in a deck chair working on his laptop :-) I'm not proposing something like that right now, but I wouldn't rule it out in future either - I'm proud of the people in the company and I want them to be the face of the company. I don't want everything to be about 'senior management' as it is in lots of places. We may find other places where putting photos of our staff makes sense - for example, tender documents often list staff, and I think having small photos included in a list like that makes it more appealing (I have this strange idea that saying no to even a picture of a person's face is harder than saying no to a piece of text *s*).
Anyway, there's the rationale. If there are people who have personal objections to photos under any circumstances then of course I respect that. If the resistance is because it seems 'corporate' or corny then I'd like you to try to overcome that for just a little while, since I sincerely believe this is helpful.
Steve
I broke my transparency rule yesterday, and let what was for me a considered decision look somewhat arbitrary and without explanation. So, why have professional photos of the staff?
First, we're going to be doing more and more work for people who don't know us personally, or even by reputation. The pool of work from people who already know us has shrunk, and the team has grown. Neither of those are bad things - from my perspective they're an inevitable part of time passing. What it also means to me is that we need to be a bit more careful/particular about how we present Cogent as a product/service.
Presenting Cogent has lots of aspects. Peter Styles from RedBubble was talking to some people last night while I was on the phone and pointed to me and remarked "these guys can be the Pivotal Labs of Australia". That assessment appeals to my ego, but it's also an interesting guide to projection, promotion and marketing. The Pivotal guys blog a lot, and they produce open source projects - we do some of each, but not as much as we could. I don't put enough emphasis on it myself.
Another aspect of presenting Cogent is making the website look "polished". It's hard to put my finger on what that really means, but we all have websites that give us a "wow" factor, even if we'd come up with quite different results if we listed those sites. It's not a single thing that makes the difference, but I think that one thing is having well-presented, reasonably consistent, photos of the staff. It literally stops the company from being a faceless entity. Hence the professional photographer on the weekend.
If you stop by the TW office you'll see that they make extensive use of real photos of real staff right in their reception - there's a wall with large scale images, not just snapshots. From memory Evan is reclining in a deck chair working on his laptop :-) I'm not proposing something like that right now, but I wouldn't rule it out in future either - I'm proud of the people in the company and I want them to be the face of the company. I don't want everything to be about 'senior management' as it is in lots of places. We may find other places where putting photos of our staff makes sense - for example, tender documents often list staff, and I think having small photos included in a list like that makes it more appealing (I have this strange idea that saying no to even a picture of a person's face is harder than saying no to a piece of text *s*).
Anyway, there's the rationale. If there are people who have personal objections to photos under any circumstances then of course I respect that. If the resistance is because it seems 'corporate' or corny then I'd like you to try to overcome that for just a little while, since I sincerely believe this is helpful.
Steve
Friday, June 26, 2009
Emerging Agile UX Practices
I am not a UX professional, but I've interacted with a number of them over the last few years, in the context of agile development. The interaction hasn't always been smooth - the prevailing mood has been that agile development was asking UX professionals to do a bad job, but I've never felt that way myself. My feeling is that UX professionals and software developers have been on similar paths in regard to agile development, but the UX people are starting a few years later and have had less leadership so their challenge has been more difficult.
Why do I think the paths for both roles are similar? Because once-upon-a-time many software developers would have said that agile was asking them to do a bad job. After all, how could you do a good job with no upfront design, no long architecture phase, no detailed requirements? But what developers realised (either quickly or slowly, or perhaps not yet) was that agile values didn't mean doing a bad job, they meant doing a good job in a different way. Part of that was learning new skills and new ways to work. The software/programming community was lucky because they had early leadership in these skills and practices.
The feedback I've gotten in the past from UX folk was essentially "when we're in an agile project we can't work the way we work now", to which my answer was a resounding YES! You (as a profession) need to find new ways to work that produce good results faster. It's not your fault that those techniques don't exist yet, or that they may be hard to discover, but that doesn't invalidate the search. But I haven't done very well at explaining that.
However, practices for UX professionals on agile projects are emerging now, and Jeff Patton has done a good job of describing the practices that he sees working in this context. Moving to agile development means developing new approaches for everyone involved, and it's good that this is now a little easier for UX.
Why do I think the paths for both roles are similar? Because once-upon-a-time many software developers would have said that agile was asking them to do a bad job. After all, how could you do a good job with no upfront design, no long architecture phase, no detailed requirements? But what developers realised (either quickly or slowly, or perhaps not yet) was that agile values didn't mean doing a bad job, they meant doing a good job in a different way. Part of that was learning new skills and new ways to work. The software/programming community was lucky because they had early leadership in these skills and practices.
The feedback I've gotten in the past from UX folk was essentially "when we're in an agile project we can't work the way we work now", to which my answer was a resounding YES! You (as a profession) need to find new ways to work that produce good results faster. It's not your fault that those techniques don't exist yet, or that they may be hard to discover, but that doesn't invalidate the search. But I haven't done very well at explaining that.
However, practices for UX professionals on agile projects are emerging now, and Jeff Patton has done a good job of describing the practices that he sees working in this context. Moving to agile development means developing new approaches for everyone involved, and it's good that this is now a little easier for UX.
Saturday, June 20, 2009
How about the small-law firm model?
In reponse to my previous blog post Robert said...
It turns out that we have some of these elements at Cogent, but we didn't have that model in mind when we created them.
We do have two groups for allocating profit share. One is like "partners" - it's myself, Marty and Simon. 50% of the profit pool goes to the partners and the remaining 50% is split evenly over all the employees. We needed that arrangement to make it sensible for the partners to stop doing personal consulting and switch to a salary-plus-profit-share arrangement, and even then we're only just now at the point where the partners would nominally break even. It's also rather abstract because as long as we're pushing consulting profits into product development there aren't any cash profits to share anyway!
So far this is very similar to the small-law firm model. The difference comes when we consider what will happen over time. It seems to me that the small-law firm model is predicated on growth, or at least turn-over - on the expectation that when you get promoted to partner there will be a supply of associates to generate profits. I'm biased against models that encourage growth (though our existing model has that bias at the moment). Over time at Cogent I'm actually expecting to reduce the partners' percentage of the profit share pool - keeping it enough to keep them interested in staying at Cogent, but gradually making more and more of the profits available to the general employees.
There are flaws in both approaches, so it will be interesting to see how the profit share approach evolves at Cogent over time.
'I've always wondered if the classic small-law firm model would work with IT shops. You know - newcomers start as "associates" and get paid a salary (possibly with bonuses), but the ultimate goal is to get recognised as a partner, in which case you shift to profit sharing.
The associate-level allows the firm to recruit new talent at a fixed risk; the partner-level allows for the co-operative mode. As long as the ratio of partner-associate is good (the case in small law firms, not large) the competition between associates isn't likely to be unhealthy - particular if the promotion to partner is merit based rather than "we've got 1 slot open this year".'
It turns out that we have some of these elements at Cogent, but we didn't have that model in mind when we created them.
We do have two groups for allocating profit share. One is like "partners" - it's myself, Marty and Simon. 50% of the profit pool goes to the partners and the remaining 50% is split evenly over all the employees. We needed that arrangement to make it sensible for the partners to stop doing personal consulting and switch to a salary-plus-profit-share arrangement, and even then we're only just now at the point where the partners would nominally break even. It's also rather abstract because as long as we're pushing consulting profits into product development there aren't any cash profits to share anyway!
So far this is very similar to the small-law firm model. The difference comes when we consider what will happen over time. It seems to me that the small-law firm model is predicated on growth, or at least turn-over - on the expectation that when you get promoted to partner there will be a supply of associates to generate profits. I'm biased against models that encourage growth (though our existing model has that bias at the moment). Over time at Cogent I'm actually expecting to reduce the partners' percentage of the profit share pool - keeping it enough to keep them interested in staying at Cogent, but gradually making more and more of the profits available to the general employees.
There are flaws in both approaches, so it will be interesting to see how the profit share approach evolves at Cogent over time.
Thursday, June 18, 2009
Allocating rewards in co-operative environments
At Cogent Consulting we compensate people in two ways - we have healthy base salaries and we offer profit share that is not linked to performance or to salary. It's close to equal shares (there are some details that mean that's not true overall). We debated this arrangement when we created Cogent and I've discussed it with colleagues back as far as 2001, but it's based on our gut feel of what would minimise conflict and maximise co-operation. It avoids the debate about relative performance that bonus systems create, that are normally circumvented by secrecy (secrecy isn't an option at Cogent). So it was interesting to read this in No Contest: The Case Against Competition by Alfie Kohn:
Notice that this is also a profit-sharing scheme, not a bonus system tied to some particular behaviours. We hope that our behaviours are driven by our principles, and that long-term this will generate profits. We wouldn't want people to change their behaviours to maximise single year's profit. If we ever detected that we'd need to review the profit-sharing scheme.
"In recent years, Deutsch and his associates have investigated not only the way tasks are set up but the way rewards are distributed. Among the possibilities are a winner-take-all system (which is what many contests amount to), a distribution proportional to accomplishment, and an equal distribution. Much as we tend to assume that competing boosts performance, so it if often taken for granted that the first two arrangements provide a crucial incentive for working hard: reserving a desirable reward for the winner is thought to promote excellence. A series of six experiments with Columbia University students, involving tasks that ranged from decoding Japanese poetry to estimating the number of jellybeans in a jar, was devised to test this assumption. The results: When tasks could be performed independently - that is, when there was low means interdependence - the system of distributing rewards had no effect on how good a job they did. There was absolutely no evidence to suggest that people work more productively when rewards are tied to performance than when everyone gets the same reward. But for those tasks where success depends on working together, there was a clear difference. A system of equal rewards, Deutsch discovered, "gives the best results and the competitive winner-take-all system gives the poorest results".
Notice that this is also a profit-sharing scheme, not a bonus system tied to some particular behaviours. We hope that our behaviours are driven by our principles, and that long-term this will generate profits. We wouldn't want people to change their behaviours to maximise single year's profit. If we ever detected that we'd need to review the profit-sharing scheme.
Wednesday, June 17, 2009
Cogent moves forward
Cogent has made a couple of steps forward this week. We'll tell you what those steps are in a minute, but first a little bit of background. Cogent Consulting really started two years ago when Marty and Steve merged their consulting revenue into one pool - now we have 11 employees, and we're going to come close to $2m in sales for this financial year. We generate enough margin to have two people on product development, we've released Runway, and we're about to start on another product venture. We still keep all our financials open and we operate reasonably democratically. Things are tracking fairly well.
So here are our two big steps. Our 11th employee was a business development manager, Rachel Morley. Having a business development manager increases our overheads, but it also gives us a more active, structured approach to business development that lets us (and indeed encourages us to) support more employees. As a result, we're looking for people who would like to become Cogent employees.
Cogent isn't your typical employer. We're interested in the usual things - people who have deep technical knowledge, people who can represent us well on client sites, people who like to learn. You've heard all that before. We're quantitatively different in that regard, in that our standards are very high, but there are some things that are qualitatively different about Cogent. If you're looking for someone to tell you what to do, in detail, then we're not the place for you. We're much more likely to give you some rather vague goal and ask you to figure out how to achieve it - and a lot of the things you need to do won't involve programming. They may involve marketing, or copy writing, or business analysis, or something as mundane as going and buying some stationery.
We truly need you to be collaborative and collegial - we don't have any places for people who want to work alone or who don't want to subject their programming thoughts to critique. Bob Sutton's "The No Asshole Rule" captures our philosophy quite well. In exchange we'll be completely transparent to you - you'll know our clients, our rates, our revenue, our costs, and everyone's salaries. You'll have the chance to be involved in decisions that you probably don't even know happen at your current employer.
We need you to be passionate. While we pay good salaries, we need you to be the sort of person who would program for enjoyment even if you weren't getting paid for it. We need you to be highly ethical - to be more concerned with doing the right thing than the profitable thing if there's a conflict.
We're interested in programmers, web developers, business analysts and possibly testers. So if you're interested in undertaking a variety of consulting work, off-site development, and product development at a place like Cogent, please drop us an email.
So here are our two big steps. Our 11th employee was a business development manager, Rachel Morley. Having a business development manager increases our overheads, but it also gives us a more active, structured approach to business development that lets us (and indeed encourages us to) support more employees. As a result, we're looking for people who would like to become Cogent employees.
Cogent isn't your typical employer. We're interested in the usual things - people who have deep technical knowledge, people who can represent us well on client sites, people who like to learn. You've heard all that before. We're quantitatively different in that regard, in that our standards are very high, but there are some things that are qualitatively different about Cogent. If you're looking for someone to tell you what to do, in detail, then we're not the place for you. We're much more likely to give you some rather vague goal and ask you to figure out how to achieve it - and a lot of the things you need to do won't involve programming. They may involve marketing, or copy writing, or business analysis, or something as mundane as going and buying some stationery.
We truly need you to be collaborative and collegial - we don't have any places for people who want to work alone or who don't want to subject their programming thoughts to critique. Bob Sutton's "The No Asshole Rule" captures our philosophy quite well. In exchange we'll be completely transparent to you - you'll know our clients, our rates, our revenue, our costs, and everyone's salaries. You'll have the chance to be involved in decisions that you probably don't even know happen at your current employer.
We need you to be passionate. While we pay good salaries, we need you to be the sort of person who would program for enjoyment even if you weren't getting paid for it. We need you to be highly ethical - to be more concerned with doing the right thing than the profitable thing if there's a conflict.
We're interested in programmers, web developers, business analysts and possibly testers. So if you're interested in undertaking a variety of consulting work, off-site development, and product development at a place like Cogent, please drop us an email.
Sunday, June 14, 2009
Cogent as a business
Updated June 18
Along the Cogent Consulting journey we've documented a few different sets of values, principles and goals. Our manifesto was created quite early. Recently we've supplemented this with a simple list of values
- Openness
- Integrity
- Respect
- Craftsmanship
- Collaboration
and goals
- Develop non-consulting sources of income
- Build some kick-ass software
- Work with good people
- Have flexibility in lifestyle and the amount of time I want to work
We should probably rationalise all this (and I just added that task to Runway), but first I want to add some more philosophical statements that occurred to me after I read Small Giants by Bo Burlingham.
- We need to make a profit, otherwise we can't achieve our other goals. That's a constraint
- Doing the right thing for the customer is more important than maximising profit
- Our families are more important than our customers
- For everything we do we should ask not 'is it profitable', but 'is it the right thing to do?'.
- Right now, product development depends on consulting, and consulting depends on business development. We need to bear that in mind when we're setting priorities.
When I started to write I thought there would be more points, but combined with our existing manifesto that seems to say most of what I need to say.
Wednesday, June 10, 2009
Is it right?
After my JAOO Australia presentation on agile values a friend of mine referred me to Barry Schwartz's TED talk about Practical Wisdom and asked me if I thought there was any relationship between the agile values and what Schwartz was saying.
I think that philosophically there's a great deal of overlap. Among other great things, Schwartz says that over-reliance on rules reduces us to mediocrity. In "The Rise of the Chaordic Age" Dee Hock says "Complex rules and regulations give rise to simple and stupid behavior." Same message. Often these rules exist because we've "learned from the past" - because someone once made a mistake that we'd like to avoid in future we add an extra rule, and then another, and then another, until we are so constrained by the sludge of regulations that exceptional behaviour has moved beyond exceptional to impossible.
One of the things that the agile movement has said is that people inevitably make mistakes, but rather than eliminate the possibility of mistakes we should reduce the probability and the cost of the consequences. But if you don't trust people to do the right thing then this seems insufficient, and you start to add more rules, and you end up with the dogmatic flavours of agile - "you MUST do XXX or YYY". That's not the way to get the benefits that agile development promises.
Some people will say that adding rules is the wrong approach, and what you should do is create incentives instead, and let people establish specific behaviours based on those incentives. No stupid rules for these folks. But both Schwartz and Alfie Kohn (author of "Punished By Rewards") argue that any incentive scheme can be subverted by bad will, and that they send the message that you should "do the right thing" not simply because it is the right thing, but because you will be rewarded for doing it. Kohn in particular talks about the need to continually increase incentives and what happens if the incentive is removed.
So what should we do instead? Here Schwartz quotes Barack Obama - "We must ask, not just is it profitable, but is it right". We need to celebrate moral exemplars, to eulogise moral acts rather than financial acts, and encourage the development of moral will and moral skill. It may be faint to many, but I hear the echo of Kent Beck's values of transparency, accountability and responsibility.
I believe in all of this at a personal level. I believe it is consistent with the values of the agile community, as I understand them. And I'm very clear that these are the values of my employer, Cogent Consulting, otherwise I'd dissolve the company.
I think that philosophically there's a great deal of overlap. Among other great things, Schwartz says that over-reliance on rules reduces us to mediocrity. In "The Rise of the Chaordic Age" Dee Hock says "Complex rules and regulations give rise to simple and stupid behavior." Same message. Often these rules exist because we've "learned from the past" - because someone once made a mistake that we'd like to avoid in future we add an extra rule, and then another, and then another, until we are so constrained by the sludge of regulations that exceptional behaviour has moved beyond exceptional to impossible.
One of the things that the agile movement has said is that people inevitably make mistakes, but rather than eliminate the possibility of mistakes we should reduce the probability and the cost of the consequences. But if you don't trust people to do the right thing then this seems insufficient, and you start to add more rules, and you end up with the dogmatic flavours of agile - "you MUST do XXX or YYY". That's not the way to get the benefits that agile development promises.
Some people will say that adding rules is the wrong approach, and what you should do is create incentives instead, and let people establish specific behaviours based on those incentives. No stupid rules for these folks. But both Schwartz and Alfie Kohn (author of "Punished By Rewards") argue that any incentive scheme can be subverted by bad will, and that they send the message that you should "do the right thing" not simply because it is the right thing, but because you will be rewarded for doing it. Kohn in particular talks about the need to continually increase incentives and what happens if the incentive is removed.
So what should we do instead? Here Schwartz quotes Barack Obama - "We must ask, not just is it profitable, but is it right". We need to celebrate moral exemplars, to eulogise moral acts rather than financial acts, and encourage the development of moral will and moral skill. It may be faint to many, but I hear the echo of Kent Beck's values of transparency, accountability and responsibility.
I believe in all of this at a personal level. I believe it is consistent with the values of the agile community, as I understand them. And I'm very clear that these are the values of my employer, Cogent Consulting, otherwise I'd dissolve the company.
Tuesday, May 12, 2009
Cogent Values and Goals
Cogent recently had an off-site meeting that we used for a wide-ranging discussion, including a session on what we considered our values and goals as an organisation. We had a fairly spirited exchange on both these topics, constrained by finding at most five points in each area that captured our business. There were many things on the margins that people thought were important, so it was difficult to hone the lists to a concise consensus. In the end we had this:
Cogent Values
Cogent Goals
I was going to add some commentary, but on reflection I think these speak for themselves (but I'm interested in your comments).
For your interest, here are the next group of goals - things that influence us, but that aren't core and didn't make it into the definitive list:
So while we would like to change the world and have some fun, we're realistic enough to understand that for us, for the people we are and the stage of life that most of us are in, there are some selfish things (money and flexibility) that are actually more important. That doesn't really make me feel great about myself, but I think it's the right decision and I think it's good that we've been clear and articulated this.
I'd be interested to know the outcomes from any other small firms that do some exercise like this.
Cogent Values
- Openness
- Integrity
- Respect
- Craftsmanship
- Collaboration
Cogent Goals
- Make more money than we can from consulting
- Build some kick-ass software
- Work with good people
- Have flexibility in lifestyle and the amount of time I want to work
I was going to add some commentary, but on reflection I think these speak for themselves (but I'm interested in your comments).
For your interest, here are the next group of goals - things that influence us, but that aren't core and didn't make it into the definitive list:
- Prove that you can run a company humanely for both customers and employees
- Learn new things, learn proven things from experienced folk
- Grow as practitioners
- Change the way people develop software
- Inspire one another and others
- To have some fun
So while we would like to change the world and have some fun, we're realistic enough to understand that for us, for the people we are and the stage of life that most of us are in, there are some selfish things (money and flexibility) that are actually more important. That doesn't really make me feel great about myself, but I think it's the right decision and I think it's good that we've been clear and articulated this.
I'd be interested to know the outcomes from any other small firms that do some exercise like this.
Saturday, May 9, 2009
Runway
Cogent is very proud to announce the release of our very first product - Runway!
Runway is an action management system. It's inspired by Getting Things Done (GTD), and will eventually grow into a more complete GTD system. Why do you need an action management system? Because your brain is a terrible secretary?Right now you're wasting precious mental energy worrying about things you need to remember to do, which ones are important, which ones are urgent, what needs to be done next, and so on. Runway provides a trusted repository for all the things that you need to do - and trust me, there are hundreds of them.
Runway is designed to be quick and simple to use, and to become intuitive over time. An action is created through a simple, single-line entry field. No check boxes, no drop-down lists, no tabbing between fields. You can also create an action by sending an email to Runway, or by sending a Twitter message. We want to extend Runway to other sources, so that you have ubiquitous capture for your important ideas.
Runway is an action management system. It's inspired by Getting Things Done (GTD), and will eventually grow into a more complete GTD system. Why do you need an action management system? Because your brain is a terrible secretary?Right now you're wasting precious mental energy worrying about things you need to remember to do, which ones are important, which ones are urgent, what needs to be done next, and so on. Runway provides a trusted repository for all the things that you need to do - and trust me, there are hundreds of them.
Runway is designed to be quick and simple to use, and to become intuitive over time. An action is created through a simple, single-line entry field. No check boxes, no drop-down lists, no tabbing between fields. You can also create an action by sending an email to Runway, or by sending a Twitter message. We want to extend Runway to other sources, so that you have ubiquitous capture for your important ideas.
Actions appear in your to-do list right away, and can be prioritised easily with drag-and-drop. You can rapidly filter your actions by when they need to be done, the context you need to be able perform the action, the time required, the energy you need, and any other tags you want to use.
Most people will use Runway to track all their actions, select the actions for the day at the beginning of the day, and then work through that list. I've been using Runway that way for over six months.
I'm really excited by Runway, because it makes my life simpler and helps me get things done. It's also a great piece of software, but I hope that people forget about that, that Runway merges into the background and just becomes an invisible helper. Please visit http://www.runwayapp.com and give it a go. Perhaps you'll have the same experience as Jonathan, one of our beta testers, who says, "Runway has changed my life!"
Sunday, May 3, 2009
Cogent Customer Brief
We're not heavily into formal contracts at Cogent, since we think they fail to capture the essence of the relationship. We understand that everyone needs to know how much they're paying and for how long, and we've also had some unfortunate experiences where we didn't do a good job of communicating expectations to the customer, so we've put together a document that describes, in the plainest language we could muster, what to expect then you collaborate on a project with us. You can find it on this page of our website.
Wednesday, April 29, 2009
Partnerships
My prior post (about excess work) generated a private email suggesting that partnerships would be one way to deal with the problem - that Cogent could enter into an arrangement with one of the larger companies to pass on excess work in exchange for some fee. That would seem to tick most of the boxes - the work gets done by someone we would have referred the customer to anyway, and we get some financial return, so I should feel less like we're "giving something away".
Yet I have some visceral reservations that I need to try to rationalise. I think it's mostly about responsibility and care - if Cogent is going to take money for something then I think we need to stand behind the result, and my experience is that you can lose a lot of money (and reputation) from bailing out one bad gig. If all I say is "here are some folk that have historically done a good job, and we don't have any stake in whether you work with them or not" then I don't feel as much responsibility for what happens downstream.
I think that's the heart of it - thanks for listening/reading while I figured that out.
Yet I have some visceral reservations that I need to try to rationalise. I think it's mostly about responsibility and care - if Cogent is going to take money for something then I think we need to stand behind the result, and my experience is that you can lose a lot of money (and reputation) from bailing out one bad gig. If all I say is "here are some folk that have historically done a good job, and we don't have any stake in whether you work with them or not" then I don't feel as much responsibility for what happens downstream.
I think that's the heart of it - thanks for listening/reading while I figured that out.
What we do with excess business
Cogent doesn't do too much in the way of overt marketing and advertising. We spend some money on conference sponsorship, and about $30 per month on Google Adsense, and I think that's about it. Despite that, we get a steady stream of enquiries, and we generally end up with more work than we can handle - particularly if it's urgent work as we're often committed a few months in advance.
If a customer wants to start something right away, we do very little (if anything) to discourage them - instead, we pass along the contact details of the best people we know who can do the work and leave it to the customer to make any further arrangements. This is the simplest approach that can work, since it means that we don't get caught in the middle either managing communications or guaranteeing the quality of the work, and in principle the customer doesn't end up paying two margins so they should get the work done at a reasonable price. We're usually passing the work to companies that are "bigger" than us, and It's also worth saying that Cogent gets referrals in the same kind of way, mostly from independent contractors.
So by all rights I shouldn't be unhappy, but I have to admit to being irked by handing my competitors leads, especially when we don't get a lot of love in return. I understand it logically - if we had available resources, we wouldn't pass work to a competitor either, and these companies are much more likely than us to have available resources - but it still feels wrong or unfair on some level.
I'd be interested to hear how other small companies deal with similar situations.
If a customer wants to start something right away, we do very little (if anything) to discourage them - instead, we pass along the contact details of the best people we know who can do the work and leave it to the customer to make any further arrangements. This is the simplest approach that can work, since it means that we don't get caught in the middle either managing communications or guaranteeing the quality of the work, and in principle the customer doesn't end up paying two margins so they should get the work done at a reasonable price. We're usually passing the work to companies that are "bigger" than us, and It's also worth saying that Cogent gets referrals in the same kind of way, mostly from independent contractors.
So by all rights I shouldn't be unhappy, but I have to admit to being irked by handing my competitors leads, especially when we don't get a lot of love in return. I understand it logically - if we had available resources, we wouldn't pass work to a competitor either, and these companies are much more likely than us to have available resources - but it still feels wrong or unfair on some level.
I'd be interested to hear how other small companies deal with similar situations.
Tuesday, April 14, 2009
Failure is mandatory
A few days ago I had a conversation with Travis about failure. In some senses I'm a hard ass about failure - I think if you protect people from failure all the time you also protect them from learning anything. It's still important to protect people from potentially traumatic failure, but traumatic failure is the exception rather than the rule.
The context was how far our duty of care extends as consults, particularly in regard to the transfer of a solution to the customer. There's typically tension around this issue - the customer wants to maximise their investment in the solution itself, and in other projects as well, and that works against either documenting the solution or getting permanent staff involved. One position is that we should go out of our way to document/record knowledge of the solution for handover, even if the customer hasn't made that a priority. I think that's paternalistic (which is ironic given the example I'll use later). I think it's our responsibility to warn the customer that there's a handover risk, and to give some advice on how we'd handle that, but in the end it's up to the customer to decide how to invest their resources, and to wear the consequences of those decisions. I think it would be quite wrong of us to divert resources into documentation against the customers wishes. We might think the customer has made the wrong decision, but it's still their decision and they won't thank you for subverting it.
Another context that we talked about was when you see someone else about to do the "wrong thing". You should definitely draw attention to the problem as you see it, but if the person goes ahead anyway, should you do anything to protect the project from the outcome? My position is that you should not - you should let the project suffer the consequences (again excepting traumatic outcomes). It's possible that your fears won't be realised, in which case you'll learn something. If they are realised, the project team will learn something (including perhaps to pay more attention to your advice) and may be able to avoid similar problems in future. If you just fix it, then what the project team will learn is that there are no negative outcomes from that behaviour, so they'll quite happily repeat, and you'll have the same personal situation later. Ad infinitum.
The same kinds of things apply to my son. I try to protect him from traumatic outcomes, but sometimes I need to let him get hurt (physically or emotionally) so that he learns not to repeat a behaviour.
The context was how far our duty of care extends as consults, particularly in regard to the transfer of a solution to the customer. There's typically tension around this issue - the customer wants to maximise their investment in the solution itself, and in other projects as well, and that works against either documenting the solution or getting permanent staff involved. One position is that we should go out of our way to document/record knowledge of the solution for handover, even if the customer hasn't made that a priority. I think that's paternalistic (which is ironic given the example I'll use later). I think it's our responsibility to warn the customer that there's a handover risk, and to give some advice on how we'd handle that, but in the end it's up to the customer to decide how to invest their resources, and to wear the consequences of those decisions. I think it would be quite wrong of us to divert resources into documentation against the customers wishes. We might think the customer has made the wrong decision, but it's still their decision and they won't thank you for subverting it.
Another context that we talked about was when you see someone else about to do the "wrong thing". You should definitely draw attention to the problem as you see it, but if the person goes ahead anyway, should you do anything to protect the project from the outcome? My position is that you should not - you should let the project suffer the consequences (again excepting traumatic outcomes). It's possible that your fears won't be realised, in which case you'll learn something. If they are realised, the project team will learn something (including perhaps to pay more attention to your advice) and may be able to avoid similar problems in future. If you just fix it, then what the project team will learn is that there are no negative outcomes from that behaviour, so they'll quite happily repeat, and you'll have the same personal situation later. Ad infinitum.
The same kinds of things apply to my son. I try to protect him from traumatic outcomes, but sometimes I need to let him get hurt (physically or emotionally) so that he learns not to repeat a behaviour.
Monday, April 13, 2009
Cogent is talking about open rates
Some of you will already know this, but Cogent is an ongoing social experiment along the lines of Thoughtworks, but with different axioms. Transparency is one of our axioms, but historically we've applied this internally and the public information about Cogent has been much the same as what you'd know publicly about any other consulting company. We're about to step that up a notch and publish our rates schedule publicly.
As usual, we see pros and cons to this decision. First thing is that no one else seems to do it, so perhaps we're just being stupidly naive and we'll discover the reasons why no one else does it in some catastrophic way.
More obvious to us that we lose one of our supply-and-demand levers. Our supply (our employees) is fixed in the short term, and it's always been hard for us to figure out what the "real" demand for our services is (we get plenty of unsolicited contacts that don't turn into any business). One of the levers we can use to affect demand is price - if we had lots of people doing non-billable work then we'd clearly think about dropping prices to get people onto billable work. Historically this would have been transparent to customers, unless they talked to one another, but with public pricing it will be clear if we offer discounts.
We may also find that we get less contacts because people look at our prices and decide that we're too expensive without even talking to us. It's not clear if that's good or bad - we may actually save time because we don't waste time doing estimates for work that's never going to proceed anyway. Or we may miss out on legitimate business.
And finally, our permanent colleagues on consulting gigs will now have a pretty good idea of what the consultant sitting next to them is being charged out at. That may or may not prove embarrassing (hoprefully not).
So why do it? We have some feedback that clients like simple, public rates schedules. It means less discussion internally about "how much as we going to charge this client?". And it means extending the reach of our core principles, and each time we've done that in the past it's been good - if nothing else it tells the employees that we "walk the walk" consistently.
We've split the rates along two dimensions - the type of work and where the work happens. We do two types of work - agile coaching from senior staff, which involves some hands-on development but is focussed on influencing the entire team rather than producing features directly, and development work, which is mostly focussed on delivering features but also on influencing the entire team to work more effectively. We have different rates for agile coaching and development work, and we also charge less for development work that's performed at our own premises. There are three reasons for charging less for working on our own premises - i) we aren't able to influence the client staff as well, so we deliver less value; ii) we prefer working on our own premises and price our services to encourage this; iii) when we're working on our own premises we're effectively competing against teams all over the world and we need to reduce prices accordingly.
So here are our rates for new work in 2009, all exclusive of GST:
Agile Coaching
Development (consulting / offsite)
As usual, we see pros and cons to this decision. First thing is that no one else seems to do it, so perhaps we're just being stupidly naive and we'll discover the reasons why no one else does it in some catastrophic way.
More obvious to us that we lose one of our supply-and-demand levers. Our supply (our employees) is fixed in the short term, and it's always been hard for us to figure out what the "real" demand for our services is (we get plenty of unsolicited contacts that don't turn into any business). One of the levers we can use to affect demand is price - if we had lots of people doing non-billable work then we'd clearly think about dropping prices to get people onto billable work. Historically this would have been transparent to customers, unless they talked to one another, but with public pricing it will be clear if we offer discounts.
We may also find that we get less contacts because people look at our prices and decide that we're too expensive without even talking to us. It's not clear if that's good or bad - we may actually save time because we don't waste time doing estimates for work that's never going to proceed anyway. Or we may miss out on legitimate business.
And finally, our permanent colleagues on consulting gigs will now have a pretty good idea of what the consultant sitting next to them is being charged out at. That may or may not prove embarrassing (hoprefully not).
So why do it? We have some feedback that clients like simple, public rates schedules. It means less discussion internally about "how much as we going to charge this client?". And it means extending the reach of our core principles, and each time we've done that in the past it's been good - if nothing else it tells the employees that we "walk the walk" consistently.
We've split the rates along two dimensions - the type of work and where the work happens. We do two types of work - agile coaching from senior staff, which involves some hands-on development but is focussed on influencing the entire team rather than producing features directly, and development work, which is mostly focussed on delivering features but also on influencing the entire team to work more effectively. We have different rates for agile coaching and development work, and we also charge less for development work that's performed at our own premises. There are three reasons for charging less for working on our own premises - i) we aren't able to influence the client staff as well, so we deliver less value; ii) we prefer working on our own premises and price our services to encourage this; iii) when we're working on our own premises we're effectively competing against teams all over the world and we need to reduce prices accordingly.
So here are our rates for new work in 2009, all exclusive of GST:
Agile Coaching
- Senior Agile Coach: $1400
- Iteration Manager: $1300
Development (consulting / offsite)
- Tech Lead/Architect: $1200 / $1000
- Senior Developer: $1100 / $ 900
- Developer: $1000 / $ 800
We're interested in what people think of this approach to pricing, so please let us know.
Tuesday, April 7, 2009
Post-agile, or just agile done well?
Ivar Jacobsen was in Melbourne recently, and did a presentation in conjunction with the ACS saying that development needed to "get smarter". Throughout the presentation he compared Smart development to Agile - the trouble was I couldn't see any difference.
For a few years now I've heard of "post-agilism", which was needed to overcome the woes of agile development - the trouble is I couldn't see any suggestions that weren't compatible with agile as it was described around the time of the Agile Manifesto.
Don't get me wrong - I don't think Agile is the answer to every software development problem. I strongly agree with Alistair Cockburn that the development method you use needs to depend *at least* on the degree of risk and the size of the team. However, I don't think that we need to find new methods just because Agile can be done badly. Maybe we should pause and work a little harder at doing things well before we move on to doing new things.
For a few years now I've heard of "post-agilism", which was needed to overcome the woes of agile development - the trouble is I couldn't see any suggestions that weren't compatible with agile as it was described around the time of the Agile Manifesto.
Don't get me wrong - I don't think Agile is the answer to every software development problem. I strongly agree with Alistair Cockburn that the development method you use needs to depend *at least* on the degree of risk and the size of the team. However, I don't think that we need to find new methods just because Agile can be done badly. Maybe we should pause and work a little harder at doing things well before we move on to doing new things.
Monday, April 6, 2009
Experience - who gets the benefit of the doubt?
Bob Martin seems to have created a tempest with his apprenticeship model proposal, but it's worth remembering that it's not a new idea. There was at least one OOPSLA workshop on the Software Studio concept last century, and Ken Auer has implemented a related model at Role Model Software. I think Bob's raising the idea because he's had enough of crappy development and he wants to provoke some thoughtful responses (though that's just my impression). His article also triggered some related thoughts on how experience is treated in our industry.
One response to Bob's suggestion was that hierarchy somehow mistreated or devalued the more junior staff. It's quite true that this is one possible outcome, but the converse is also true. In non-hierarchical teams it's possible for the senior staff to be devalued! You can bring this into focus by thinking about a team of 2, one junior and one senior, and then asking yourself who should get the benefit of the doubt in a deadlock.
My experience is that many "juniors" think that they should get the benefit of the doubt. I've seen this attitude start immediately after graduation. I don't think that's right, or fair (but of course I'm arguing from an old guy's perspective). I've seen someone get upset because they didn't have carte blanche to replace existing frameworks, even though the changes would affect 30 people. I've seen someone simply defy the team conventions and go ahead and do it their preferred way, even after a team-wide discussion of the alternatives.
Everyone needs to understand that part of being a software developer is learning to *convince* people that your way is right - if other people don't understand, then you need to take ownership of the failure and find better ways to convince people. There is always going to be an "incumbent" position - sometimes incumbency is just the way it's done, and sometime incumbency is the older, more experienced person who perhaps can't articulate their position either. Someone needs to get the benefit of the doubt, and my leaning is towards experience.
I can't comment directly on the quality of very recent graduates, because I've stopped advising companies to hire them. My gut feel is that graduates come out with a good foundation for a career in software development, but still have a lot to learn. Sadly (for them) the salary for a graduate seems to be about $50K, and I can get people with 10-15 years of experience for just over $100K, and I'll take the more experienced person every time. I don't believe (as some people have implied) that mastery starts at 4 years and everything past that is wasted.
One response to Bob's suggestion was that hierarchy somehow mistreated or devalued the more junior staff. It's quite true that this is one possible outcome, but the converse is also true. In non-hierarchical teams it's possible for the senior staff to be devalued! You can bring this into focus by thinking about a team of 2, one junior and one senior, and then asking yourself who should get the benefit of the doubt in a deadlock.
My experience is that many "juniors" think that they should get the benefit of the doubt. I've seen this attitude start immediately after graduation. I don't think that's right, or fair (but of course I'm arguing from an old guy's perspective). I've seen someone get upset because they didn't have carte blanche to replace existing frameworks, even though the changes would affect 30 people. I've seen someone simply defy the team conventions and go ahead and do it their preferred way, even after a team-wide discussion of the alternatives.
Everyone needs to understand that part of being a software developer is learning to *convince* people that your way is right - if other people don't understand, then you need to take ownership of the failure and find better ways to convince people. There is always going to be an "incumbent" position - sometimes incumbency is just the way it's done, and sometime incumbency is the older, more experienced person who perhaps can't articulate their position either. Someone needs to get the benefit of the doubt, and my leaning is towards experience.
I can't comment directly on the quality of very recent graduates, because I've stopped advising companies to hire them. My gut feel is that graduates come out with a good foundation for a career in software development, but still have a lot to learn. Sadly (for them) the salary for a graduate seems to be about $50K, and I can get people with 10-15 years of experience for just over $100K, and I'll take the more experienced person every time. I don't believe (as some people have implied) that mastery starts at 4 years and everything past that is wasted.
Wednesday, January 14, 2009
The cloud...it's full of *people*!
This is the story of the development of a small piece of software to deploy to the cloud, developed by a cloud of people, and how nominally unrelated groups interacted along the way.
I'm interested in having a way to easily create new Rails servers in the cloud (specifically AWS) and to deploy my apps to them. I'm familiar with the bits and pieces that are involved but I hadn't stitched them together. I started to write a Rails app to manage my AWS assets, partially as a path to learning extjs, but when the AWS Console came out I ditched that and started back on the original problem. Here are the different bits that I planned to use, and how I became aware of them:
The last piece to this puzzle is a web site called oDesk (http://www.odesk.com/). I was talking to a client last week and she mentioned that she thought Australian Rails developers were the most expensive in the world. When I asked her what she was doing she said that she was using off-shore Rails developers and was finding good people who would work for US$20/hr, from India but also from the USA. It seemed like the recession in the USA was having interesting side affects. I was definitely curious how using oDesk would work from a client perspective.
So I had this small problem that I didn't have time to work on, and my curiousity about oDesk, and it seemed natural to solve them together. I posted my job on oDesk - produce a script that, given an EC2 instance and an ESB volume, would automatically configure the instance to use Rails with Passenger and Postgres, storing the data on the EBS volume. In addition, provide a test application so I can verify the Capistrano deployment to the instance. I posted this as a fixed price job, and my plan was to use a few different people and compare the outcomes. I posted my job last Sunday (it's currently Thursday).
I had my first response in a few hours, from within Australia! Twenty four hours didn't bring any more responses, so I pushed my job in front of a number of Rails developers who had high ratings. Many declined as they were too busy, some thought the suggested price was too low given the need to learn about AWS, and some still haven't responded. I got two more positive responses, one from an experienced developer in St. Petersburg and one from a less experienced developer on the east coast of the USA. The bids were $333.33 (Australia), $250.00 (Russia), and $166.67 (USA). The expected development times were 1 day, 1 day and 1 week.
I accepted the responses and nominated a starting date of Tuesday. On Wednesday I got my solutions from Australia and St Petersburg. I've tested the Russian solution and it works fine. There seem to be couple of issues in the Australian solution but the developer
is being very responsive, and the USA solution isn't due yet.
What's more interesting is what else happened in the 24 hours of development. The Russian developer forked passenger-stack on GitHub and hosted the fork as a public repository on GitHub (which was fine by me). However, I found out about this through an email from Marcus Crafter, who knew I was looking for a Postgres solution and pointed the fork out to me! Then I saw these tweets from Ben Schwarz (@benschwarz) "@kouky Just so you know, someone forked passenger-stack overnight rolling postgres support in, I'll try and pull everything together :)" and "@atnan @kouky, I've just blindly merged the PostgreSQL stuff as I'm up in the hills. I'll check it all out tomorrow :)" The status message for passenger-stack on GitHub now says "Adding support for PostgreSQL. Sprinkle will prompt the user to select MySQL/PostgreSQL server and the relevant Ruby database drivers."
So it looks like my little experiment ended up sponsoring the change that I wanted in the core implementation of passenger-stack. I suppose I could have asked Ben to do it originally, but that seemed like an imposition and I would have only achieved one of my goals. Now I have a script to configure my EC2 instances automatically, a better feeling for oDesk and off-shore development, and I've discovered that you can always sponsor the features that you want in a piece of open source software. Well worth my costs :-)
I'm interested in having a way to easily create new Rails servers in the cloud (specifically AWS) and to deploy my apps to them. I'm familiar with the bits and pieces that are involved but I hadn't stitched them together. I started to write a Rails app to manage my AWS assets, partially as a path to learning extjs, but when the AWS Console came out I ditched that and started back on the original problem. Here are the different bits that I planned to use, and how I became aware of them:
- Sprinkle, by Marcus Crafter (http://github.com/crafterm/sprinkle/tree/master) - Sprinkle handles deployment of resources on Ubuntu boxes in a pretty easy-to-understand way. Unfortunately I have a strong preference for Postgres over MySql (blame Simon Harris) and the default Sprinkle deployment recipes all use MySql. Still, it didn't take me long to make some extensions to install Postgres instead of MySql.
- Passenger-Stack, by Ben Schwarz (http://github.com/benschwarz/passenger-stack/tree/master) - Sprinkles for Apache, Passenger, Mysql, Memcached & Git. I found out about this via a Ryan Allen tweet.
- Amazon Web Services (http://aws.amazon.com/) - well known, but the fairly recent addition of Elastic Block Storage (EBS) gives a way to easily create persistent data that outlives a given instance, say for a Postgres database.
The last piece to this puzzle is a web site called oDesk (http://www.odesk.com/). I was talking to a client last week and she mentioned that she thought Australian Rails developers were the most expensive in the world. When I asked her what she was doing she said that she was using off-shore Rails developers and was finding good people who would work for US$20/hr, from India but also from the USA. It seemed like the recession in the USA was having interesting side affects. I was definitely curious how using oDesk would work from a client perspective.
So I had this small problem that I didn't have time to work on, and my curiousity about oDesk, and it seemed natural to solve them together. I posted my job on oDesk - produce a script that, given an EC2 instance and an ESB volume, would automatically configure the instance to use Rails with Passenger and Postgres, storing the data on the EBS volume. In addition, provide a test application so I can verify the Capistrano deployment to the instance. I posted this as a fixed price job, and my plan was to use a few different people and compare the outcomes. I posted my job last Sunday (it's currently Thursday).
I had my first response in a few hours, from within Australia! Twenty four hours didn't bring any more responses, so I pushed my job in front of a number of Rails developers who had high ratings. Many declined as they were too busy, some thought the suggested price was too low given the need to learn about AWS, and some still haven't responded. I got two more positive responses, one from an experienced developer in St. Petersburg and one from a less experienced developer on the east coast of the USA. The bids were $333.33 (Australia), $250.00 (Russia), and $166.67 (USA). The expected development times were 1 day, 1 day and 1 week.
I accepted the responses and nominated a starting date of Tuesday. On Wednesday I got my solutions from Australia and St Petersburg. I've tested the Russian solution and it works fine. There seem to be couple of issues in the Australian solution but the developer
is being very responsive, and the USA solution isn't due yet.
What's more interesting is what else happened in the 24 hours of development. The Russian developer forked passenger-stack on GitHub and hosted the fork as a public repository on GitHub (which was fine by me). However, I found out about this through an email from Marcus Crafter, who knew I was looking for a Postgres solution and pointed the fork out to me! Then I saw these tweets from Ben Schwarz (@benschwarz) "@kouky Just so you know, someone forked passenger-stack overnight rolling postgres support in, I'll try and pull everything together :)" and "@atnan @kouky, I've just blindly merged the PostgreSQL stuff as I'm up in the hills. I'll check it all out tomorrow :)" The status message for passenger-stack on GitHub now says "Adding support for PostgreSQL. Sprinkle will prompt the user to select MySQL/PostgreSQL server and the relevant Ruby database drivers."
So it looks like my little experiment ended up sponsoring the change that I wanted in the core implementation of passenger-stack. I suppose I could have asked Ben to do it originally, but that seemed like an imposition and I would have only achieved one of my goals. Now I have a script to configure my EC2 instances automatically, a better feeling for oDesk and off-shore development, and I've discovered that you can always sponsor the features that you want in a piece of open source software. Well worth my costs :-)
Subscribe to:
Posts (Atom)
Posts
