The end of passion

I'm supposed to be doing my US tax return right now, but I got distracted (what a surprise) and read this post from March 2009 by James Shore. At the end of the post James says two things that resonated strongly with me:

"If the point is to Be Agile, there's no need for Agile to actually work."

and

"Starting now, I'm reorienting my business to focus on people who want to be great."

Last night I said goodbye to me colleagues at Cogent Consulting, and one of the things that I meant to say but forgot was that for me interesting agile consulting work in Melbourne is now rare. I would have struggled to say why until I read James' post - I think it's that once-upon-a-time my consulting work was mostly with people who really wanted to be great, regardless of their current performance. It might not have been everyone in an organisation, but in each organisation it was enough people that greatness was the dominant theme. At some point those gigs dwindled - on the new gigs some people may have wanted to be great, but the team/division/company as a whole wasn't really interested. They pay lip-service to greatness, but it's the "we want to be great so long as we don't need to change too much of what we do" variety. My drive is to help people be great, and as that work has withered so has my passion. I may be willing to do work I'm passionate about for peanuts, but I think there's a thread in my mind which says if you can't give me work I'm passionate about then show me the money (and I think that makes me, well, human).

Now my soon-to-be-colleagues in Goldman Sachs may read this and shake their heads, so I need to be clear. I've always felt that enough of the people at GS wanted to be great to make working there worthwhile. It would be naive to think that it's everyone, everyday, but in my experience it's definitely enough. So I hope to have the pleasure of having my passion rekindle there as well.

I think this reinforces Cogent's recent decision on medium-term focus as well. The future of Cogent Consulting isn't to help other people with their agile transitions. The future is to blend Cogent's passion for software development with the needs of people who are passionate about their own domain, outside of software development. The result will be delivering excellent solutions that change people's lives - sometimes because no one else could do it economically, sometimes because no one else could build the partnerships that Cogent could build, and sometimes because no one else could do it at all.

Helping people transition to agile development - 10 years

Forming my own consulting company - 2.5 years

Finding outlets for my passion - a lifetime

Seven Norms of Highly Powerful Software Teams

I found this, a handwritten copy no less, while cleaning out a filing cabinet tonight. At the bottom I've written "Charlie Alfred, Education Services Manager, Object Design Inc, Object Magazine, May 1994". Apparently I've been carrying this around for almost 16 years. Since Google didn't find the article, I reproduce here for your pleasure (and so I can find it again myself).

Affinity for win/win

We strive to create an environment where the personal aspirations of each team member contribute to the goals of the team. We each are personally committed to achieving the goals of the team without sacrificing our lives outside work. We firmly believe that on the strongest teams the goals of the team agree with the goals of its members, rather than supersede them.

Common Mission

We share a common mission that is challenging, worthwhile, and clearly understood by all. This mission may be perceived by others as impossible or not worth the sacrifice. In this way, the mission helps us to identify our partners. Also, the more impossible the mission seems, the more we are forced to work together if we hope to succeed.

Quality

Quality is at the core of everything we do. If something is worth doing at all, it is worth doing right the first time.

Understanding Requirements

We strive to fully understand the requirements of the systems we plan to construct, including being acutely aware of how those requirements evolve over time. Furthermore, we actively validate our understanding of these requirements with the system's intended beneficiaries.

Interdependence

Individually, we each contribute strengths that counterbalance each other's weaknesses. Collectively we possess all of the skills, attitudes, and habits necessary for team success. Wherever an essential skill, attitude or habit is missing (or lacking), one or more of us makes it a priority to develop it, or we seek to add a new member who possesses it.

Receptivity

We are open to sources of new ideas and new ways of doing things, and use the norm to resolve conflict. We listen to each other's points of views, aware that our commitment to the same mission means that differences of opinion identify alternate paths to the same goal rather than competing paths to different goals. Also, we continually learn about and apply the latest paradigms, processes, and products in our technical, organisational and interpersonal activities.

Efficiency

Around here we work smarter, not necessarily harder. We sprint all out toward our goals when it is necessary to do so, but also know how to relax and rejuvenate. We make the 80/20 rule work for us. We take no shortcuts and cut no corners on the 20% of our tasks that produce 80% of our results. If any sacrifices must be made (in the interest of time), they always fall in the 80% of our activities that produce 20% of our results.

Thanks Charlie!

Why Post-Agilism isn't

I have a strong dislike for the term "post-agilism" and it will take somewhat more than 140 characters to explain why. Sorry twitter.

Post-agilism is a strawman argument. It postulates that agilism is zealous, doctrinaire and narrow-minded, and then posits that because of these deficiencies we need something that is "post-agilist". Implicit in this is that people who call themselves "agilists" have got it wrong.

This is simply another turn in a vicious circle. The "post-agilists" solution is a restatement of agile principles. Agile didn't start with hype, it didn't start with zealotry. On the contrary, Agile by declaration is adaptive and focussed on getting working software delivered. If you say you're doing agile but you don't believe in these principles then I'm sorry folks, you're just wrong. You probably weren't there at the beginning, and you've been misinformed, quite possibly by people who cherry-picked things out of context. We don't need another revolution, we need help with the swinging pendulum of the current phase.

So for the "post-agilists" out there - change your message. Say "wow, lots of people are doing agile wrong, we need to stop that, let's do agile right". Otherwise in 10 more years we'll have post-post-agilists decrying the shortcomings of how people have interpreted post-agilism (though more likely because of a lack of any structure than from too much).

Waste that we watch for

• Partially Done Work
  
• Relearning
  
• Hand-offs
  
• Task Switching
  
• Delays
  
• Defects
  
• Extra Features
  

Are we there (agile) yet?

I get asked this a lot, and let me say that I personally don't care very much. I do care whether or not the development process is effective or not, and mostly you know the answer to that before you ask me. But if I needed to form an independent opinion here are some questions that I'd ask you about feedback cycles. The underlying assumption is that a team will almost continually make mistakes (that vary dramatically in size) and that a lot of our practices and processes are about exposing those mistakes as quickly as possible. So how long is it between the these mistakes and the practice/process/activity that will expose that mistake, and how do you make that period smaller? These questions aren't presented in any particular order.

• A feature doesn't add any value


• Our code is unmaintainable


• We have duplicate code (or other quality problems)


• A developer introduces a new defect


• We've broken an existing feature


• What we're building doesn't match what was expected


• A feature is costing too much to be worth implementing


• A feature is unusable


• A feature works in isolation but not when integrated with other features


• The entire project is costing too much


• The project is going to cost too much

In addition, how long from a the time a feature is requested to the time that it hits production?

There are certainly other things that can go wrong with a project, but I think that reducing feedback cycle duration (time between event and feedback on the event) requires applying many agile principles and practices.

Should we lower quality to go faster?

Here's an email from one of our consultants, and my response (edited for anonymity and to removing the internal cursing!).

---

So, our product manager asked me today whether we could (just) let quality slide for a while, in the interests of getting stuff done faster.

I gibbered and muttered for a bit, and then gave an answer which amounted to "I don't know how to do that". Which is true; having personally experienced the benefits of practices like pairing, automated tests, and refactoring, and having done things that way exclusively for a quite a while, I'm not sure I know HOW to do it the old way.

I'm not sure that the his concern has gone away, though. He describes his experience as "mostly startup-like stuff", throwing stuff together quickly, and potentially throwing in away a few short years later. He's certainly never worked in an agile environment before, and is doubtful about the value of some of our practices.

Any advice?

---

As with most companies with a history of software development, there are people who have been burned by non-agile development, and people who haven't had that experience, and they have very different perceptions.

Here are some principles:

1) The customer is in control of the quality sliders for a project. We (IT, not just Cogent) are a service provider who give advice and recommendations, and then act on the instructions/direction of the business.

2) If the customer asks us to do something we (Cogent) fundamentally disagree with, we tell them that we are not the right group to provide that service and we quietly, respectfully disengage. Think of a surgeon who was told to perform operations without sterilisation, and that someone else would deal with the infection later.

3) There certainly are times when "not doing things the right way" is appropriate. I wrote the first cuts of Cogent Times with no test cases, because I didn't know if it would get any traction (i.e. be useful enough for people to use it). I don't regret it, and I would do it that way again. Now I'm adding test cases incrementally. When Runway was only developed by Simon it didn't have tests either - we demonstrated that this didn't scale. Kent's blog on the Flight of a Startup describes this well.

4) I think it's important to understand timeframes though. I believe that letting (internal) quality slide for a while will get stuff done faster for a few weeks. Other people seem to think that it will let them go faster for months or years. I disagree. Put simply, dropping quality won't get things done faster. We have the courage of our convictions here - we use these practices when we spend our own money on our own applications, when we really, truly, want to go as fast as we can.

5) The impact of lower (internal) quality is exacerbated by:
number of people working on the application
quality of the people working on the application
size of the application
level of understanding of the problem domain
level of change in requirements
flexibility of the technology stack
tolerance for defects in production

If your application is being built in a modern language, by one skilled person who is familiar with the product domain, working with people who know what they want and can describe it in detail without hesitation, false starts or iterations, and no one loses any money (or lives) if there are production defects, then what we do is definitely overkill. I've yet to see an environment that matches this description.

Having said all that, in the bigger picture the product manager isn't necessarily the person who makes the decisions on where the quality sliders are. Those are frequently decisions that need to be made at a higher level as they affect the total life cycle cost of the application, and the product manager is only responsible for the development part. This is the classic breakdown mode of process change - the people who bear the cost of the process change are not the people who reap the benefits of the product change. Producing a stable, defect free application that can be extended isn't necessarily in the product manager's interest, though it may very well be in the company's overall interest, especially when funding is allocated per project rather than per product. I present that argument mostly for completeness, since I think that the payback period of our practices is short enough that they're in the product manager's interest as well.

Things get even worse when the organisation funds work as "projects" rather than investing in "products". In these situations the value of feedback is reduced (since success is judged on the delivery of the promised features) and the focus is on the short term (the delivery phase for those features).

However it's also fair for the customer (and I mean the big-picture customer, not just the product manager) to suggest experiments - a good question in those situations is what will be measured to determine if things are better, and better for who? You need all the stakeholders involved in these discussions, not just the product owners.

Also bear in mind that we are not the only people to say these kinds of things. I believe part of what you're seeing is descent into the "easiest" possible implementation of Scrum. Here's what they're implicitly asking for:

"To be able to change their mind on requirements each iteration, and to obtain the benefits of this flexibility, without either performing enough up-front analysis to understand the fundamentals of the problem domain, or making any investment in the technical robustness of the application to support change"

This is very appealing from a business perspective, but it's an illusion. Jeff Sutherland has said he's never seen a hyper-productive Scrum team that didn't use the XP technical practices. Martin Fowler has written on what he calls Flaccid Scrum. The trend is to no longer consider the practices you mention as "agile", but simply "contemporary" - things you should be doing without a second thought.

So my first response to these kinds of suggestions (actually, my second, after I stop cursing) is that if the proposals would make things go faster then we'd happily implement them. But they won't. If I'm challenged further then I would ask them to find someone else on the technical staff who believe that the proposals will make things go faster, and then I'd step aside for them.

Seriously, hospital administrators don't tell surgeons how to maintain hygiene and where to cut. The customer can do whatever they want - we don't have to do it for them.

Not sure if that helps or not.

Steve

Roles in a self-organising team

One of the things that I'm asked to do fairly regularly is define the roles for an agile team, be it for Scrum, XP or some other flavour of agile. I usually end up doing this based on required skills, but my first impulse is to refuse, because I don't believe that a self-organising team actually has roles and I think that defined roles limit both the individual and the team. Let me examine that point before I explain how I do go about defining roles and why.

There is one role on a self-organising team - team member. The responsibility of that role is to bring all their skills to bear on the team's problems and organise their contributions to maximise the team's effectiveness.

When something needs to be done on an agile team everyone on the team should be able to see it, and in many cases it will be obvious which individual or individuals are the best suited to getting it done. The simplest approach is to allocate tasks based on skills, but the team might decide to use other criteria as well - they may need to grow their capacity in some area and decide to give the task to someone with less experience to give that person some practice; they may have limited capacity and be forced to give the task to someone who needs to learn the required skill, usually with mentoring from someone on the team. There will also be team-specific and project-specific reasons for not allocating tasks based simply on skill set. The important thing is that the reasons are well-articulated, and that the team member accepts responsibility for the task rather than having it arbitrarily assigned to them.

Although multi-skilled teams are normal and effective, provided the team is dedicated to achieving a result, has sufficient time and is reasonably self-aware you can imagine any arbitrary skills starting point. We could take a team that consisted entirely of testers and give them some work to do. The first tasks they encountered would require some analysis, so someone from the team would need to do this. They wouldn't complete the analysis as quickly or to the same quality as someone with the relevant experience, and they might not enjoy it as much. They might also need to get outside expertise (training and/or mentoring), but they would get through the work eventually. The team might find someone who wanted to grow into this role and make them the primary analyst, or they might decide that no one enjoys it and the role should be rotated through the team (this is probably less efficient, but it explicitly ranks satisfaction and retention ahead of efficiency). The same pattern can be repeated with development and anything else the team needs to get done.

Don't get me wrong - this is clearly a slow path to success and I wouldn't recommend it, but it is a valid approach. It's a contrast to the behaviour I see more often, which is "I can't do X, because I'm a Y". "Can't" is definitely the wrong word - finding the right word will determine how you respond to this assertion. If the person is really saying "I don't like doing X" then keep the assignment short and try to find some other way to make it attractive. If they're saying "I don't know how to do X", then they need training, guidance and support. If they're saying " I won't do X" then they probably shouldn't be part of the team.

That's why I don't like roles - why do I provide descriptions of roles and responsibilities? It's because I would rather start with a team with diverse skills than spend a lot of time developing the same skills, and most organisations understand that as a request for roles. I also appreciate that most people have gravitated towards a role that suits their preferences and skills, and asking people to move into other areas requires both time and motivation that we usually don't have a the beginning of project.

My hope is that companies that go to the trouble of assembling multi-discipline teams will also recognise the benefits of keeping those teams together for long periods of time. Time is the essential ingredient to building role-independent teams - it helps build trust between the team members, it lets people experiment and learn, and it gives the team opportunities to get feedback on different approaches to task assignment.

An interesting exchange

I wrote this in response to a customer email, but it seemed like something that I could put here:

On 14/10/2009, at 9:15 AM, ... wrote:

Morning Steve,

Saw this article and thought it was interesting. http://www.javaworld.com/community/node/3530

Regards
...

From: Steve

It is interesting. I think it's hard to get the tone right when you're dismissive of agile zealots (who it's hard to agree with, almost by definition), without being dismissive of agile in general. Having been involved with agile for a long time I have a nice clear centre that tells me that agile should be adaptive, but I've also seen the problems that come from being too adaptive too early.

I totally agree that the process for one or two really talented, motivated people is different from the process you use for larger groups/applications. Alistair Cockburn has been very explicit about this in his writing about the Crystal family of methodologies, and Kent Beck has touched on the same subject from a different perspective when he wrote his blogs about "Flight of the Startup". The hard part for most of us is dispassionately looking our our team and ourselves and changing process over time. It's easy to err on the side of too much, too soon or too little, too late, but very hard to judge it just right.

On the "where's the Access of today", that's a question that we've thrown around within Cogent as well. My personal opinion, shared by some others in the company but not everyone, is that we've gone backwards over the last 10 years in productivity and our ability to deliver things quickly. Our capabilitity has increased - we can deliver bigger, more sophisticated applications - but in the process everything seems to have become harder.

I agree with the poster who mentioned the combination of Ruby on Rails and Heroku - that's the combination that I'm using for my personal projects now. I think it's the combination that's the winner - there are other frameworks that to me look more promising than RoR but with each of them I have to manage my production environment myself (for some of them even setting up a production environment is a challenge). Heroku takes care of that and make deployment as simple as possible.

On the OS X desktop MacRuby looks very promising, leveraging Inteface Builder with the power of Ruby. I don't play in the Windows space, so I don't have any idea what the possibilities are there.

...

Steve

Code Review for Git

I wanted to find a way to contribute to projects that I couldn't pair on, and to be able to scan a project for possible changes without sitting down and taking up someone's time. Sounded a lot like automated code review, so I had a look around at available code review tools. There are a few available - Review Board, Smart Bear and Crucible among them. I know Atlassian and generally like their products so I decided to install Crucible and give it a try.

There were a few confusing configuration problems. Until I read the documentation more closely (at all?) I wasn't aware that the beta Git plugin only worked against local git repositories, and I had to change a configuration file to convince Crucible that Ruby files should be treated as text instead of binaries, but once I got past these problems I clearly had a powerful tool for distributed, multi-party code reviews available to me.

So I created a review, looked at some code, and saw one thing that I would change. I wrote the comment, but things just felt wrong. It had taken me a while to type to comment (I'm not the world's fastest typist, but nor am I the slowest), and it felt like a waste of time. It would have been far faster to make the change and annotate the corresponding commit, but what I had was a code review tool, not an editor. Something wasn't working for me.

When I do a code review I want to achieve two things - I want to fix the code and I want to educate someone (where that someone may be me, if I wrote the original code). What I really wanted was something (an editor) that would let me focus on changes between versions, but let me change the latest version and save it. My VCS could store the comments against the (small) changes and I could point people to the VCS history to see what I'd done, potentially using the same tool.

After looking at a few different tools last night I've settled on Changes. Changes lets you compare directory structures and files, and it comes with some Ruby scripts that will help you retrieve specific revisions from Git, drop them into temp directories and review the differences. I've modified the script to use the current directory if you're comparing to HEAD, so that changes to the head are saved directly to disk and can be committed.

Here's my process:

1) Use GitX to determine the revision I want to use as the base
2) Use changes to view the differences
3) Incrementally work through the files either making changes or excluding the files.

I've created a (silent) video that shows me working through one review. I only make one small change, so it's mostly scrolling and excluding. One small point - the video uses an older version of my Ruby script that created a temp directory even if you were looking at HEAD, so changes were lost. I've fixed that in this version.

I saw some comments that Changes was rather expensive (US$49.99), but it's really cheap compared to many code review tools so I'm pretty happy with this approach.

What would you like in a code quality tool?

Although continuous integration has been a huge step forwards for the development community I think that over time we've tended to treat it as a rather large bucket that we could keep tossing things into. Sure, someone should immediately address that method that's too long, but should it really stop the build and the release of the next version if all the tests pass? My feeling is no, it shouldn't.

I think that the agile community could do more to encourage various parallel continuous processes that test different aspects of the application. In particular, we could separate the static analysis of the code quality from the behavioural analysis that we perform with tests, be they unit, integration or functional tests. If we separate the two aspects then we can look to improve them at different rates and in different dimensions. This separation is the motivation behind an idea that Marty Andrews and Simon Harris have been kicking around for a few years (Marty is the author of Roodi and Complexian, and Simon is the author of Simian, so they have a long-running interest in code quality). Cogent is now planning to turn this into a product - something that you can point towards your source code repository and simply unleash.

We don't have a name for the product yet (suggest one if you like), but it will run static analysis tools (like Checkstyle, Simian, Roodi, Reek, Flay) against each revision of your code base so that you can see trends over time, and it will produce warnings when the values of the metrics exceed certain thresholds. The product will be free for open source projects.

We'll start with a simple, web-based version, probably to run over Ruby projects in public Git repositories (because that's low-hanging fruit for us), but we're going to be very community driven. With this in mind we have a survey that will let you influence the development direction, even before we start. If a tool like this is something that interests you, please fill in our product survey. As a bonus, the first 500 people to complete the survey will get unlimited free access to the product in recognition of their contributions.

Emerging Agile UX Practices

I am not a UX professional, but I've interacted with a number of them over the last few years, in the context of agile development. The interaction hasn't always been smooth - the prevailing mood has been that agile development was asking UX professionals to do a bad job, but I've never felt that way myself. My feeling is that UX professionals and software developers have been on similar paths in regard to agile development, but the UX people are starting a few years later and have had less leadership so their challenge has been more difficult.

Why do I think the paths for both roles are similar? Because once-upon-a-time many software developers would have said that agile was asking them to do a bad job. After all, how could you do a good job with no upfront design, no long architecture phase, no detailed requirements? But what developers realised (either quickly or slowly, or perhaps not yet) was that agile values didn't mean doing a bad job, they meant doing a good job in a different way. Part of that was learning new skills and new ways to work. The software/programming community was lucky because they had early leadership in these skills and practices.

The feedback I've gotten in the past from UX folk was essentially "when we're in an agile project we can't work the way we work now", to which my answer was a resounding YES! You (as a profession) need to find new ways to work that produce good results faster. It's not your fault that those techniques don't exist yet, or that they may be hard to discover, but that doesn't invalidate the search. But I haven't done very well at explaining that.

However, practices for UX professionals on agile projects are emerging now, and Jeff Patton has done a good job of describing the practices that he sees working in this context. Moving to agile development means developing new approaches for everyone involved, and it's good that this is now a little easier for UX.

Cogent Customer Brief

We're not heavily into formal contracts at Cogent, since we think they fail to capture the essence of the relationship. We understand that everyone needs to know how much they're paying and for how long, and we've also had some unfortunate experiences where we didn't do a good job of communicating expectations to the customer, so we've put together a document that describes, in the plainest language we could muster, what to expect then you collaborate on a project with us. You can find it on this page of our website.

Failure is mandatory

A few days ago I had a conversation with Travis about failure. In some senses I'm a hard ass about failure - I think if you protect people from failure all the time you also protect them from learning anything. It's still important to protect people from potentially traumatic failure, but traumatic failure is the exception rather than the rule.

The context was how far our duty of care extends as consults, particularly in regard to the transfer of a solution to the customer. There's typically tension around this issue - the customer wants to maximise their investment in the solution itself, and in other projects as well, and that works against either documenting the solution or getting permanent staff involved. One position is that we should go out of our way to document/record knowledge of the solution for handover, even if the customer hasn't made that a priority. I think that's paternalistic (which is ironic given the example I'll use later). I think it's our responsibility to warn the customer that there's a handover risk, and to give some advice on how we'd handle that, but in the end it's up to the customer to decide how to invest their resources, and to wear the consequences of those decisions. I think it would be quite wrong of us to divert resources into documentation against the customers wishes. We might think the customer has made the wrong decision, but it's still their decision and they won't thank you for subverting it.

Another context that we talked about was when you see someone else about to do the "wrong thing". You should definitely draw attention to the problem as you see it, but if the person goes ahead anyway, should you do anything to protect the project from the outcome? My position is that you should not - you should let the project suffer the consequences (again excepting traumatic outcomes). It's possible that your fears won't be realised, in which case you'll learn something. If they are realised, the project team will learn something (including perhaps to pay more attention to your advice) and may be able to avoid similar problems in future. If you just fix it, then what the project team will learn is that there are no negative outcomes from that behaviour, so they'll quite happily repeat, and you'll have the same personal situation later. Ad infinitum.

The same kinds of things apply to my son. I try to protect him from traumatic outcomes, but sometimes I need to let him get hurt (physically or emotionally) so that he learns not to repeat a behaviour.

Experience - who gets the benefit of the doubt?

Bob Martin seems to have created a tempest with his apprenticeship model proposal, but it's worth remembering that it's not a new idea. There was at least one OOPSLA workshop on the Software Studio concept last century, and Ken Auer has implemented a related model at Role Model Software. I think Bob's raising the idea because he's had enough of crappy development and he wants to provoke some thoughtful responses (though that's just my impression). His article also triggered some related thoughts on how experience is treated in our industry.

One response to Bob's suggestion was that hierarchy somehow mistreated or devalued the more junior staff. It's quite true that this is one possible outcome, but the converse is also true. In non-hierarchical teams it's possible for the senior staff to be devalued! You can bring this into focus by thinking about a team of 2, one junior and one senior, and then asking yourself who should get the benefit of the doubt in a deadlock.

My experience is that many "juniors" think that they should get the benefit of the doubt. I've seen this attitude start immediately after graduation. I don't think that's right, or fair (but of course I'm arguing from an old guy's perspective). I've seen someone get upset because they didn't have carte blanche to replace existing frameworks, even though the changes would affect 30 people. I've seen someone simply defy the team conventions and go ahead and do it their preferred way, even after a team-wide discussion of the alternatives.

Everyone needs to understand that part of being a software developer is learning to *convince* people that your way is right - if other people don't understand, then you need to take ownership of the failure and find better ways to convince people. There is always going to be an "incumbent" position - sometimes incumbency is just the way it's done, and sometime incumbency is the older, more experienced person who perhaps can't articulate their position either. Someone needs to get the benefit of the doubt, and my leaning is towards experience.

I can't comment directly on the quality of very recent graduates, because I've stopped advising companies to hire them. My gut feel is that graduates come out with a good foundation for a career in software development, but still have a lot to learn. Sadly (for them) the salary for a graduate seems to be about $50K, and I can get people with 10-15 years of experience for just over $100K, and I'll take the more experienced person every time. I don't believe (as some people have implied) that mastery starts at 4 years and everything past that is wasted.

Design Improvement bids close this Friday

A reminder for anyone considering the Design Improvement workshop that bids close this Friday. We've got 12 distinct bidders at the moment, so if you're interested you should probably make a bid near your final limit rather than trying to be tactical about it.

Also a little advance warning - we're looking at running an EAT Design Patterns workshop in May, so watch out for that or let me know if you're interested in getting mail when it's announced.

Do unto others...

Ron Jeffries enigmatically once said of the XP practices, "they're only rules". What I think he meant was that the practices are a social contract within the team - everyone is expected to abide by the contract, but the team is also free to change the contract when that's appropriate. What we don't want on an XP team is someone unilaterally deciding that the rules don't apply to them. Instead, we want them to raise an issue in a retrospective so the team can hear why some rule isn't working for that person, so that the team can provide advice, support, suggestions, or possibly change the rule.

Developers understand this in the development team context, and also with regard to the rules covering the interactions of the customer management teams with the developers, but I find that they are rather more tolerant of the rules covering the interactions going the other way. In particular, I regularly hear people suggesting that although the customer/project man ager wants to do things a certain way, the developers have decided to do something else "in the interests of the project". What's worse, I feel the urge to do this sort of thing myself!

This pattern appears mainly in two contexts - priorities, and quality. When developers know better, the choose to implement features in an order different to the order requested by the customer, usually because it will be "more efficient". This is fine when everything gets done but if there are some hiccups the customer may be left with features they didn't really need while also missing features they think are more important. While one order may truly be more efficient, that may not be the most important thing from the business perspective and if there's disagreement the final decision on priority rests with the customer.

The equivalent behaviour regarding quality is harder to detect. Sometimes the developers decide to implement lower quality than the customer requested, but more often I see the developers decide to implement higher quality. The context is usually that the developers have two approaches - one can be implemented quickly but hard to change/maintain, and the other is slower to implement but easier to maintain - and the customer prefers the "quick and dirty" approach. In spite of this, the developers decide to implement the "nice" approach. Once again, this is fine if the developers can somehow do it in the same timeframe (overtime perhaps?), but not if it's done at the cost of other features.

Developers need to recognise (intellectually and emotionally) that they are in a service industry, and when push comes to shove they take direction from the business, represented by the customers and management. Developers have a responsibility to present their views and recommendations, with all the supporting information, and then let the business make the business decisions, which include how to spend time/money. When business and development don't agree there are two main reasons - development haven't presented their position in a way that the business understands, in which case the developers need to improve their presentation and influence skills, or business understands the development position quite well but it isn't the most critical factor in the outcome. Sometimes developers don't have the big picture! In neither case the problem that "the business is stupid" or "the business made the wrong decision", though they certainly may make decisions that the developers don't agree with (it undoubtedly happens the other way around as well!).

Of course, none of this applies when people are trying to make decisions that they aren't actually responsible for - in that situation everyone needs to say "thanks very much for your input, but I don't think you're responsible for the final decision". It's common for this to happen with estimation, when the business tells the developers what needs to be done (their responsibility) and how long it's going to take (the developer's responsibility). Sometimes the lines are blurred though - the business is making "draft" decisions, fully expecting them to be reviewed by developers, rather than final decisions - and the situation needs to be clarified.

So if you're a developer who interacts with the business then you have a responsibility to respect the decisions of others, and you'll probably benefit from honing your influence skills as well as your technical skills.

Easy Access Training

In 2007 I'd like to try a few different things, and the first of these is what I'm calling Easy Access Training. Many developers have trouble getting budget and/or time off for training, so I hope to offer one day of training per month on a weekend, and have the price float depending on demand. I expect the rate to come out lower than a commercial course, which seems to be about $600/day (+GST), but I honestly don't know how much lower; probably it will depend on the courses that I offer. There seems to be a lot of interest in test driven development, so I'm starting there, but I'm keen to hear what other courses people woule be interested in on this basis.

What do people think of this idea?

Shu Ha Ri

I'm fascinated to read various posts (mostly blogs) about how "Agile" is bad, sometimes accompanied by a caveat that "agile" is ok. To me any distinction between the two is artificial - from its inception agile development has encouraged adaptation of the method. I see the distinction arising for two reasons - (i) simple misunderstanding, since adaptation isn't included in every description of agile development; and (ii) a mismatch between the question and the answer in shu-ha-ri terms.

In these discussions "Agile" is characterised as a fixed set of practices that must be adhered to, regardless of context or experience. While some people may approach there use of agile this way, I feel this is a misinterpretation of agile development - false doctrine, or if you want to be extreme, heresy!

Adaptation is certainly part of extreme programming, even if this word isn't used. In the second edition Kent and Cindee talk about reflection as on of the XP principles, and using root cause analysis as one of the corollary practices. These are both part of adpating XP to the local context. When the agile manifesto said that "...we have come to value...responding to change over following a plan" I took that to include the process itself! During a recent panel discussion I was asked which agile practice I would introduce first, if I could only introduce one, and my eventual answer was "retrospectives". If you introduce regular, open, honest retrospectives then given enough time and some knowledge of alternatives you'll eventually get the process that you need. And I think that I'd classify that you'd probably get an agile process. Diana Larsen speculated on the same lines.

So if adaptation is an intregral part of agile development, why doesn't everyone understand that. I think the answer is embedded in how we teach and talk about agile. In "Agile Software Development" Alistair Cockburn talks about Shu, Ha, and Ri as three stages in learning. In the Shu stage, the student needs a clear set of instructions or rules, and the faith that these rules will give them the results they need. In the Ha stage, the student can automatically apply the rules, but has begun to understand that the rules don't always give the best results. They start to look for these exceptional cases and apply different rules in these contexts. They gain flexibility but, and this is the important part, they need to be fluent in the basic rules first. Kent had something similar in mind when he suggested renaming the XP practices etudes - things that you practiced for fluency, but that you didn't necessarily apply rigorously every single day. In the Ri stage the student has moved beyond rules, but that's beyond the context of this discussion!

Most people, and I don't think this is restricted to software development, want to skip the Shu stage and move straight to Ha. They want to hear that the rules don't apply all the time and then decide when and when not to apply them. I hear this all the time - we want to do risk driven pairing, we'll only write tests for the complex parts of the code, things like that. The inconvenient truth is that this approach doesn't work. Until you've personally applied the practices in both appropriate and inappropriate situation you're just not in a position to make those decisions. You'll apply the practice where you want to apply the practice, but that won't be the same as where you should apply the practice.

So when I'm introducing people to agile practices, I'll say that you pair all the time, that you write unit tests for every method, that you should have 100% code coverage from your unit tests. I know these things aren't true, and it's not what I would say to someone who already had some experience with the practices, but they are very useful first approximations. They are Shu statements. People experienced with software development, but not with agile practices, can sense that these aren't complete answers - if they think that this is all there is to agile, that there isn't more sophistication futher down the track, then they are liable to reject agile as snake oil. It's easy for them to get this impression from introductory presentations, since it's hard to come up with material suitable for everyone in the audience, and it's even easier if the presenter hasn't moved past Shu themselves. We teach students in groups of similar ability for a reason!

In an ideal world this distinction between "Agile" and "agile" would disappear, but I'm not hopeful. Instead I thiink we'll see two things happen - some people will create branded, proprietary agile methods that aren't subject to community reinterpretation, and other people will stop talking about 'agile' altogether and just 'do stuff that works for them'. Time will tell.

Photos from Bangalore

I know that before I went to Bangalore I was really interested in what the physical environment would be like, so I want to share these photos from the business side of things (the more general side will need to wait until more photos get uploaded). If you'd like to hear more about our personal life in India, have a look at Amanda's blog.

The office building that I was working in was part of an IT campus, right next door to the golf course, that also included IBM and Microsoft (Microsoft is down where the bridge is in the photo). You might think from these pictures that it was a very western environment, and it certainly was inside the buildings , but outside, where building was still in progress, the contrasts became more apparent. The roads were kept clean by people with horsehair brooms who came out at night (the brooms looked just like the one that our maid used, with short handles that meant that you needed to bend over to use them), and when you look at the picture of the building site (right next to our office), note how many people and few machines you see. Construction here is very labour intensive.

 
 

Bangalore was a study in contrasts. The house next door to our apartment was palatial (and apparently occupied by late teen boys who wouldn't have been out of place cruising Chapel St, doof doof music blaring), but two doors further up the street the garbage pile was being browsed by cows and goats, and the ironing was being done by the side of the road, with charcoal powered irons. My office had a very pleasant cafeteria, but less than a kilometre away this is where breakfast was being served (sorry for the blurring, but the car was moving!).

 


 
 

Traffic was roughly equivalent to peak hour on Hoddle St in Melbourne, but a lot more chaotic. Horns are used to indicate "I'm here", since lanes, indicators and overtaking are all rather free form. Auto rickshaws fill the gaps between the cars, motorcycles fill the gaps between the autos, and there are usually some bicycles thrown in just for good measure. Rapid construction leaves the footpaths a bit of a mess, so there are frequently pedestrians on the roads, though it doesn't making crossing the road any easier.

Agile assignment in Bangalore

As some of you will already know, I've just finished a three month engagement for a Wall St bank, helping them refine their use of agile development practices. I'm going to refrain from comment on the specifics of the engagement, and instead focus on the things that I learned myself.

1) Indian programmers are just the same as western programmers - same strengths, same weaknesses. Some are good, some are not so good, some are quiet, some are extroverted. I've know this intellectually, but now I know it emotionally as well.

2) Recruitment in Bangalore is just as difficult as it is anywhere else. It might even be harder, since in Bangalore you're competing directly against Microsoft and IBM (who were on the same campus I was on), as well as Google. Most of us don't face that level of competition in our local regions. And sure there are lots more graduates in India, but if you want people with 5 or 10 years experience the ratios are less favourable. Plus Infosys wants thirty thousand new programmers in 2007 (anecdotally), and that might include just a few of the good ones ;-)

3) There are cultural differences, but they are swamped by the twin tyrannies of distance and time zones. Remote offices are dealing with comparative strangers, and between the USA and India there are only a few hours of effective overlap in a day (it's a bit better between India and Australia). Communication over email and phone just isn't the same - you need lots of face to face experience to overcome this.

4) For distributed development you need people who are willing and able to travel. Include that in your profiles.

5) You can't effect rapid change in a distributed group, especially if there isn't a strong consensus to begin with. And forming that consensus takes a long time. The Forming-Storming-Norming-Performing cycle takes a lot longer for a distributed team - there are plenty of opportunities for miscommunication, and it takes a lot longer to sort the miscommunications out.

6) Everyone's ability to influence is diminished by distance. This was certainly a personal challenge, and I don't feel that I really rose to the occassion.

That's enough to get the things that are distracting me out of my head so that I can focus on some pressing work, but I'm sure that more things will occur to me later. When I get a chance to right a blog while I'm online I'll include some photos so people can get a better sense of the work environment here in Bangalore.