Installing Git on my Mac

Inspired by Dr Nic's post I wanted to install git on my Intel-based Mac. Why? There were two motivations - I frequently do work on the train or tram, and less often on planes, which is the reason Nic describes, but I also wanted to be able to commit mini-milestones on my local machine that I'm not really ready to commit to the "point of truth" on svn. This is the point where I reveal that I don't do all of my programming test first, but that's a longer thread. Suffice it to say that there are periods where I need to work to get the test coverage back over the threshold (currently set to 99.2%) and I don't want to commit to svn until the test coverage is back there.

I installed git-core from MacPorts, but when I ran git-svn I didn't have a "clone" command, which is what all the cool kids are talking about. I'm still not sure that I know precisely why that was the case, but in poking around on the web I found a few places with installation instructions, and eventually things were working, so I'll add what I did to the pool (my main motivation is letting you know the problem I was trying to fix though).

1. Followed the instructions from http://www.beyondthetype.com/2007/6/15/guide-to-installing-git-on-a-intel-based-mac to install the latest version of Git from source


2. Re-installed subversion using the package from http://downloads.open.collab.net/binaries.html. Why? Because the last step in the previous instructions (setting the PERL5LIB) didn't help, and I couldn't find the svn-perl libraries anywhere. The instructions on http://speirs.org/2007/07/22/getting-git-svn-working-on-the-mac/ were helpful. After the subversion installation I had a directory /usr/local/lib/svn-perl, which did the job.


3. Added export PERL5LIB='/usr/local/lib/svn-perl' to my .profile


4. Added /usr/local/git-1.5.2.1 to PATH in my ,profile (early on, to make sure git was found here first)


5. When I tried git-svn I got an error concerning Term::ReadKey when perl tried to prompt me for a password. I tried to install Term::ReadKey from CPAN interactively (sudo perl -MCPAN -e 'shell'
   , then 'install Term::ReadKey', but got "Can't exec "./make": No such file or directory at /System/Library/Perl/5.8.6/CPAN.pm line 4566.". The answer was to change directory to ~/.cpan/build/TermReadKey-2.30 and execute the following commands: sudo perl MakeFile.pl; sudo make; sudo make test; sudo make install. Now I have the Term::ReadKey module installed!


6. When I tried to 'gitify' after that, I got this message : "error: More than one value for the key svn-remote.svn.fetch: :refs/remotes/git-svn". This seems to have been caused by previous failed attempts - removing the xxx.git file and running 'gitify' again seemed to work.

For people like me who are less than clear about the result, here's what you get:

1. A copy of your project in '../project_name.git'. This is your git working directory.


2. Your repository is in .git in the git working directory.


3. Change into the git working directory and start doing your stuff


4. You'll probably find these commands useful:
   
   
   
   • git status : tells you what's uncommitted
   
   
   • git add : adds a new file to version control. "git add *" seems to do everything you'd want :-)
   
   
   • git checkout -f : reverts local changes
   
   
   • git commit -a : commits any modifications (but not new files) to the repository
   
   
   • git-svn rebase : merge svn changes into your git repository
   
   
   • git-svn dcommit : commits all your changes back to svn
   
   
   
   

And if I'd known it was going to be this hard when I started, I wouldn't have.

Active Scaffold

After quite a bit of writing and business related non-programming work, I've got my hands back onto Rails. I'm actually crossing a few boundaries by writing an application to help me manage the invoices and employee payments for Cogent (the accounting system does a great job from a purely financial perspective, but I need something that can reflect our internal policies and procedures, which still makes it sound much grander than it is). I've also made a personal commitment to try to understand more about the gems and plugins that I'm using, which brings me to Active Scaffold.

I've used ActiveScaffold in the past for the administrative part consumer facing applications, but this time around I decided to have a closer look at what I could do with it. It's still an exercise in progress, but I've been very impressed with what I've seen so far. Let's look at three simple things first - ordering columns, changing the format of a column value, and filtering the displayed rows.

You tell a controller us use Active Scaffold for CRUD actions by adding a line to your controller. For example, if I want to use ActiveScaffold to maintain the Sale model object then I could set up a controller like this:

active_scaffold :sale

However, the active_scaffold method also takes an optional block, which you can use to customise the behaviour of the scaffold. In particular, there is a method to set the columns in the list view, that you can use like this:

active_scaffold :sale do |config|
config.list.columns = [:invoice_number, :customer, :amount, :status, :salary_package, :entry]
end

In this case I've specified that there will be six columns in the list view, in precisely the order specified in the array, where the symbols in the array correspond to accessor methods on the Sale model object.

Changing the format of a column is done outside the scaffold configuration, in a helper method. The :amount column is some amount of money, so it would be nicer to format is as a currency amount. The formatter needs to be named {column_name}_column, and it receives the entire model object as its only parameter, so a method to format the amount column in the Admin::SalesScaffoldController would look like this:

module Admin::SalesScaffoldHelper
def amount_column(sale)
number_to_currency(sale.amount)
end
end

Finally, filtering the displayed rows is so simple that I stumbled across the functionality by accident. In my Sales model, the status attribute is an enumerated value that contains, among other values, 'Open' and 'Closed'. I set up a link to the controller like this

link_to :controller => 'admin/sales_scaffold', :action => 'index', :status => 'Open'

and expected to need to explicitly provide some filtering inside the controller. Instead, it worked before I'd done any coding at all! It turns out that any parameters that correspond to attributes are used to filter the results in the table. You can specify a single parameter, or you can specify multiple parameters, in which case they'll be and'd together.

I'm really impressed with the capabilities of Active Scaffold, and I intend to explore them in more depth for applications where the tabular presentation of Active Scaffold is a good fit, and for providing functionality quickly while a user interface design is refined.

Introduction to Rails Workshop

It's been quite a while since I blogged, mainly because I've had my head down writing new workshop material. We're running an introductory Ruby On Rails workshop on July 21 (just two weeks away now), and although it's been a lot of effort I'm pretty happy with the way the material is working out.

One day isn't very long to look at the equivalent of Java and a full Java web development stack, so rather than simply repeat what's already available in books and screencasts we're going to cover things that we find useful in our day to day development work. We'll be giving attendees a whirlwind tour of Ruby, a brief guide to Rails, and then we'll focus on tools and practices that aren't quite so accessible to the ruby newbie.

So if you've heard about Ruby and/or Rails but haven't made the plunge yet, or you know someone like that, go have a look at the course description, check out our pricing policies and send us a bid. At the moment all you need to bid is $100, so you may be getting some really cheap training!

New introduction to agile methods

It's been a while since I posted, and it's mainly because I've been busy updating our introduction to agile methods. It's done now, at least well enough to show people. Warning - this is a large (7mb) pdf file. It's not intended to stand alone - the real value comes from someone talking it through - but you may still find it interesting, and there may be parts you can use (with accreditation please).

From Noam Chomsky

Who always says it better than me...

"If you assume that there is no hope, you guarantee that there will be no hope. If you assume that there is an instinct for freedom, there are opportunities to change things, etc., there's a chance to contribute to the making of a better world. That's your choice."

A New Agile Manifesto?

Brian Marick makes some great points here about things that are important to agile development, but weren't the first order of business when our major concern was getting permission to do it. If the default is that agile is acceptable, Brian points out that it's essential to understand what teams need to execute agile processes successfully.

Brian's pulls are "Skill, Disciple, Ease and Joy". Parts of this resonate with something I've said to plenty of people - too many software developers consider pain to be part of their jobs. I've seen some terrible things done and excused with "it's always hard", or "it has to be that way". For change to occur, we first need to see the potential for something better ("it shouldn't hurt like this"), and then we need to be able to envision that better thing (which requires skill). And the result is frequently things that are easier to use for everyone.

The joy thing is going to be even harder though. Programmers seem to have an habit of finding someone else to blame for their pain, rather than looking closely at themselves. If you don't believe me, go read some Dilbert. Dilbert's problems are caused by Catbert, by PHB, sometimes by Wally, and he's generally the good natured, innocent bystander. Now these characters are intentionally caricatures, and I don't want to stretch the analogy too far, but I think they capture the software development zeitgeist. We'd do better if we stopped seeing ourselves as victims, and involved ourselves in finding solutions to the problems we see, especially the ones that impact us! If you think your boss is making stupid decisions, make it your business to understand their perspective on the decision - maybe you're not even looking at the same problems!

Anyway, Brian says things better than I can right now, so go read his blog.

Cogent, Year One

When I started Cogent I had a couple of aims, some of which were poorly articulated at the time. I wanted to create an environment that didn't have the employer/employee tension that I experienced in most workplaces; I wanted to be able to work with the people I wanted to on the jobs I wanted to, and be able to say no to the other stuff; and I wanted to create an environment that was attractive to other developers. I'm pleased to say that at lot of this has been accomplished (I can't say that I've accomplished it since lots of people have helped, particularly Marty Andrews), but I'm now also more aware of the limitations of the approach

It's been just over a year since I started consulting again. After doing some Rails and R&D work at IBS I was really lucky to be able to spend three months in Bangalore working with a Wall St bank. It not only gave me a chance to see the Indian Silicon Valley from the inside, so that it's no longer just an abstract concept, it also gave me a chance to catch up with people I hadn't seen since I left New York in 2001. Then I was able to take six weeks off, which was exactly the flexibility that I was looking for when I started consulting, and now I'm working at Sensis with a bunch of good people, including some people working under the Cogent umbrella.

Over that time Marty has gone from being someone simply billing under the Cogent umbrella to effectively being a partner in the business, which has made things both easier and much more interesting. We've got six people billing effectively full time consulting work through Cogent, and another two people billing part of their work that way. Everyone keeps the vast majority of their own revenue, leaving just a little bit for shared expenses. And I've been learning more about accounting and running a business, which has its good and bad points. That's all the good stuff.

What I've become aware of lately is that while the current business model is 'fair', in the sense that everyone keeps they earn, it has some drawbacks, the main one being that there's no financial incentive for collaboration. There's still room for altruism, and none of us are particularly selfish, but financially there's no reason for me to spend time helping someone else build up their skills, or increase the rate they can charge, especially if means spending less time on something that would help me directly. I can live with that situation, but I don't really feel good about it.

As a result, we've started (just started) thinking about different relationships people might have with Cogent, reflecting different levels of risk and independence. The model we have now is great for people with deep skills who are happy to independently find their own work and bear the associated risks. It's not so good for people who want more predictability, or who would like to work more collaboratively, or have someone who was helping them improve their skills. One possibility is a full employment model, though we'd still have our signature financial transparency and profit sharing of some sort. Cogent takes the risk of finding work and paying salaries, and gets a share of profits in return. The employee gets security and predictability, and gives up some of their revenue in return - it's a pretty traditional model at the core. But we'd expect people to eventually move away from this into a looser arrangement, where they bear more of the risk and get more of the revenue, and eventually into the associate arrangements that we have now. We'd want to have enough flexibility so that no one needed to completely abandon Cogent as their needs and contributions changed.

Early days yet. We'll have to see what happens.

Just when you least expect it...

I rarely post about personal things on this blog, but I'm going to make an exception. What follows is an email that my wife sent to me yesterday. Jamie is our three year old son - I hope that I can just retain a fragment of the magic that he holds lightly in his hands right now, and that he never understands that you can lose it.

---

Tonight while walking home, Jamie asked me where the moon was, and I said, "I don't know - maybe it's behind the clouds." He said, "When I get big and go into space, I will get the moon and bring it down for you. Okay, Mummy?"Awwwwwww.

RailsBrain

If you need to refer to the Rails API documentation frequently, as I do, then you'll really appreciate RailsBrain.

Design Patterns - last reminder

A final reminder that if we receive enough bids we'll be closing the Design Patterns workshop to attendees midnight this Friday. Post your bids by email.

Template Methods in Java and Ruby

Template method is a pattern commonly used in Java frameworks to allow application specific configuration of a generic framework. Most of the behaviour is contained in a generic Base class (usually abstract), and the consumer provides their specific implementation in a concrete subclass that overrides methods invoked from the superclass template. Some form of dependency injection (often Spring) is used to let the application know what concrete class needs to be instantiated. Testing the concrete extension usually means testing that it invoked the responsibilities of the superclass as expected. Let's look at an example.

class Extension extends Base
{
void extensionMethod()
{
this.abstractBehaviour(someParameter);
}
}

I want to test interactions, so normally I'd use a mock, but I can't mock out the superclass. A brute force approach is to roll my own mocking by creating a new subclass for testing.

class TestExtension extends Extension
{
boolean wasInvoked = false;
void abstractBehaviour(Object parameter)
{
wasInvoked = true; // could be more complicated
}
}

public void testExtension()
{
TestExtension extension = new TestExtension();
extension.invokeFrameworkMethod();
assertTrue(extension.wasInvoked);
}

An alternative is to inject an object that encapsulates the behaviour as a strategy, like this:

class Extension extends Base
{
Extension(Implementation implementation)
{
this.implementation = implementation;
}

void extensionMethod()
{
this.implementation(this);
}
}

Then I can inject a mock or stub implementation and verify that it is invoked, and separately verify that the implementation does what I expect by injecting a mock/stub Base into the implementation. That certainly separates the two concerns, and each concern is neatly unit tested. It's also quite a bit of code. It also points us in a new direction for the framework - injection of strategy objects for the customisable pieces of code instead of using template method and inheritance.

Ruby can handle the problem using both these approaches, but there's a third alternative that isn't available in Java. In Ruby, I can directly change the behavious of the framework class.

class Base
def extensionMethod
abstractBehaviour(parameters)
end
end

All done. No new classes, no injection, and the entire application is aware of the change once the code is loaded. But how do you test it? We go one step further, and override the behaviour of just the Base instance that we are testing, like this:

specify "Should invoke abstract behaviour" do
base = Base.new
class << base
attr_reader :was_invoked
def abstractBehaviour(parameter)
was_invoked = true # could be more complicated
end
end
base.invokeFrameworkMethod
base.was_invoked.should == true
end

Of course, in Ruby you might still benefit from the flexibility of using injected strategy objects rather than a template method, but you might also be able to get away with something that is even easier than either of the Java alternatives.

From Java to Ruby, Rails for Java Developers, and Everyday Scripting with Ruby

Bruce Tate is a popular Java author (Better, Faster, Lighter Java, Bitter Java and Bitter EJB) and blogger, so it's significant when he shifts his focus from Java. In the earlier Beyond Java (which I haven't read), Tate looked at what lead to Java's dominance, and the limitations that were creating opportunities for other programming languages. In From Java to Ruby: Things Every Manager Should Know (Pragmatic Programmers) Tate focuses entirely on Ruby, which he sees as a strong (but not certain) contender for the 'next' programming language (though it's unlikely that anything will be as ubiquitous as Java has been). As the title indicates, the book is targeted at managers. If you're a manager or team leader who needs an overview, perhaps because your developers are advocating Java, or you're a programmer who needs to think about about how to advocate for Ruby without your organisation this is a good book. If you're a programmer looking for details on programming in Ruby, you're better off with one of the books I talk about next.

If you're already familiar with web application development in Java Rails for Java Developers by Stuart Halloway and Justin Gehtland is an excellent introduction to Rails. It runs through a point by point comparison of web application development in Java (using Spring, Hibernate and JSPs) and in Ruby on Rails. It's a short, accessible book (292 pages), but it's not a replacement for Agile Web Development with Rails (Pragmatic Programmers) - if you're doing serious Rails development you'll still want that on your shelf as well. But developers who only need a comparison and an introduction, perhaps before they decide whether to jump into the deeper water, will definitely appreciate "Rails for Java Developers".

Finally, I'm reading (but haven't quite finished, because it's sitting on my desk at work as a reference)
Everyday Scripting with Ruby: For Teams, Testers, and You by Brian Marick. Although ostensibly targeted at non-programmers who need to 'get something done' with Ruby, I'm finding this to be an excellent Ruby introduction for anyone. It has the clearest explanation of regular expressions that I've come across - I'm weak on regular expressions, but after reading this book I was confident enough to apply them both willingly and successfully on my current project. Once again, this book isn't as deep as the Pickaxe book (Programming Ruby: The Pragmatic Programmers' Guide, Second Edition), but I think it's much more accessible. Definitely recommended.

Hard Facts, Dangerous Half Truths & Total Nonsense

Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management by Jeffrey Pfeffer takes aim at many of the management fads of the last 20 years - not individually, but as a group. It critiques the anecdotal approach used by many management gurus and finds it lacking. It points out that in many cases a management technique might be present in a successful company, but also present in a large number of failing companies, and many (most?) management authors only present one half of the equation.

"Hard Facts" doesn't give any easy solutions to this problem. It suggests that every company is unique, and that it's always a good idea to run some sort of internal pilot to validate a new management approach.

This isn't a great book, but you might find it very useful if you're trying to critique the management fad du jour at your company.

Programming is important - really important

I wrote this in response to a comment on my post about Dreaming in code, but I wanted to expose it to people who view this via RSS, so here's the comment and my response.

---

Finally, something I can talk about.

I disagree totally.

Programming is not about code.

For me, a program or application doesn't live in the computer. It is not the bits and bytes. It extends much, much further.

For me, an application extends across everything the code affects. Not only is it the code, but it is also the users’ and the stakeholders’ mental models, processes and understanding of what the code does, what their organisation is and how to use the code to enhance their organisation.

Applications extend beyond computers into the people.

Example - a content management system is more that just the DotNetNuke code. It is also assigning the roles of writers and editors, it is understanding the people who will read the content and shaping the navigation and tone of the articles appropriately, it is teaching everyone in the organisation that they can request changes, and communicate their important stories with the world through the web.

When I am writing an application, I am not writing code. I am talking to people, I am trying to understand their issues, ideas and viewpoints. I am trying to work my way to the root of the problem. I am working out what questions to ask, talking to end users and training people to see things in different ways.

The application exists both in the computer and in the minds of people. As application building is not about code, it's not creative writing.

You don't look at the source code for great pieces of software. Or look at the architecture of great pieces of software. You don't look at their design.

What are patterns if not examples of great pieces of software and design?

What is MSDN - the magazine? Yes it is a Microsoft advertising vehicle, but it still has great code to read, understand and appreciate.

What are all the sites, forums and blogs on coding (including this one) but a sharing of knowledge on building applications and computing?

---

I use some of these words differently to you. I don't think a 'program' is that same as an 'application', and I don't think that 'software/application development' is the same as 'programming' at all. So I agree with all the things you say, up to the quote, about applications and application development, but at some point in application development we need to do some programming and to me that is absolutely, positively, by definition, about code. Programming isn't the only thing that we do in application development, but it's an important thing and I think there is a tendency to devalue it; to treat it as a typing exercise in which all programs that perform the same function are treated as equally worthy (you didn't say this, but I hear it a lot). I think application development would benefit from improvements to many disciplines, but I feel that programming is one of the most neglected disciplines at all.

When we write a program we write for two very different audiences - the program needs to be understood by the computer, which is apparent when it does (or doesn't) perform the expected functions, but it also needs to be understood by human beings. Most of our effort currently goes into pleasing the computer, but in some sense this is the easier of the two audiences. Gabrielle is saying (and I agree strongly) that we need to put more of our effort into satisfying the human audience for code. I'll assert as a corollary that finding ways to make code more expressive to people will also make it easier to write code that satisfies the computer as well, but I can't offer a proof in a mathematical sense.

As to what are patterns, and the code in places like MSDN and most web sites (including this one), they bear the same relationship to great programming that a power tool catalogue has to great building. They're focussed predominantly on efficiency, and on satisfying the computer, not on the aesthetics of programming as it's experienced by a human being. Christopher Alexander presented a keynote at OOPSLA many years ago, which I had the honour of attending, at which he said that he was ashamed of the way that the software community had applied patterns; that the heart of his idea was creating environments that appealed to our humanity but that software patterns has been sterilised - stripped of aesthetics and reduced to the equivalent of screwdrivers (my words, not his, but I think I'm faithful to the intent). Show me an article, or rather not one exceptional article but a body of work, that addresses different ways to name variables within the same programming idiom to improve expressiveness, or how to write code that brings a smile to your face, and we'll be heading down the path that Gabrielle is talking about, and we'll have taken one tiny programmatic step towards Alexander's goals.

You once asked me in email why I went back to uni to study psychology - my reply should have been because I think it would make me a better application developer. If I wanted to become a better programmer I wouldn't choose psych, I'd choose creative writing and literature, and that I haven't speaks to lack of time and dedication to my craft, not to a lack of need.

Dreaming in Code

I've always been interested in books that tell the story if creation from the inside, and particularly when they related to software. I remember being very impressed by Tracy Kidder's The Soul Of A New Machine when I read it way back at university, and another of Kidder's works, House. This week I finished another book in the same tradition, Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software by Scott Rosenberg.

"Dreaming in Code" recounts the development of Chandler, an open source personal information manager (PIM) that's now been in development for four years. I have to admit that when I saw the project start date, I assumed that the book documented a rise and fall, and I was really surprised to find out that the project was still in development, but without a widely used release, after all this time. It's not really important though. "Dreaming In Code" is a journey, not a destination, and it was revealing to see that developers with great pedigrees have the same problems that I do on my projects.

There were a few quotes along the way that should have been worth saving, but in the end I'm left with these:

"People write programs. That statement is worth pausing over. People write programs. Despite the field's infatuation with metaphors like architecture and bridge-building and its dabbling in alternative models from biology or physics, the act of programming today remains an act of writing - of typing character after character, word after word, line after line. Tools that let programmers create software by manipulating icons and graphics shapes on screen have a long and sometimes successful history... But these have generally served as layers of shortcuts on top of the same old text0based code, and sooner or later, to fix any really hard problems, the programmer would end up elbow-deep in that code anyway.

People write programs

...Is programming a kind of creative writing? The notion seems outlandish at first blush. Any discipline that involves complex mathematics and symbolic logic does not seem to share the same cubbyhole with poetry and self-expression. Yet the programming field could learn much from the writing world, argues Richard Gabriel, a veteran of Lisp and object-oriented programming who is now a Distinguished Engineer at Sun. 'My view is that we should train developers the way we train creative people like poets and artists. People may say, "Wall, that sounds really nuts." But what do people do when they're being trained, for example, to get a master of fine arts in poetry? They study great works of poetry. Do we do that in out software engineering disciplines? No. You don't look at the source code for great pieces of software. Or look at the architecture of great pieces of software. You don't look at their design. You don't study the lives of great software designers. So you don't study the literature of the thing you're trying to build.'"

And Rosenberg's Law:

"Software is easy to make, except when you want it to do something new"

with it's corollary,

"The only software that's worth making is software that does something new."

RCov measurements

I'm busy setting up a Rails development project at a client site, and we've chosen to use rSpec for specification/testing, and rCov to report coverage. They work quite well together, but there's one caveat - classes that aren't loaded don't appear in the coverage report at all, so for a single class there's effectively no difference between 0% coverage (no tests at all) and 100% coverage. Of course this is an oversimplification, since Ruby loads files, not classes, but it's a good enough approximation on most projects, and there's clearly some sort of problem regardless of the details.

Our solution has been to force rSpec to load everything in app/models and app/controllers before the specs are run. We do this in the rspec_helper, and since this is loaded multiple times (on different paths) it's also useful to restrict this code so it only runs once.

First, here's the code that loads the models and controllers:

class ForceLoader
def self.run
["models", "controllers"].each do | app_component |
directory = File.join(RAILS_ROOT, "app/") + app_component
Dir[directory + "/**/*.rb"].each { |file| require_dependency file }
end
end
end

There are two things to note about this code:

1. we use require_dependency for consistency with other Rails loading, rather than require or load;


2. we need to ensure that the path of the file passed to require_dependency is the same as the path used by default by Rails. Ruby loading is path passed, and if you refer to the same file with two different path representations you may load it twice.

Next, let's look at the code that we put in rspec_helper.

begin
ForceLoader
rescue
require File.dirname(__FILE__) + '/force_loader'
ForceLoader.run
end

If we can't reference ForceLoader we load it and run it. Once the class is loaded the rescue code won't be invoked, so this ensures once only execution.

Hopefully this approach will give you more accurate coverage reports with minimal overhead - it's certainly uncovered at least one problem on our project so far. It can also be extended to cover other parts of your app in a fairly straightforward way.

Amazon links in Ruby

As I mentioned in an earlier post, I read a lot, and I want to be able to comment on the books I like, with links to a page about the book on my preferred book seller, Amazon. I also have an associates account with Amazon and I'd like to include that in the link, even though the last time I made any money from that was about 2001! Making the links has, frankly, been a pain the butt, but I finally dusted off my Ruby and used Amazon Web Services (AWS) to make this easier.

First I tried Ruby/Amazon, but this seemed to be using an old version of the AWS and I couldn't figure out how to do an ISBN based lookup, and I eventually abandoned it. In hindsight I should have done this earlier - the functionality I need was pretty easy to write directly in Ruby, and only the latest version of AWS seems to handle both 10 and 13 digit ISBNs correctly.

So here's my code:

require 'rubygems'
require 'hpricot'
require 'open-uri'

isbn = ARGV[0]

ACCESS_KEY = '01WXX7HHK8GBB3BFYX02'
ASSOCIATES_TAG = 'cogentconsult-20'

site = 'http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService' +
'&AWSAccessKeyId=' + ACCESS_KEY +
'&AssociateTag=' + ASSOCIATES_TAG +
'&Operation=ItemLookup' +
'&ResponseGroup=ItemAttributes,Images' +
'&IdType=ISBN' +
'&SearchIndex=Books' +
'&ItemId=' + isbn

doc = Hpricot(open(site))

author = doc.at("author").inner_html
title = doc.at("title").inner_html
detail_page = doc.at("detailpageurl").inner_html
image = doc.at("smallimage/url").inner_html

puts ''

html = "<a href='#{detail_page}'><img src='#{image}' alt='#{title}'></a><a href='#{detail_page}'>#{title}</a> by #{author}"

puts html
puts

Open-uri made the http access a piece of cake - definitely use this instead of Net::HTTP - and Hpricot was equally adept at giving me just the parts of the returned XML that I needed.

I can run this at the command line using "ruby booklink.rb someISBN" and I get the html for both an image link and a text link, that I can then paste into my web pages and edit ass appropriate. Hopefully I'll now be less reluctant to write about the books I've read.

Code Retrospectives

I bet that most of the people who read this blog have been through a code review at some point - a session where a bunch of people get together, look at a piece of code, identify all the things they consider problems and document them. Then the developer goes away and fixes all the problems.
Usually these reviews are relatively infrequent, time consuming, and exhausting. The focus is on finding errors and correcting them, and I know that when I was having my code reviewed this way I found the process quite stressful as well.

I think this model of reviews is pretty much broken. These days I want to get "code reviews" done at least a few times each hour, whether there's anyone else around or not, and the only way to get that is to use automation. In the Java world, tools like Checkstyle, Simian and Complexian provide a wealth of checks - far more than a human team could check in any reasonable amount of time. And despite the names these checks aren't just "style" checks - they help you enforce metrics thresholds as well (which is a separate discussion). Plus there's much less emotion associated with feedback from an automated tool, and the tool has infinite patience.

Once significant chunks of what's covered by traditional code reviews have been handed off to automated tools, the humans are free to use code reviews in a different way. Instead of looking for mistakes, the team can look for opportunities to learn. Accept that the code is the way it is, that everyone was doing the best they could at the time, and review the process that created the code rather than the code itself, all the way back to the root cause. Certainly you'll find some things in the code that must be fixed before you move forwards, but lots of things will be tolerable if you can fix the process so that they don't recur.

In this context, "process" is a broad term - it includes the way you hire people, the training you give them, the tools, the incentives, the requirements; anything that might impact the final code. The Toyota "Five Whys" can be a good way to get to the root cause. (make some note in here)

At this point we've transcended code reviews - what we're doing is far more like a retrospective, focussed tightly on our code. Although retrospectives are usually associated with ends of iterations and releases, we can use code retrospectives much more often.

So instead of treating your code reviews as opportunities to find the mistakes that your colleagues have made, switch to code retrospectives and improve the process that created the code.

Design Improvement to Design Patterns

Marty ran the Design Improvement workshop last weekend - another full house and another good day. There are some pics on the wiki, and also some blog posts from attendees.

The next workshop is Design Patterns on May 19, and we're really lucky to have secured Andy Bulka as the workshop leader. We've already got four people bidding, so if you're interested you might want to get in now.

Commenting on ideas

I had this email exchange with a friend, and since it's related to change I thought I'd share it (plus it means I don't have to write a separate blog entry!).

---

Steve,

We have discussed giving feedback before and that when you start to
give it your automatically
at odds with the person receiving it.

For example, they present a shape that is a triangle and you
believe a square is better, so
suggesting a different shape automatically suggests that they are
incorrect and those that
also have supported the triangle or have not suggested another
shape are also somehow
incorrect. At least this is how I remember you explaining it to me.

Given that your at odds with the person receiving the feedback, is
there any way to give it
in such a way as to lessen the possibility of it being taken
negatively?

I could try the feedback sandwich approach but do you have any
others that you use with
success ?

If you think the feedback will not make a difference do you suggest
keeping quiet, even if
you think your expected to speak up ?

---

My reply:

Feedback is incredibly difficult, and success or failure can depend
as much on tone and body language as the words used.

First, you need to accept that they may be right and you may be
wrong. I guarantee that whether you accept this or not will come
across somehow. I've used to ask leading questions, until someone
pulled me up on it, and I now I try to make sure that words and
thoughts are a bit better aligned - questions when I'm in doubt,
statements when I'm not.

Maybe the first thing you need to do is better understand why they
used a triangle - "I thought of something different - can you tell
me why you used a triangle?". When you can echo back to the other
person why they used a triangle, and they nod or agree throughout,
then you're on a good track. Then rather than just say "a square is
better", maybe you can say "ok, I understand why you used a triangle,
but I'm concerned about these things as well, and I think that a
square addresses those concerns".

It's quite possible that neither the square or the triangle address
all the concerns - that you've each uncovered some overlapping issues
and some unique issues. Or you've uncovered the same issues, but have
given them different priorities. There may not be right and wrong
solutions, or even outright better or worse.

Although I don't always speak up myself, I don't think it's healthy
to stay quiet when you're asked for feedback, if you have feedback
related to the matter on hand, not general feedback like "I think
you're an idiot"! The times I stay quiet are when I don't think I can
present the feedback appropriately, perhaps because I don't have the
verbal skills, or perhaps because I'm agitated by something at the
time. I certainly need to get better at this myself (this is a case
of do as I say, not do as I do). Try not to offend people, but the
decision about whether the feedback will make a difference doesn't
really belong to you.

Drawing the reader in

I read a lot of material - some short, some long. A lot of it is mandatory work-related reading, so I can get quite selective about the remaining pieces - if you want me to read something, you need to convince me. I'm willing to give you some of my time upfront, but that may be less than 5 seconds (I kid you not). If what I read in that period seems interesting then I'll give you some more of my time, but even then my commitment isn't open ended; you need to continually convince me to keep going (at least until I can see the end in sight). So how do you do this?

You should be able to see the solution in a typical newspaper article. The article probably has this structure:

• the impact of the story is captured in the headline;


• the first paragraph gives a summary of the whole story;


• the next few paragraphs expand on the first paragraph, but still omit details;


• the details are contained in the main body of the article

Each reader should be able to read until they've got the level of detail they need - they shouldn't be forced to read from beginning to end just to get an overview.

We have the same problems whenever we write a document presenting something complex (especially when we need someone to make a decision based on the material), and particularly when we write a resume. Make sure that the first few paragraphs contain some sort of summary, something that will convince the reader that it's worth continuing. If a resume starts with the details of the candidates current job, it better be a very interesting job, and if I'm your reader you've got less than 5 seconds to convince me. Bullet points are easy for me to scan in that time; dense text isn't. As you increase my interest and commitment, you can be more demanding of me and I'll be more tolerant.

When you're working on the web you have other structure you can work with as well. Maybe you can build something with Javascript so that the details are available only when the reader asks for them; in that case you don't need to adopt the linear, gradual descent style of the newspaper, you can make the details available wherever they're needed. You can also present sidebars, summaries and pictures that might not have a place in a text only, paper based document.

You may or may not need to give the reader a sense of closure. If you need a decision or an action, you need to make that clear early on, but you should also restate it at the end of the work. You might also benefit from a summary of the main points, related back to the original introduction. But for something like a resume you can probably get away with "here are the rest of the details" - it's expected, and the reader knows they'll only be interested in some of it, but you still should make it easy to pick one point from another.

Be conscious of this approach - see if you can spot in what you read, and bear it in mind when you're writing. Eventually you'll refine it until the transitions are seamless, and your reader will never know why they find your work so interesting!

Building a store

I just finished watching the video of Building the Store from ClickableBliss, which explains why Mike Zornek build his own web store, and what he learned.

My notes from the presentation:

• ELC Technologies RoR::PayPal - not a complete interface, doesn't provide express checkout, which is essential


• vPayPal gem : never got it to work, seemed to have deficiencies, used as a reference


• Tobias Lutke's PayPal gem - not specific to websites payment pro, related to instant payments


• ActiveMerchant : couldn't get certain things related to PayPal to work, code submitted by contributor, not maintained. But it's now 1.0 and may work better.


• SOAP : not used SOAP before, but was the eventual solution


• These experiences were from about August 2006.


• If you're going to accept confidential information in a Rails application, make sure you don't log it:
  

  
  class ApplicationController < ActionController::Base
  filter_parameter_logging "password"
  filter_parameter_logging "credit_card"
  end

  
  

Sorry if I'm cryptic - the video is worth watching :-)

Quicksilver

My Quicksilver education continues. I watched the Merlin Show on using menu items from within Quicksilver (episode 8), and thought it was a great idea (especially for Textmate, which I'm also trying to learn). But when I tried out the example in my QS, it didn't work; at least not at first. So here are the prerequisites for getting menu items to work:

1. Activate "Enable Advanced Features" in Preferences -> Application


2. Activate the "User Interface Access (+)" plugin


3. Make sure that "Enable access for assistive devices" is selected in OS X System Preferences -> Universal Access. I don't have this enabled by default because I use iKey during training courses and it only works if access for assistive devices is turned off.

Hopefully that's it! But if not, Howard Melman has written an excellent Quicksilver User's Guide.

Design Improvement bids close this Friday

A reminder for anyone considering the Design Improvement workshop that bids close this Friday. We've got 12 distinct bidders at the moment, so if you're interested you should probably make a bid near your final limit rather than trying to be tactical about it.

Also a little advance warning - we're looking at running an EAT Design Patterns workshop in May, so watch out for that or let me know if you're interested in getting mail when it's announced.

Upgrading to Webgen 0.4

I use Webgen to build my website, and I've just finished upgrading to the latest Webgen release, 0.4.2, from version 0.3.8, which was over a year old. It was a bit of a pain, and the Webgen documentation is a little lacking, so I thought I'd share my experiences.

First, I could see that I needed to change the format of my block declarations in my content pages from

blocks:
- {name: content, format: textile}
- {name: sidebar_heading, format: textile}
- {name: sidebar, format: textile}

to

blocks: [[content, textile], [sidebar_heading, textile], [sidebar, textile]]

Not a big deal, though I needed to do it in every file (I need to find a way to get rid of this duplication). I thought that would be enough to get me running, but to my surprise every page generated the message "Invalid structure of meta information".

The problem was that my files (originally created in Windows and now living on my Mac) had lines ending in \r\n and Webgen didn't like this. Things worked better when I changed each file to have lines ending simply in \n.

Next I had to replace references to block content in templates from from

{block_name:}

to

{block: block_name}

After that I was 90% of the way there, with all the standard stuff covered. The next step was to handle two places where I'd used included content.

Webgen 0.4 changes the order of the steps in the evaluation of page files. The new order is:

1. convert to HTML;


2. then ERB;


3. then webgen tags.

In earlier versions I believe the steps were performed in the reverse order. This change in ordering broke some of my "custom" code, where I was including common Textile content into a number of pages by reading it from a file via ERB. Since ERB is now run after the conversion to HTML from Textile I was getting the included Textile content rendered as HTML without translation.

One thing I was using this for was to include a common sidebar. However Webgen now supports nested templates, and that's a better way to solve the common sidebar problem. I now have a template that looks like this:

---
template: ../default.template
--- content
{block: content}
--- sidebar_heading, textile
Services
--- sidebar, textile
<div class="training">
* <a href="{relocatable: eat.page}">Easy Access Training</a>
* <a href="{relocatable: index.page}#softwareDevelopment">Software Development</a>
* <a href="{relocatable: index.page}#coachingAndMentoring">Coaching and Mentoring</a>
* <a href="{relocatable: index.page}#training">In-house Training</a>
</div>
For enquiries regarding any of our services, please <a href="mailto:info@cogentconsulting.com.au">email us</a>.

Pages that want the common sidebar use this template and only need to provide the body content.

The other thing I was using an ERB include for was to provide a common set of textile link aliases (things like [three_rivers_essay]http://www.threeriversinstitute.org/steve%20hayes%20essay.htm). These need to be in the same file as the textile source at the time it's converted to HTML which is before either ERB or tags are handled, so it looked like I was out of luck. My solution was to write my own tag, called "site", which provides the same sort of functionality. A reference to the tag looks like {site: three_rivers_essay} or {site: {name: three_rivers_essay, text: "a different piece of text"}} - I'll include the plugin code at the end of the post.

It's good to see Webgen in active development, I like nested templates, and it's quite easy to write plugins. It would be good to see better documentation (hopefully that will come from the new activity as well) but Thomas Leitner provides excellent support via the forums.

---

class SiteTag  ["Suncorp-Metway", "http://www.suncorp.com.au"],
"three_rivers_essay" => ["software tyranny", 'http://www.threeriversinstitute.org/steve%20hayes%20essay.htm']
# and so on
}
infos( :name => 'Custom/Site',
:summary => 'Return standard sites and labels')
register_tag 'site'
param('name', nil, 'The name of the site that will be linked to')
set_mandatory('name', true)
param('text', nil, 'The text that will be displayed for the link')
def process_tag(tag, chain)
return self.link
end
def link
name = param('name')
definition = @@definitions[name]
raise('Could not find site named ' + name + ' in ' + @@definitions.to_s) unless definition
url = definition[1]
text = param('text') || definition[0]
return "<a href='#{url}'>#{text}</a>"
end
end

Maven doesn't work for me

I sometimes joke that I don't know anything at all, I just know people who do know things and ask them what to do. The joke is that this is partly true - in software development it's impossible to know everything, so it's important to rely on people you trust. Which brings me to Maven 2.

I've had a number of people that I respect recommend that I start using Maven 2, so I thought that I would try it on a project that's just starting out. On the other hand, I heard from people on related projects that they'd tried it without success. No problem, I thought, I've got access to some people who have used it, who have a fairly good idea of what I'm after in a build, and I'll be able to get some guidance from them. Given that I had conflicting advice and I was the one advocating Maven, I decided I should do the implementation myself rather than delegating the potential pain to someone else, and that's what I did.

I've spent a day and a half on this and got something that's working ok, but when I get to work today I'm going to back it all out and go back to the Ant-based approach that other teams are using. Why? Because although Maven 2 may work, I can't make it work the way I want it to (the "I" in that sentence is quite important).

First, I love the Maven 2 dependency mechanism. It's great, and even though Maven doesn't rock my world the experience has encouraged me to go and look at Ivy (though that's in the future). My problem is that Maven 2 wants me to conform to its view of the world, and that doesn't match what I want to do. That would be fine if there were plenty of hooks to let me do it my way, even if that was expensive, but either these hooks are missing or the documentation on how to use them is nonexistent or inaccessible (and it's probably a mix of all of these).

So what do I want my build to do that's "non Maven"?

1. I want to run separate unit and integration tests in one project, and have reports (test results and coverage) for both of them, and I only want to run the tests once to get it. There is a note in the documentation that says that Cobertura will force the tests to run twice and that this will be fixed in future, but every configuration I tried ended up running the tests even more than this.


2. I want to be able to package the code with instrumentation, run integration tests using this code in a WAR, and measure the coverage.


3. I want Checkstyle (and Simian and Complexian) to run over all my code, including the tests

I tried separating the integration tests into a separate module and deploying a jar from the "core" module to the integration module, but that didn't give me coverage of the integration tests because the code in the jar wasn't instrumented. The alternative seems to be to compile the code again so I can run the integration tests over it - this seems like a great big hack, and seems to predict more great big hacks down the way.

The idea behind Maven 2 is great, and if you're happy with the outcomes that's great, but the current implementation and documentation doesn't provide the level of customisation I'm looking for.

If anyone has a POM that handles the three points that I would like in my build, than I would be more than pleased to receive it and blog about my experiences using it. I want Maven 2 to work for me, and it's disappointing that it doesn't.

Tasmania Trip

Not long ago I had a week's vacation in Tasmania. If you're interested you can read a little more on my personal blog.

Carbon offsets follow up

Following the post on carbon offsets I received an email from Climate Friendly. I'm impressed that they're watching the blogosphere (my wife hates that word), and that they're willing to participate in even small discussions like ours.

---

I've read the discussion and wanted to mention that the approach that Climate Friendly takes to the issues you're discussing is to communicate a complete message covering efficiency, renewables and offsets plus advocating very strong government policy. For example a number of our customers have a Prius, run their home on Green Power and offset their car emissions, while others do whichever of these they can afford today. Our interest in promoting efficiency is evidenced by the fact that we have many of Australias leading energy efficiency companies, engineers and architects as customers. Picture sof the buildings they have designed are in the URL below. One of these cut emissions 60% by efficiency measures and then used our GreenPower plus carbon offsets to achieve Climate Neutral status. We are very active promoting the achievement of Climate Neutrality through a combination of efficiency, GreenPower and offsets.

My contribution to the more general question of how offsets can be meaningful in an overall context is summarised in an article sent to New Internationalist regarding their article last year about carbon offsets.
URL is
http://www.climatefriendly.com/pdfs/climate%20friendly%20-%20%20Reduce%20Renew%20and%20Offset%20strategy.pdf

This covers voluntary action but we need regulatory action too, urgently.

An article advocating strong government policy is available at http://www.climatefriendly.com/newsletter/goodpolicy.php

Kind regards
Joel Fleming | Managing Director | Climate Friendly

By the way for our emission calcs we are very careful to use the correct emission factors for the baseline case that our projectrs avoid.
For the NZ wind project the carbon credits are based on an "Avoided" gas power plant ..which creates a very low quantity of credits. This means that if you buy carbon creidt from us its not overclaiming by using "brown coal" baseline.

Presentation at Melbourne Patterns Group

I'll be doing a presentation on the Visitor pattern at the Melbourne Patterns Group on Wednesday. I've attached the announcement below (cutting out the bit that said what I was talking about - oops!).

---

The next patterns group meeting will happen on March 7th at the usual time and location: 6:30 pm at ThoughtWorks Office, Level 11, 155 Queen Street, Melbourne 3001. Parking should be
available near the building.

Please let me know if your coming for the meeting so that I can arrange for the pizza. You can rsvp by forwarding this email to sreeni@gmail.com. Since the building entrance is locked from 5:30 pm, myself or a fellow ThoughtWorker will be patrolling the entrance between 6:30 and 6:45 to let people in. Please call me at 04 3396 3725 if you are left waiting.

Cost of change

In software development we talk a lot about the cost of change. Cost of change is also a hot topic at the moment - how expensive will it be to reduce our carbon emissions? What will the impact on the economy be? So I was interested to read this in The Undercover Economist:

"Ask the polluters and they will tell you that reducing their pollution is like stopping breathing - it would be very expensive to stop, so somebody else should make the changes. But it's not really hard to find out the truth. Regulators can find out how much it costs to reduce pollution by telling people to either change their ways or pay a charge. Watch which decision they make. Judge them by their actions.

The EPA tried this in the case of sulfur emissions. They set up an auction for the right to emit sulfur dioxide, which causes acid rain. Polluters were given a quota of emission permits and could either buy more permits in the auction or reduce their emission by shutting down, installing sulfur scrubbers, or buying cleaner coal. When the EPA simply tried to tell them to install sulfur scrubbers, the power generators argued that it would be very expensive to do so, and they lobbied hard to stop the mandatory regulation. Even the EPA estimated that the cost of reducing sulfur dioxide emissions by one ton would probably be in the range $250 to $750 and might be as high as $1,500. But when the EPA conducted the auction in 1993, very few polluters made high bids. The companies had been exaggerating their costs. By 1996 permit prices had fallen to $70 a ton, and even at that price many polluters were buying cleaner coal or installing scrubbers rather than buying permits to continue polluting.

The regulators discovered that getting rid of sulfur dioxide was so cheap that few people were willing to pay for the right to keep producing it. ... The clever thing about the auction was not that the sulfur emissions were reduced - that could have been required by law - but that legislators all over the world found out how much sulfur scrubbers really cost. It created a basis for further legislation: not making rules in the dark, but in full knowledge of the (modest) cost."

So how would this work in software? Instead of telling people to reduce their cyclomatic complexity, I'll sell my teams some CC permits and see who bids for them and how much they pay. Same thing for test coverage. How about LOC as well? Ok, at some point we need to conscious of the Law of Unintended Consequence, but I like this idea a lot.

Design Improvement Workshop

We just took our tenth bid for the Design Improvement Workshop on March 24. We'll continue taking bids until midnight March 17, moving up in $50 increments. Remember that ties are won by the person who bid first at that price, so it's an advantage to bid early!

I'm really looking forward to seeing everyone on the day - I think this will be a great workshop.

Update - One subtle point is that increasing your bid doesn't necessarily increase what you pay - the final price doesn't change until everyone increases their bid.

Tools of Trade?

I came across this post, suggesting that developers be responsible for their tools of trade in the same way that mechanics are. I guess it caught my eye in part because of the picture of a Herman Miller chair. It turns out that I have one of those chairs, and there have been times when it's become my office chair and people have said "when do I get one of those?", and my response has been "when you buy one for yourself, just like I did". I've had it for at least 12 years, I think it's been a great investment, and I don't expect I'll ever work for a company that will buy me one. On the other hand, computers are (for me at least) consumables that don't last more than a year or two, even if my accountant wants me to depreciate them over three. Who should contribute to each of these, how, and how much?

I'm fascinated by the underlying theme because I think it will be relevant to things that Cogent does over the next few years. At the moment the majority of our work is consulting and the vast majority of the revenue goes back to the consultant who earned it. The accounting is pretty straight forward, but we're starting to look at different things (the EAT program for example), with different people contributing to different initiatives, we're also about to become liable for payroll tax. What's interesting to me is that keeping the books open and the accounting "fair" is already stretching my knowledge of both accounting and my accounting software. Marty and Simon are off thinking about software and training, and I'm thinking about accounting automation!

Marty and I will be talking some more about financial aspects of Cogent later in the week, and discussing some fairly simple models. This is all new to us, and we'll no doubt struggle. If you're interested in hearing more about this side of Cogent add a comment or drop me a line and I'll blog more about it.

Let's talk about tests, baby...

I just finished another TDD workshop, and I'm sitting in the airport waiting for my flight. As usual, I tried to emphasise that a single test should test one thing, and one thing only. As usual, people were having trouble doing that, which is perfectly understandable if you've only encountered the idea of TDD a few hours ago! To help people, I suggested that they articulate the objective of the test to their partner - "This test checks that ....." - and that if their description of the objective include "and" they should think about replacing that test with a number of more tightly focussed tests.

I also went a bit further and suggested that if the description could be expressed succinctly as "When X happens, we get Y value", then they were talking about a state test, and if the description could be expressed succinctly as "When we do A, B happens" then they were talking about an interaction test. I haven't tried to verify the approach yet, and I'm sure it won't be a blanket rule, but it seems like a reasonable heuristic. Since I don't get to program much these days, I'll look forward to feedback from people who do. :-)

Hybrid car or carbon offsets?

I'm in the market for a new car, and strongly motivated by Tim Flannery's "The Weather Makers" I started to do the research on buying a hybrid car. Here in Australia that really means either the Honda Civic or the Toyota Prius, which start at $32,990 and $37,400 respectively. Since I particularly want a hatch, it's really the Prius. This is a fairly substantial margin over what I'd normally pay for a car - my current car is an Astra which my wife and I have found to be great value, and if we replaced that with the new model, which starts at $21,990. Not equipped to the same level as the Prius, but enough to keep us happy :-)

Fifteen thousand dollars is a pretty big margin so out of curiousity, and prompted by Marty's post on carbon neutrality, I decide to have a look at what it would cost me to offset my carbon emissions rather than use a hybrid vehicle. The results were a big surprise to me.

I used two different sites to calculate our family carbon debt. I assumed a new Astra manual, which uses 7.4 L / 100km (from the Australian Green Vehicle Guide), and that we do 20,000km per year. We already use green electricity, and we use an average of 150Mj of gas per day over the year (the vast majority in winter) for hot water, cooking and heating. I've also assumed that I do 3 business trips to Brisbane in a year, we take the family to Sydney once a year to visit grandma and to the equivalent of New York once a year to visit grandpa. The two sites gave comparable results, but the best breakdown came from Greenfleet. Our carbon debt is approximately:

• 3.34 tons from the car


• 3.51 tons from the house


• 41.07 tons from air travel


• 47.92 total

What surprised me is that the contribution of my air travel is over 12 times the contribution of my car! The conclusion is that I need to participate in some carbon offset program, but the interesting part was the cost - to offset all this carbon debt at CarbonFriendly (whose shop sucks, just by the way) was going to cost about $1000.00, and the car part of that would only be about $70 per year, so why should I go to the much greater expense of a hybrid vehicle?

I'm interested to find out of anyone else has gone through a similar decision process. Logic tells me that I should forget about the hybrid and pay for the carbon offsets, but part of me feels like this is somehow cheating. What do you think?

Great Interview with Kent Beck

I frequently suggest that software development is a social problem, not a technical problem. Kent explores this idea in this interview, and shows that his thinking on the topic is clearer and deeper than mine!

Design Improvement Workshop

The January EAT workshop was a great success, so we've decided to do it again. I'm pleased to announce that Marty Andrews will be presenting our Design Improvement workshop on March 24. As with our previous workshop, we're limited to 10 places and pricing will be determined using an auction process. You can see the latest bids and access the workshop details on our
wiki.

Roles and titles

In general, I've been opposed to giving people titles on projects, especially titles such as architect, senior designer, or even tester. I've been opposed because I didn't want the titles to lead other people in the team to feel that they didn't have responsibilities in these areas. My preference is that everyone feel responsibility for every aspect of the project - that the team own all the problems.

Of course people who worked with me at IBS are now saying "hang on, there were people with specific titles/reponsibilities", and there were. For each team we identified three different lead roles, each of which required different skills sets. Let's say that they were design-focussed, people-focussed, and process-focussed. The argument was that when everything was working well things were a team responsibility, but if something went pear shaped I, as a manager, wanted to be able to go to one person and say "what's going on?" - I was trying to handle the failure modes. I don't think this worked as well as it should have, mainly because in my time at IBS I spent most of my time working as a recruiter or a fire fighter, and not really much as a manager (and I need to take the blame for that).

So where am I going with this? In the project I'm currently involved with we have titles/relatively strong roles, and I'm enjoying it. We have two people in the BA role, a test manager, an architect, and me (I'm nominally the development lead, but I haven't really figured out what I do that isn't covered by someone else yet, but I'm sure I will eventually). What I'm enjoying is being able to leave certain things to other people - in some sense to be able to flick them over the wall and not think about them at all. Now don't get the impression that we're sitting in bunkers lobbing artillery shells - that's definitely not the case. There's plenty of collaboration, and we definitely help each other out where we can, but we're also willing to trust other people to do their bits, ask questions where they need to, and keep the rest of the team informed (this is helped by all being in a project room and not having cubicles).

Reluctantly, I think I was avoiding titles because I didn't trust everyone on the team. Not in a suspicious, I-have-to-keep-an-eye-on-you kind of way, but at a level I wasn't really aware of. Still, if you're read the first paragraph carefully you can sense the distrust, either towards people's sense of responsibility or towards their technical ability. I think there are certainly times when that's appropriate - if I had a team of grads I don't think I would give them some roles and 'flick things over the wall'. I'd want to assign tasks, monitor them fairly closely and see how things went. I think that my mistake was trying to create a blanket rule, rather than assessing each case individually, which is a manager's job after all.

If I get a chance to do this again, I think I'd prefer to more formally acknowledge roles within the team. If it turns out that one person has all the lead roles, then lets say so openly - it certainly tells us a lot about the current skills mix within the team and how we need to change it. If a team is mature enough to handle having no formal roles, I think they'll be mature enough to handle having formal roles as well.

Weasel words

In the Development Team of the Future (DTOTF, copyright pending), all communication will be clear, open, forthright and well articulated. Sadly, our current development teams still consist of human beings, using writing and the spoken word, so there's not much hope of transitioning to DTOTF any time soon.

Ok, funny intro (at least I hope so), but what's the point? As I've said before, I think that most commercial software development is a social problem, not a technical problem, in the sense that we know we can build something but we need to figure out how to build it as effectively as possible with the team that we have. It's about people, and people are often poor communicators.

One of the things I encounter fairly often is people using what I think of as "weasel words". They're words that hide conflict under the cloak of respectability. When you're hit with a weasel word you need to choose between abdicating your position (often with some new emotional baggage) or moving to open contradiction and conflict. If you're inadvertently flinging weasel words around you may be surprised by the results!

So what are the weasel words? There are lots, but my list starts with these: "just", "must", "have to", "pragmatically", "realistically", "in the real world", "in practice". "Just" diminishes, trivialising the problem; "must" and "have to" beg the question "or what?"; "pragmatically" and "realistically" imply that the other person isn't being either; "in the real world" and "in practice" label the other position as somehow irrelevant. They all have uses in the right context, but they're also frequently disrespectful.

If you find yourself about to use one of these words (and these are just examples, not an exhaustive list) ask yourself if they're necessary, if they're adding value to your expression. Or try living without them for a while and see how things go. My personal quest for the last 18 months has been to eliminate "just" from my vocabulary, or at least minimise my usage. After all, it's just one word - how hard can that be?

Whose problem?

This post takes me back to my earlier theme of developers as change agents, which had a holiday while I was thinking about the EAT workshop. In my earlier posts, I talked about different people perceiving a problem differently, but there's a particular aspect of this that deserves more attention - even if we all see a problem, who's problem is it? To mangle a Mel Brooks quote, "My problem is a tragedy, but yours is a comedy".

The best way I can describe this is to talk about a real life situation. It's not a once off situation, so I don't have to reveal any secrets or worry about offending individuals. This sort of thing happens all the time. It happens when management is unhappy about development for some reason, but are a little vague about the exact problem. "We have to do better!" echoes around the table in closed door meetings around the board table. Perhaps it's the sales folk speaking loudest, "We're not meeting customer expectations" they say. This message gets passed down the hierarchy; "Make it better" is the refrain.

There are a few layers involved, but eventually the message trickles down to the developers on the front line. Their team leader gets the message clearly (sort of), "Job number one is to make the development process better", and heads off to do so. She has a meeting with the team to tell them that she wants them to do better, but they're quite happy with the way they develop software at the moment, thank you very much. They feel they have a pretty good idea of what needs to be done, defect rates are manageable, hours are pretty good, they get to solve some cool technical problems, and they've each got some nifty noise cancelling headphones. The message falls on deaf ears. As time goes by, each of the team leader's initiatives fall by the wayside, and management sees no change at all.

What's happened? Two things that I can see - the people who needed to live the change didn't actually have a problem. They were quite content, thank you very much. The message "management has a problem, you need to suffer some discomfort" doesn't really resonate very well. Although most of use help our neighbours to some extent, our selfish genes respond most fiercely when they're directly threatened, not just indirectly. So one of management's responsibilities (and I use 'management' in a very broad sense) is to make sure that people are motivated to solve personal problems. The simplest way is to translate problems into actual threats - "do this or you're fired", but it only works in the short term and not very well even then. Regardless, as a colleague many years ago said in reference to our startup CEO "I'm not working my ass off to make him rich". He would have worked his ass off to make himself rich though!

The second thing is that even after we all agree a problems exists and we all need to act, we still need to agree on how we'll measure progress. We need a metric, even if it's a very approximate one. Once we have a metric we can be involved in selecting as well as implementing change strategies, because we can try them out and see what worked, possibly in near-real time. Without that, we're reduced to following change implementation orders - not only is this less than fun, it doesn't give everyone the chance to suggest changes, and as well all know, all of us is smarter than any one of us! It's also helpful to have other metrics in place so we have a chance to measure unintended consequences of our changes.

So if you want some other folks to change their behaviour, make it personal and make it measurable. Encourage them to feel your pain, and help you to assuage it, rather than sitting pack entertained by the comedy of your suffering.

First EAT workshop

We ran the first EAT workshop on Saturday, and it went very well. The setup at the venue wasn't perfect, but we think we have a way to set it up better next time. I like an environment with plenty of light, but if anything we had too much yesterday (at least for projection) so we'll try setting up in the ante room, which has blackout curtains, next time.
And the feedback from the participants was very good. Thanks very much to Marty for co-teaching and for providing the data projector, and to Nigel for coming along and adding his expertise. I'm getting used to working on my MacBook, so I'll gradually get better as well. Alec Clews, one of our attendees, has posted his comments.

 

 

Although the examples and discussion for this workshop were in Java, we're now able to offer the workshop in .NET as well. We're unlikely to run the TDD workshop again for a while, but we'll probably run a design improvement workshop in March - contact us if you'd be interested in that.

Testing images

This is a photo I took while travelling in the US.

I hope it posts ok, but no luck so far.

Best Practices - Global or Local?

I came across "Debating the merits of pair programming" this week, and found it irritating. Lest anyone think I'm picking on a particular piece or author, it's just the latest irritant in a series. You could conclude that I was a just a grumpy old man, or that it was very early in the morning and I hadn't gotten my 'day face' on (and there'd be an element of truth to both these thoughts), but let me try to explain myself.

The article presents quotes both supporting and opposing the practice of pair programming, based on personal experience, and some academic work along with critiques o that work. Implicit is that we should be able to somehow look at both sides of the argument and decide which is right and which is wrong. This is the process that someone might be going through if they were trying to decide if pair programming (and by the way, I think it should be called 'pair development', but that's another story) was an industry best practice (you should be able to hear the fanfare in the background).

I have a problem with the idea of industry best practice, and as a corollary with arguments about whether something is industry best practice or not. Fundamentally, it's a problem of context. When we hear about a best practice, what we're usually hearing is someone saying "that worked in my context", where the context may include the developers, the technology, the customers and a plethora of other things. And I love to hear about what worked (or didn't work) in your context - it's the first step to deciding if that might work for me too. The second step is to understand the similiarities and differences between our contexts.

There's another problem with the presentation of the arguments about best practices, especially when there's a bun fight between the relativists and the absolutists. Absolutists are the people looking for industry best practice - one practice to rule them all, one practice to bind them. Relativists are the folks saying "this works sometimes - it worked for me", or "sometimes this doesn't work". When I talk about pairing, or agile development in general, I try to be a relativist. I can say that it has worked for me. I can try to describe the context in which it worked, how it might work in different contexts, and how it might fail in some contexts. Then I come up against an absolutist saying "Pairing doesn't work", "agile development doesn't work". Come on dude, I just told you that I have a proof by existence - it worked for me! Most times I walk away feeling like I've just been called, indirectly and implicitly, a liar. And sometimes I get a bit heated when I feel like that :-)

So here's my proposal. How about we stop looking for industry best practices, and put some more energy into looking at practices which some of us have seen work and figuring out what contexts encourage or discourage these practices. I'm game, how about you?

PS. Apologies for turning off comments in recent posts - I'm using a new tool and I missed setting an option properly.

Test Driven Development workshop notes

I've just finished porting my TDD workshop notes to S5, adding all the code examples, and in general buffing them up for the EAT workshop on Saturday. As usual, I feel that the value of the workshop comes from the dialogue and the interaction, not just from the slides, so I'm happy to make those publicly available, provided you leave the accreditation in place. I'm very interested in any feedback you might have - for example, I've just realised I need to add some Creative Commons licensing to the material, but I'm sure you'll have other things to say!

Do you really have a problem?

In the last post I talked about change capital, a way to think about how many changes you can confidently make and which changes they should be. However, once you've decided where to make your change investments you need to think about how to implement the change.

Most changes are triggered by a perceived problem. However you need to keep in mind that not everyone thinks there is a problem worth solving - they may be quite happy with the current situation, see a different problem, or agree that there is a problem but not think it's worth solving. So your first step is to articulate the problem that you are trying to solve, and then form a consensus around the problem description.

If you're anything like me articulating the problem, even just to yourself, is a very useful exercise. It takes an intuitive understanding and transforms it into something a little more rigorous. It forces you to consider your terminology, establish some consistency, and gives you a chance to check your reasoning. Including quantification of the problem, answering how much this is costing you, can also be very enlightening - sometimes problems are very annoying, but don't actually cost you very much. If that's the case you might want to put more effort into quantifying the annoyance - does that have an economic cost as well? I frequently find that articulating and quantifying the problem forces me to acknowledge that there isn't a significant problem - at least no one worth investing my precious change capital on.

Even after you've convinced yourself that you've got a problem, you need to remember that not everyone else sees the same problem. Some people will probably agree with you, some people will see the same problem but won't think it's worth fixing (especially compared to other problems), and some people won't see any problem at all. And maybe they're right! Maybe there are benefits to the current situation that you haven't seen. Perhaps the benefits accrue to other people, so it's worth bearing the apparent pain. Maybe there's something else that makes the situation fine in a larger context. Software developers frequently believe that their employer should be focussed on optimising software development, but that's often not the most important thing at a corporate level. Maybe it's more important to leverage other staff, even it's detrimental to software development. Try to see things from other people's perspective. You'll know you're doing a reasonable job of this when you can paraphrase their position, echo it back to them, and see them nodding their heads.

It's really only when you get to this point - where you understand the other person's position well enough to describe it accurately to them, and you still believe that there's a problem, that you can start to move towards a solution.

EAT update

With just over a week to run, we've got 10 bids for the Test Driven Development workshop, which is great - this means that bidding will close on midnight Friday. If there's sufficient demand in the next few days I'll look at running this workshop again in February.

EAT details

BarCamp Melbourne

I've read about BarCamp, especially in the blogs of Kathy Sierra and Tara Hunt, so it's good to see that there will be a BarCamp Melbourne in a few months. I hope to be able to attend but I'm not sure about that at the moment, which is why I'm not on the attendee list.

Marty Andrews co-teaching TDD course

I'm pleased to announce that Marty Andrews/ will be teaching the EAT TDD course with me on January 20. Marty has a wealth of experience in agile development, particularly TDD, and it will be valuable for people to get both our perspectives.

So, you want to be a change agent

Most software developers would say that they're not interested in being a change agent, that all they want is to be left alone to do their own work. In one extreme case I had someone tell me that they actively didn't want anyone else to learn how they worked, because that was their individual competitive advantage!

The trouble with this model is that most of us work in teams, even if the team is virtual and distributed, and most teams agree that they need to have consistency or, oh the horror!, standards. That means that if you come up with a new idea, you'll frequently need to convince other people to go along with it, or at least not oppose it, so you need to be a change agent.

The first thing to realise as a change agent is that you can only achieve so much. I think of myself as having a limited amount of change capital, that I need to spend wisely. Some changes have relatively low benefit, but require so little investment that you can and should do them straight away. On the other hand, some changes will have enormous benefits but will be difficult to achieve and may make the team resistant to further changes - this is frequently the case when a manager forces a change on a reluctant team. You need to pay attention to the return on your change capital investment, and investment it appropriately. Another common mistake is to try to make many simultaneous changes, spreading change capital thin and putting all the initiatives at risk. Successful changes generally depend on some minimum investment, and if you're not willing to make that investment you're better off doing nothing at all. Also, although some people are more resistant to change than others, everyone has a limit to the number of changes they can deal with and at some point people are liable to say "enough!". Flexibility is important, but so is stability and predictability and we need to make sure that we have the right balance.

So regardless of the quality of your approach, which I'll talk about in another post, you also need to pay attention to the quantity of change.