Why be a consultant?

In the last week a few different people have asked me about the pros and cons of being a consultant, so I thought I'd start the conversation by posting some thoughts here.

First, the pros. The most important thing for me is increased freedom to choose what I work on. I keep in touch with quite a few people in IT, particuarly in Melbourne, and I frequently hear about interesting projects. As a consultant, I get the chance to be involved in them, while as an employee I'm restricted to what my current employer is working on. I intentionally said "chance to be involved", because of course some of those aren't using consultants, and some are using consultants, but for some reason I'm not suitable. Even so, right now there are plenty of opportunities. When you're working inside a company, not only are there fewer choices, but most companies only present you with one - the one they'd think you should or would most like to work on. You rarely get given choices at all.

I can also increase the choices available to me by changing by rates. At the extremes, it would be easy to set my rate high enough to not get any work at all, and if I said that I was willing to work for free I'd have lots of choices. There's a happy medium somewhere in between, and it's up to me to find it, taking into account how interesting the work is, how much I need or want to earn, and anything else that seems appropriate. Some people say that you should always charge the same amount, and I do have a standard basic rate, but I think it would be self defeating to let something I really wanted to work on slide by because we couldn't find an agreeable rate (on the other hand, there's always a bottom line below which I'll just walk away and write some articles or work on my own software). In principle an employee can "change their rate" as well, by changing their salary, but it's a little more complicated for an employee!

Another pro is the treatment of intellectual property. To be clear upfront, I've got no interest in intellectual property that I'm not entitled to, or that I created for a client while I was under contract. However, when you're an employee the common law on intellectual property is quite broad, and covers anything related to your employment, even if it's something that you do outside of working hours on your own equipment. Many companies don't seek to enforce their rights, but some companies do, and you can easily end up in a grey area where a previous employer might seek to enforce their rights to something successful. It's not an issue to me right now, but I can see how it could be, and it was one of the drivers that lead me to consulting in the first place, many years ago.

Flexibility of working hours is another pro. Not so much on a day to day basis, but over the course of a year I can to some extent choose how long my contracts are, how long between contracts, and whether they're part time or full time. In the next 12 months I hope to spend more than the standard four weeks with my family. This needs to be balanced against how much I want to earn - it can be hard to say no when there's work available.

Because one of the downsides, probably the major one, is less predictability. When my current contract ends, I can't be sure that there will be more work available - particularly interesting work - starting at the right time. In some sense, there's more flexibility in this area when you're an employee, since your employer may choose to take you off one project early to start you on another one. That's not really acceptable behaviour for a consultant, at least not in my view.

Another con is that you need to be looking for work more often, almost continually. You need to pay more attention to keeping in touch with your network, broadening it, and establishing and maintaining a reputation. I think this is called marketing :-) None of this is paid work, and it can be time consuming. I view writing articles, presenting at user groups and conferences, and even writing this blog, all as parts of my marketing efforts. I need to approach them with more discipline than if I was an employee.

A logistical concern is that, at least initially, the timing of your pay can be less predictable as well. As an employee, you got paid a fixed amount each month, probably on a fixed day. As a consultant, you do some work, you invoice for it, then you wait for payment on the invoice, then you can pay yourself. The invoice may be for a small piece of work, but it might also be for a larger piece of work. The client may pay promptly, or they may drag their feet a little. Many companies quite reasonably don't pay until 30 days after invoice, which even on a monthly invoice can be 60 days after you first started the work. I handle this by paying myself a fixed monthly wage, leaving money in the bank when there's an excess, but the transition from employment to consulting can be awkward.

When you're a consultant, you also need to do everything yourself. I've already mentioned marketing, but you also do sales, accounting, web site construction, email configuration, do your backups, pay bills, produce invoices, create your own business cards, and anything else that needs to be done. To do your accounting, you probably need to understand a little more of the tax lawy than you do right now - for example, about alienation of personal services income. Doing all these things yourself can be a huge change from working in a large company with lots of support staff. The corresponding plus is that you get a lot more insight into how business works, and when every dollar comes out of your own pocket, you become more conscious of costs and you get a better insight into why your managers were so concerned with restraining expenditure.

There is probably a lot more, but I need to head off and do some other business now. The last thing I want to bring up is that I'll try to remember all these things as more people get involved with Cogent. People need to be able to choose their work, change their rates, be flexible with their time. I don't want people working with Cogent to leave because they're not satisified in these areas. It will be a different kind of company *s*.

When Agile Doesn't Work

When I first started talking about agile development, way back in the last century, I was inclined to present agile solutions to every problem that was presented to me. I think this said more about me than about agile - that I was actually insecure about my ideas and understanding, so I felt I needed to look authoritative by solving every problem. As I became more secure, I was happier to say "agile doesn't solve that problem - nothing solves that particular problem". I think this is a very important transition, and with the benefit of hindsight I think it makes people sound more authoritative, or at least more trustworthy, when they readily admit limits to whatever they're advocating (I think politicians would benefit from the same approach).

The secret that zealous attackers of agile, and its zealous defenders as well, don't want you to know is this - nothing works all the time. Every approach you have needs to be modified to take your context into account, particularly your people (see Alistair Cockburn on this). I can be confident that my assertion is true because it contains a very subjective term used repeatedly in software development discussions - "works". When you have a discussion about sofware development, be very careful to check that everyone is using the same definition. There is one cheap defense of agile that would suggest that it can always "work" - agile methodologies are adaptive, so if your methodology isn't working right now, then change it until it does work. Technically that's true, but I don't think it's particularly helpful (nor is it helpful for people to ignore the principle of adaptability so they can dogmatically dump on agile, but I've already written about that).

One are of agile that might not work for you is design. I teach a course on TDD, and one of the things that I emphasise in the course is that TDD and BDUF are end points of a continuum, and that very few people (if any) really work at the end points of the continuum. In the most zealous TDD shop, some thought is being given to the future requirements and overall shape of the application, and in the most zealous BDUF shops some design is being done in response to problems detected during development (or even later). There are lots of factors that influence where you should be on the continuum, but they include design experience, technical experience, and domain experience.

One way in which agile can go wrong is by being too close to the TDD end of the continuum (it's less likely to go wrong by being too close to the BDUF end of the continuum, because of the inherent biases in adopting agility). Things go wrong when the design is moving in a bad direction, often in small increments, and the development team can't detect the problem and respond to it. I used to think that all developers with more than a few years of experience could tell when the design was heading in the wrong direction, but experience is teaching me otherwise. It makes me more sympathetic to the "architecture" groups of large companies, who want to somehow check everything that's going on in their organisation.

If you've got people who can't tell when a design is going wrong I think your choices are:

1. Don't hire them in the first place, or get them off the team in some other way


2. Educate them so that they can see what's going wrong


3. Change your process so that there's more oversight or review, to try to stop designs going wrong in the first place

In principle, XP "out of the box" uses (2), with pairing as a major mechanism for education. This works provided you have enough people with the appropriate design skills. My preference is for (1), but it's hard to find good people - that's one of the motivations for eliminating growth from my company objectives. Most companies go for a combination, with an emphasis on (3). Personally, I think that (2) can be very difficult, and that one of our weaknesses as an industry is how to demonstrate skill shortcomings and then correct them.

If you feel that agile approaches to design "don't work", I encourage you to think about whether they don't work globally, or they don't work for your team right now.

Don't abandon agile

Reading blogs last week I came across Agile People Still Don't Get It along with people responding to the post, including links to the Pliant Alliance, and what appeared to be plenty of posts concerning "large 'A' agilists", who remain nameless. I don't want to single out any specific writer, since it was a common thread, but it appears to me that the agile approach to development is being politicised, fractured, and misrepresented. While I'm willing to believe there are plenty of well intended authors, I also suspect manipulation and hidden agendas.

A classic approach to attacking an idea, often used in politics, is to redefine it then attack the new definition. This is often referred to as attacking a "straw man". A variant on the approach is to take to characterise an individual's views as representative of a much larger group of people, then use an attack on the individual's approach/ideas as a de facto attack on the more general group. A third is the ad hominem attack - questioning a person's motives rather than their position. These can all be applied consciously and maliciously, or in a simply sloppy way through careless construction. I think we're seeing a little of each, though it's impossible to tell. Plus we're seeing the impact of the internet level-playing field, where all voices are potentially equal and if enough people are saying something then it gradually aquires the weight of truth.

For example, while I was reading, I found some interesting questions asked on the Pliant Alliance site:

Just so we don’t get accused of being complete snotty Agile bashers, I wanted to point out this page on the XP website. It basically says what PSD says, which is to fix the process when it is broken and improve the way software is being developed.

So this is a bit of a conundrum. If the XP website itself has some pliant leanings, why has XP, along with the other Agile processes gotten so many people frustrated? Has agile become the victim of book sellers and process consultants? Is it the monetization of the process(es) that has led “agile” to mean something completely different than what was originally intended? And if so, how could we have let this happen?

Adaptation has been part of agile processes from the beginning - I don't understand why it is so often ignored. Is it ignored willfully, deliberately, or through ignorance? The answer is inherently both - I think there are people who ignore it willfully, and their version of agile, san adaptability, is repeated enough that for many people it's the only version of agile that they hear about. The same can be true of any agile practice. Of course it would be easy to go too far the other way and use "then just change it" to address any and all shortcomings, but it is core to agile development that if some practice isn't working, then the team should consciously and deliberately change it. To pick one agile practice, probably an XP practice), say that doesn't work in my context, and then generalise to "agile doesn't work" is a big leap at best, and deceptive at worst.

Another thing that distresses me is that many critics want to write off agile completely, not just in any particular context. It distresses me because I don't want to go back to the situation I was in two years ago, where I frequently had to justify agile development to a hostile crowd, rather than an agnostic one. I'm very happy for people to say that agile doesn't work in their environment (though it would be even better for them to be even more specific, and possibly say which practices didn't work), or with their team, or with their customers, but I wish people would retract the blanket statements. I suspect it's a reaction to the zealous undercurrent within the XP community, who frequently (when I was watching the XP discussion group) would say that "if XP isn't working for you, you're not doing it properly", but those people never spoke for the people who wrote the agile manifesto.

It's possible that "agile" is impossible to defend, because you can pick any from any author and tag it "agile" without too much risk of contradiction. In the last five years, lots of thinking about agile development has matured. Certainly there are things that I said five years ago that I wouldn't say now. And certainly, especially early in agile development's history, provocative things were said to move the debate. This seems to have been lost on many people - if agile hadn't taken an extreme position early on (pun intended), it would have been very easy to ignore. Instead it provoked a response. Even if we only consider TDD, "agility" has changed the way that we develop software.

Hi, I'm Steve Hayes, and I do agile development. It's an inclusive, people-centred approach to doing iterative, incremental software development. It uses a combination of technical and social practices to increase collaboration and reduce feedback cycles. Agile development works best with an experienced team, as do most development methodologies. For the best results, agile practices should be customised to suit the strengths and weaknesses of your team, and the peculiarities of your domain. Agile development is not suitable for every project, or even every team. It has strengths, and it has weaknesses. There is no silver bullet.

Return to Consulting

I've been putting off blogging because I wanted my first post to be "non-trivial" - not just a book review, or a short comment about something that happened at the office. Then I realised there was an easier approach - to start at the beginning. So I'm going to talk about why I quit my permanent position and returned to consulting.

One person said it's because I need to be in control, and that's part of the answer but not really enough. More complete is that I don't like the models of control that are inherent in traditional business structures, where decisions are made at the top and rolled out to the employees. The models actually make sense from one perspective, because most people in the company don't have all the information they need to make balanced decisions. If I'm going to make decisions on how to spend money, then I really need to know how much money we have to spend (how I spend my next dollar does depend on whether it's my last one or there are a thousand more to follow) and what impact my decision will have on revenue. So to be a full participant in decision making I need to know about the company financials. But if the employees knew all the financials, then they'd be privy to a whole load of things that most owners want to keep secret - including how much money the company is making, how much the average salary is (or even relative salaries, especially if I'm going to make hiring decisions). The employees would start to ask questions about who decisions were benefitting.

Now don't get me too wrong - I believe that many corporate decisions are made in the interests of the employees, and could be explained in the context of the full financial context. And I think that lots of people wouldn't object to owners getting rich if it was clear that they were adding a lot of value to the company in some way. But I also believe that explaining these decisions would take a lot of time and education, and would require articulating things that are currently only understood in an intuitive way, so it's certainly a difficult path. Far easier to say "trust me". But once you rely on "trust me", it's really really tempting to slip in a couple of self-serving decisions as well.

So the first reason I'm consulting again is that I don't like the "trust me" environment. I also have some apparently weird ideas about structuring a company to maximise participation and satisfaction, and I've discovered that I can't implement them unless I have something "at risk" - unless I'm the one (or one of the ones) who suffers if the ideas are wrong. If I'm an employee, I get told that "we can't sell that to the customer", or something else about the revenue stream. As an employee I'm an expense, not responsible for "paying the bills". So when I say that I wouldn't work with a particular customer I'm told "we have to", or something along those lines. I've discovered that if I want to implement my ideas then I need to be in control of the revenue (or at least there can't be some "other" who is control of it).

And lastly, I have a weird attitude about what I can and can't do when I'm an employee - I certainly feel that I need to do the work that I'm told to do, that I can't just so "no, I won't do that", even if the worst thing that can happen is response is to be fired. As a consultant I don't have the same reservation. I say it's weird because in principle there's no difference - I refuse to do something, I'm told not to come back and I stop getting paid. I'm ok with the consequences in both cases, but as an employee I won't actually do it. That's my problem *s*.

I've been a bit of tease about my "weird ideas", but this is enough for one post, and I need to hold back something to write about next.

Return to Consulting

I've been putting off blogging because I wanted my first post to be "non-trivial" - not just a book review, or a short comment about something that happened at the office. Then I realised there was an easier approach - to start at the beginning. So I'm going to talk about why I quit my permanent position and returned to consulting.

One person said it's because I need to be in control, and that's part of the answer but not really enough. More complete is that I don't like the models of control that are inherent in traditional business structures, where decisions are made at the top and rolled out to the employees. The models actually make sense from one perspective, because most people in the company don't have all the information they need to make balanced decisions. If I'm going to make decisions on how to spend money, then I really need to know how much money we have to spend (how I spend my next dollar does depend on whether it's my last one or there are a thousand more to follow) and what impact my decision will have on revenue. So to be a full participant in decision making I need to know about the company financials. But if the employees knew all the financials, then they'd be privy to a whole load of things that most owners want to keep secret - including how much money the company is making, how much the average salary is (or even relative salaries, especially if I'm going to make hiring decisions). The employees would start to ask questions about who decisions were benefitting.

Now don't get me too wrong - I believe that many corporate decisions are made in the interests of the employees, and could be explained in the context of the full financial context. And I think that lots of people wouldn't object to owners getting rich if it was clear that they were adding a lot of value to the company in some way. But I also believe that explaining these decisions would take a lot of time and education, and would require articulating things that are currently only understood in an intuitive way, so it's certainly a difficult path. Far easier to say "trust me". But once you rely on "trust me", it's really really tempting to slip in a couple of self-serving decisions as well.

So the first reason I'm consulting again is that I don't like the "trust me" environment. I also have some apparently weird ideas about structuring a company to maximise participation and satisfaction, and I've discovered that I can't implement them unless I have something "at risk" - unless I'm the one (or one of the ones) who suffers if the ideas are wrong. If I'm an employee, I get told that "we can't sell that to the customer", or something else about the revenue stream. As an employee I'm an expense, not responsible for "paying the bills". So when I say that I wouldn't work with a particular customer I'm told "we have to", or something along those lines. I've discovered that if I want to implement my ideas then I need to be in control of the revenue (or at least there can't be some "other" who is control of it).

And lastly, I have a weird attitude about what I can and can't do when I'm an employee - I certainly feel that I need to do the work that I'm told to do, that I can't just so "no, I won't do that", even if the worst thing that can happen is response is to be fired. As a consultant I don't have the same reservation. I say it's weird because in principle there's no difference - I refuse to do something, I'm told not to come back and I stop getting paid. I'm ok with the consequences in both cases, but as an employee I won't actually do it. That's my problem *s*.

I've been a bit of tease about my "weird ideas", but this is enough for one post, and I need to hold back something to write about next.