Rules

I was exchanging tweets today with Liam on how to decide when to use private methods and when to put the logic in a new class. This got me thinking about rules, or at least, rules as applied to software development.

I love rules. I'm a rules addict. I find them strangely liberating; they restrict my options so that I can better focus on the creative aspect of whatever I'm doing. They seem to promise repeatability, predictability, and a degree of mechanical perfection that just needs to be nudged together by the developer.

Luckily for me there's no shortage of rules. There's SOLID, the 4 rules of simple design, DRY, Tell Don't Ask and a whole bunch of step-by-step patterns and refactorings to mindlessly apply. ;)

Of course, it's never quite that simple. Rules should come with a disclaimer, WARNING: Some assembly required.

No matter how many books I read, how many blogs I subscribe to, or how many patterns I memorised, a nice design never really seemed to naturally emerge from all that knowledge. So I tried quizzing experts on the rules they use. And pretty much without fail the answer was "it depends". You give them a specific case, they give you an answer. You ask for a rule, and they tend to reply with words like "generally" and "favour", not a specific procedure.

At a course a few years back I even quizzed the instructor until I managed to distil the procedure for getting beautiful designs from BDD. The instructor himself couldn't explain the procedure, as he said much of the decisions are based on context developed from experience. Of course, that didn't stop me trying to get to the rules at the heart of it. Nor did those rules assure my success at producing good designs.

Rules are the question, not the answer

Much as I generally dislike some of the grandiose metaphors that are typically attributed to software development by software developers, one that sticks out as quite relevant here is the Japanese martial art concept of Shu Ha Ri, which describes the three stages of learning required to master a discipline. (At least as I understand it. Please correct any egregious errors I make :))

The first stage, Shu, is all about rules. The beginner memorises the rules and applies them dogmatically. They may not see opportunities, or be able to take advantage of them, because they are sticking to the rigid forms they are learning.

The second stage, Ha, is when the beginner gains enough experience and has internalised the rules enough to start pondering the reasons behind the rules. They slowly begin straying from the rules as they start to notice the shortcomings when applying them to the myriad of situations the world throws at them. They also start to gain a better sense of appraising the relative success or failure of these experiments, and so start to apply rules on a case-by-case basis.

The final stage, Ri, is when the practitioner has risen above the rules. Every situation is dealt with on its merits. An observer may notice many elements of the rules in action, but the practitioner is merely doing what is required of the situation as governed by their experience; the context they have built up by applying and deviating from the rules.

While all this may seem a little twee, I can't count the large number of times I have talked to people that really seem to know their stuff, and found they seem to have this uncanny knack of seeing straight to the heart of a problem and solving it in an elegant, seemingly effortless way. I can see many of the rules present in their approach, yet their solution never seems to be one I could simply derive by applying the rules or a specific formula. In hindsight, the solution tends to seem obvious; but never before I've seen it.

This has led me to believe that rules are the question, not the answer. They are what starts us on a path of enlightenment. It is only once we have memorised the rules and start noticing their limitations that we start asking ourselves why the rules are there and how they work. Once we start this questioning, we start gaining some mastery of the discipline. And it is only after we no longer need the rules that we are truly proficient, and we have uncovered the answer of how to write good software.

And, as many consultants will tell you, that answer is "it depends". ;)

So now what?

Well, I'm trying to break my rules addiction. I like to think I've spent way too much time in the Shu phase due to it's inherent safety (rather than as the result of fundamental incompetence), and that it's time to start looking past the safe-haven of those rules. More than anything I think this comes down to writing loads of code, trying out new approaches, taking risks and questioning the meanings and motivations behind the rules.

How much time do you spend looking for rules on how to develop software right? If the answer is "lots", maybe it's time to let go of them?

Finding functional neatness with Haskell

I've recently started to learn Haskell (used it at Uni over a decade ago, but I'm trying to actually appreciate it this time :)). Here is an example I found when working through the recursion chapter of the excellent Learn you a Haskell tutorial. Yes, I know this is a very corny example, but the fact I was able to figure it out just from the algorithm description in the tutorial, despite being a Haskell newbie, really impressed the beauty of the language upon me.

Remember quicksort? Picking a pivot, then putting it in the middle of the sorted smaller elements and sorted larger elements? Typically you end up with a wad of procedural code, such as this fairly typical C# example.

In Haskell, you work more with describing what something is, rather than how it works. So we end up with something like this:

quicksort :: (Ord a) => [a] -> [a]
quicksort [] = []
quicksort (x:xs) = smaller_sorted ++ [x] ++ larger_sorted
    where smaller_sorted = quicksort [y | y <- xs, y <= x]
          larger_sorted = quicksort [y | y <- xs, y > x]

The first line is defining the type of the function, it takes a list of orderable types and returns a list of those types. The second line is our recursion edge case: sorting an empty list returns an empty list.

The third line is the definition of our algorithm. Given (x:xs), the list head (x) and tail (xs), we put the head between the sorted list of everything smaller and the sorted list of everything larger.

The where clause has our recursive calls using list comprehensions. The line [y | y <- xs, y<=x] selects the elements of the tail (xs) which are less that our pivot (x).

Neat huh?

We can also try and bring the functional-style with us to C#, but you pretty quickly start missing pattern matching, and built-in list syntax like head, tail and concatenation. In many cases (like this one) the C# compiler also won't be able to get you the tail call elimination possible in functional languages like Haskell, which tends to make recursive implementations less desirable.

public static IEnumerable<int> QuickSort(IEnumerable<int> elements) {
    if (!elements.Any()) return new int[0];

    var head = elements.First();
    var tail = elements.Skip(1);

    var sortedSmaller = tail.Where(x => x <= head);
    var sortedLarger = tail.Where(x => x > head);

    return QuickSort(sortedSmaller).Concat(new [] {head}).Concat(QuickSort(sortedLarger)); 
}

This is simpler than the more procedural version, but not as simple as the Haskell version. And we still have to stay concious of differences such as how tail recursion is handled over different platforms which can have a huge impact on performance. With that in mind though, it's neat enough for me to keep digging into functional programming, as even if I don't end up doing much with pure functional languages the techniques can provide some very interesting alternative ways of solving problems in any language.

New Year's resolution

I don't generally go in for the whole New Year's resolution thing. I'm constantly looking for ways to improve (and have a lot of flaws to choose from ;)), and I'm not going to wait for a specific date to start trying to fix things. In this case though, the timing has coincided quite nicely. So here it is:

I will not knowingly contribute, or condone the contribution of, bad code to a project.

Let me tell you a story.

A tale of two projects

Before I start, let me assure you that we are talking about internal code quality here. DRY, SOLID code. The scenarios discussed below are all uncompromising about product quality. The final products go through strict verification protocols to ensure quality. This is obviously completely independent of the internal code design, which is more to do with how easy the code is to change in light of new requirements etc.

So, towards the end of 2010 my team had several quick projects to get through. All had unachievable deadlines, all had a small scope and small domain, and all were of relatively low importance (more research experiments than shippable products).

For the first of these we had 6 weeks. We used an Agile project approach with one week iterations, did TDD, wrote acceptance tests, etc. Clean code all the way. The software that shipped at the end of the third iteration was good enough to fulfil all the major requirements and everyone was really pleased. We added some extra polish and features in the remaining weeks.

The last of these went quite differently. We were given three days. We estimated we'd need three weeks. We communicated that the deadline could not possibly be met, but resolved to do everything we could to get it working as quickly as possible.

After a bit of debate, we decided to ditch any attempt at clean code and just do whatever it took to ship it as quickly as possible. It was a small project with next to no maintenance requirements, so we figured write-only code would be ok. No SOLID, DRY code meant no slaving over the IDE extracting interfaces, or driving out a nice design using rapid feedback from TDD. We wouldn't use acceptance tests, but would manually test as required, and only automate any testing that was easy. We had a continuous build, but skipped the full deployment build to ship at the end of an iteration; probably because we didn't have iterations. The scope was small, so we decided to use only very broad, coarse-grained stories and just work until they were done. We still had our rigorous verification protocol to pass, but it was a small project, how hard could it be?

As I said, we were meant to have it done in 3 days. We figured it would take three weeks. It should have taken three weeks.

It ended up taking three months.

The way to go fast is to go well

Our decision to abandon any pretence of working towards clean code was our undoing. Not making much of an effort at design or removal of duplication resulted in tight coupling and obfuscated code. It was effectively a write-only code base; the simplest of changes became all but impossible, including fixing what should be simple bugs or catering for clarifications to requirements. It was very difficult to find seams for testing bits in isolation, and changes would frequently ripple through the code. Our lack of rigour around project planning and story breakdown made it impossible to track our progress. Time saved by not automating repetitive tasks was re-spent ten-fold on mindless, error-prone tasks when we should have been writing code to finish the project.

Let me make this very clear: anyone that tells you working toward clean code is a waste of time and will slow you down is completely wrong.

You cannot trade internal quality for speed. I give you about 30 minutes of coding before you need to change something where an automated test would have made you faster. As you can read from the earlier post, I was pretty convinced of this already, but experiencing such a dramatic illustration by far surpassed my expectations in terms of the return on investment in clean code. Previously I would have guessed we'd start feeling pain after about a week of hacking, but it started almost immediately. I am now adamant it is impossible to have anything but an illusion of productivity without writing clean code.

As Uncle Bob puts it, the way to go fast is to go well.

Clean code is a journey, not a destination

None of the above means gold plating your code. Your code need not be a shining beacon of architectural and OO greatness, gazed upon with awe by artisans for centuries to come. Your code is never completely clean; clean code is not an end state, it is about the process of reducing coupling, reducing duplication, simplifying, and clarifying intention whenever you work with your code.

This does not mean you must practice TDD. It does not mean you need to use a particular language or framework. It does not mandate a project methodology. It doesn't mean you can't spike. It doesn't mean you can't knowingly take on manageable technical debt. It doesn't even mean you must unit test. It means that you need to do what it takes to ensure your code is maintainable. That your code has the qualities we associate with clean code: it works, it is maintainable, it is correct. It is not about trading quality for speed, because that is an illusion. It is about doing what it takes to produce what's required as quickly as possible, and that means you can't afford to write cruddy code.

In short, don't ship $#*!.

Conclusion

And so back to my resolution, which was always a high-priority goal of mine but never an unwavering commitment, never before a line I simply will not cross. I will never deliberately contribute or support the contribution of bad code to a project. I will always strive to write clean code, not in pursuit of some programming utopia, but in the sincere belief that it is the only way to quickly and effectively deliver a project.

I wish you all the best of health, happiness, and clean code for 2011. :)