Friday, 21 August 2009

Coding tiger, hidden responsibility

During a recent pursuit of the Single Responsibility Principle, I stumbled upon an interesting place a responsibility can hide. Here's our task: we want to write a PersistenceService class that will let us save some Results to a file. These results will be stored as a serialised array of integers (int[]).

I'm going to use a bit of an unusual approach for writing the array out to the file. Commonly we'd have a serialiser class that would open up a file (or ask a dependency to open up the file), write the serialised data, then close and dispose the file. Instead we'll invert control a little. We'll pass in an existing file stream, and the serialiser class will write to it. The type and lifetime of the stream will be managed as an entirely separate responsibility.

Coding tiger

public class PersistenceService {
    private IFileStreamer _fileStream;
    private IIntegerArrayDataSerialiser _serialiser;

    public PersistenceService(IFileStreamer fileStream, IIntegerArrayDataSerialiser serialiser) {
        _fileStream = fileStream;
        _serialiser = serialiser;
    }

    public void SaveResults(Results results, string filePath) {
        _fileStream.Write(filePath, stream => _serialiser.Serialise(results.GetFoos(), stream));
    }
}

Here you'll see we are using an IFileStreamer to manage the lifetime of the stream, and using IIntegerArrayDataSerialiser to manage the serialisation. Our PersistenceService class itself just coordinates those dependencies, which should be its single responsibility. Let's run through the two collaborators before we move on, just so we have a working code sample.

public interface IIntegerArrayDataSerialiser {
    void Serialise(int[] data, Stream stream);
}

public class IntegerArrayDataSerialiser : IIntegerArrayDataSerialiser {
    private IFormatter formatter;

    public IntegerArrayDataSerialiser(IFormatter formatter) {
        this.formatter = formatter;
    }

    public void Serialise(int[] data, Stream stream) {
        formatter.Serialize(stream, data);
    }
}

We'll create our IntegerArrayDataDeserialiser with a BinaryFormatter (an IFormatter from System.Runtime.Serialization.Formatters.Binary in the framework), and we are basically just wrapping this class. Finally, we have our FileStreamer:

public interface IFileStreamer {
    void Write(string path, Action<Stream> streamProcessor);
}

public class FileStreamer : IFileStreamer {
    public void Write(string path, Action<Stream> streamProcessor) {
        using (var stream = File.OpenWrite(path)) {
            streamProcessor(stream);
        }
    }
}

The FileStreamer has the single responsibility of managing the lifetime of the stream used to write the file. It is basically a creational responsibility. And, believe it or not, this all works just fine.

So, have you spotted the hidden responsibility? It's in the PersistenceService. If you have, then nice work! I didn't spot it at all, although a colleague was able to point it out to me later. What I did spot though was a sign of potential trouble, not from the code, but from the tests...

Testing times

How would you test the PersistenceService as implemented? Despite the simplicity of the implementation, it is quite messy. If you haven't noticed the responsibility have a go at writing a test and see what gives you trouble. Here was how I finally got the test to pass:

public class PersistenceServiceFixture {
  [TestFixture]
  public class When_saving_data {
    private PersistenceService sut;
    private FakeFileStreamer fileStreamer;
    private FakeSerialiser serialiser;
    private int[] data;
    private Stream stream;
    private string path;

    [Test]
    public void Should_write_serialised_results_to_file_stream_from_given_file_path() {
        Assert.That(fileStreamer.PathUsed, Is.EqualTo(path));
        Assert.That(serialiser.DataSerialised, Is.SameAs(data));
        Assert.That(serialiser.StreamUsedToSerialise, Is.SameAs(stream));
    }

    [SetUp]
    public void Context() {
        stream = new MemoryStream();
        fileStreamer = new FakeFileStreamer(stream);
        serialiser = new FakeSerialiser();
        data = new[] {1, 2, 3, 4, 5};
        path = "some path";

        sut = new PersistenceService(fileStreamer, serialiser);
        sut.SaveResults(new Results(data), path);
    }

    class FakeFileStreamer : IFileStreamer {
        private Stream stream;
        public string PathUsed;

        public FakeFileStreamer(Stream stream) {
            this.stream = stream;
        }

        public void Write(string path, Action<Stream> streamProcessor) {
            PathUsed = path;
            streamProcessor(stream);
        }
    }

    class FakeSerialiser : IIntegerArrayDataSerialiser {
        public Stream StreamUsedToSerialise;
        public int[] DataSerialised;
        public void Serialise(int[] data, Stream stream) {
            DataSerialised = data;
            StreamUsedToSerialise = stream;
        }
    }
  }
}

I had to manually fake out both dependencies. I needed to put some behaviour in my FakeFileStreamer so that it would call our Action<Stream> function that the PersistenceService passes to it from the IIntegerDataSerialiser. By exposing the arguments used to call our FakeFileStreamer and FakeSerialiser we could then assert on them within our test. Not exactly pretty, is it?

What we'd like to do is to use our trusty mocking framework to generate an IFileStreamer and IIntegerArrayDataSerialiser, stub out a value and assert that the stream was written to.

Unfortunately we can't do that, because of a lurking menace...

Hidden responsibility

Let's take a closer look at the PersistenceService.SaveResults() method:

    public void SaveResults(Results results, string filePath) {
        _fileStream.Write(filePath, stream => _serialiser.Serialise(results.GetFoos(), stream));
    }

This is simply coordinating the class' two dependencies, right? It calls the Write() method of our IFileStreamer, and passes in the Serialise() method of our IIntegerArrayDataSerialiser. That coordination is one responsibility. So what is making this so annoying to test? My bearded colleague was able to walk me through the following steps to reveal (and subsequently fix) the problem. (This post is my translation of his explanation. His explanation was great, any mistakes made in this account are mine. :))

Well, firstly, we'd really like to be able to assert something like this in our test:

fileStreamer.AssertWasCalled(x => x.Write(path, stream => serialiser.Serialise(results.GetFoos(), stream));

That way we could be sure that our PersistenceService was producing the expected call to our fileStreamer, passing in the correct pointer to our serialiser.Serialise() method. So what's stopping us? Well the AssertWasCalled() method on Rhino Mocks will check the arguments for the call. It will get the path right, but we are creating a new Action<Stream> delegate to pass in our serialisation function, so we can't compare it to the one used by the PersistenceService.

Did you notice that? I didn't, despite hours of looking :). We are creating a new delegate. If we had that delegate instance we could compare the arguments in our test. But we don't, because of this line in PersistenceService:

    public void SaveResults(Results results, string filePath) {
        _fileStream.Write(filePath, stream => _serialiser.Serialise(results.GetFoos(), stream));
    }

That simple stream => _serialiser.Serialise(...) lambda declaration is creating a delegate instance. So now PersistenceService isn't just coordinating dependencies, but it is creating a new object! That's two responsibilities! We are violating SRP, and that's what our test is telling us.

Resassigning responsibilities

My bearded colleague suggested we move the responsibility for creating this serialisation delegate. Instead of the PersistenceService creating it from our IIntegerArrayDataSerialiser, he reasoned, why not just get the IIntegerArrayDataSerialiser to create it for us? After all, serialisation is its job, and for serialisation our current design needs an Action<Stream>. (Although it is not really an IIntegerArrayDataSerialiser any more, we should probably rename it to a IIntegerArraySerialisationFactory or something. We'll keep it as is just to keep things simple -- its hard enough to follow code samples on a blog without following renames too.)

public interface IIntegerArrayDataSerialiser {
    Action<Stream> GetStreamSerialiser(int[] data);
}

public class IntegerArrayDataSerialiser : IIntegerArrayDataSerialiser {
    private IFormatter formatter;
    public IntegerArrayDataSerialiser(IFormatter formatter) {
        this.formatter = formatter;
    }

    public Action<Stream> GetStreamSerialiser(int[] data) {
        return stream => formatter.Serialize(stream, data);
    }
}

Notice this class' responsibility is clearer now. Before it was pretty much just a wrapper around IFormatter. Now its responsibility is creational: it is creating a delegate for serialisation from its IFormatter. Let's see how this affects the rest of the code. Our FileStreamer can remain unchanged, so now we just need to update our PersistenceService to call the new factory method on our IIntegerDataArraySerialiser.

public class PersistenceService {
    /* ... snip ... */
    public void SaveResults(Results results, string filePath) {
        _fileStream.Write(filePath, _serialiser.GetStreamSerialiser(results.GetFoos()));
    }
}

Now let's see if this makes everything easier to test.

[TestFixture]
public class When_saving_data {
    private PersistenceService sut;
    private IFileStreamer fileStreamer;
    private IIntegerArrayDataSerialiser serialiser;
    private int[] data;
    private string path;
    private Action<Stream> streamProcessor;

    [Test]
    public void Should_write_serialised_results_to_file_stream_from_given_file_path() {
        fileStreamer.AssertWasCalled(x => x.Write(path, streamProcessor));
    }

    [SetUp]
    public void Context() {
        streamProcessor = stream => { };
        data = new[] { 1, 2, 3, 4, 5 };
        path = "some path"; 
        fileStreamer = MockRepository.GenerateStub<IFileStreamer>();
        serialiser = MockRepository.GenerateStub<IIntegerArrayDataSerialiser>();
        serialiser.Stub(x => x.GetStreamSerialiser(data)).Return(streamProcessor);

        sut = new PersistenceService(fileStreamer, serialiser);
        sut.SaveResults(new Results(data), path);
    }
}

A bit simpler, no? We can now get rid of our hand-coded test doubles and the logic they contained. And now we have access to the Action<Stream> delegate instance that was giving us trouble before. As a result, our assertion (fileStreamer.AssertWasCalled(...)) is very straight forward and is checking that our PersistenceService did exactly what it was meant to.

Now this was mainly done as a coding exercise, so I'm not advocating this design at all, but I do find the realisation that an anonymous delegate or lambda is actually a creational responsibility really useful. It's definitely going to help me divide up responsibilities better in future.

Nothin' but .NET, Sydney 2009: Final thoughts

It's been almost two weeks since the Sydney edition of Nothin' but .NET finished. I've blogged my thoughts from each day of the course, but the time since the course has given me a chance to reflect on the whole event and has provided me with some much needed perspective. Please excuse me while I rant, but I need to get this off my chest.

Note: this post contains little to no technical content and is filled with disgusting sentimentality. This post may induce feelings of nausea, boredom, or the desire to unsubscribe from this blog. We recommend that, if you choose to read this post, you do so with a vomit bag or bowl in close proximity. Do not ingest this blog post. This post is for external use only. This post is highly flammable. This post may contain traces of nuts. If you are pregnant, or think you might be pregnant, then congratulations! This post may cause hair loss, wind, bloating, itching, convulsions and/or the desire to listen to Rick Astley. Using this post for the disposal of human remains may be a violation of local health ordinances. You have been warned...

A brave new world

By far the most important, exciting, daunting, confronting and demoralising thing that I found out from the course is that I know nothing about what Object Oriented design is really about. I know a couple of design patterns, SOLID, a bit of GRASP, and OO techniques like polymorphism and inheritance, but I've never put it all together in the way JP demonstrated in class.

The average size of each class was around 20 lines of code, including noise like whitespace, braces on new lines, namespace declarations etc. Each class had only one responsibility which was concisely implemented and easy to understand. This made testing really pleasant -- there was no time spent fighting the design, it all just fell into place. And this wasn't just for stuff that JP had done before, this kind of design emerged no matter what we threw at him.

My first reaction was, somewhat embarrassingly, a feeling of complete and utter incompetence. I've been doing OO programming professionally for a decade or so, and all that time I've been doing it wrong! This was quickly followed up by a feeling of demoralisation: now that I'd seen what OO design is meant to look like, I still couldn't pick up the skill to do it myself.

But somewhere behind my fairly pathetic self deprecation I was excited. This was amazing! This was what I'd been desperately trying to work out for myself for years, and it was using all the tools I already knew about!

The best way I can describe it is with a classic DaveSquared contrived analogy (TM). Say you've spent the entirety of your life in one house. Never been outside. You get by ok, but you've always felt that you were missing something. So you've kept tweaking the rooms, repainting the walls, rearranging the furniture. Then one day someone comes along, opens the front door and shows you this entire other world you never knew existed. While the basic laws of physics work the same between your house and this new world, it's full of all these complicated things that fit together in ways you can't even begin to comprehend: people, cars, trains, people, planes, shops, people, towns, countries, commerce, politics, laws, education, planets, galaxies, quantum mechanics, twitter etc. It's confronting, it's daunting, and you feel like a complete idiot for never noticing it before. But it is also brimming with exciting opportunities.

The journey

Being shown that this other world of OO design existed was invaluable, but I still had no idea how to actually work with it. I knew the basic values and concepts, and could understand JP's explanations, but I just couldn't apply it all when it came time to code.

Initially this was a huge source of disappointment for me. My expectation from the course was that I would walk out of it a better developer. But come the end of the final day I found I couldn't do a single thing any better than when I had walked in. It seemed all I was left with was a few bits of knowledge I couldn't use, and the feeling that I was a much worse developer than I initially thought (and, as people who know me can probably confirm, this is coming off a fairly low opinion of myself already! Quite an achievement!). This is quite a realisation to face up to when you've spent around 80 hours with a top notch developer and your company has shelled out a huge amount of money for the privilege.

Part of the reason for this was that JP, as he pointed out several times during the week, is a developer, not a teacher. He simply demonstrates this awesome stuff, and answers any questions you can throw at him. It's kind of up to the attendees to extract as much from the week as they can. But demonstrations alone (IMO) don't really help build you into a better developer. That's sort of left up to you. Another major contributing factor was that I was probably well out of my depth in terms of my existing knowledge.

But I think the most important factor was a matter of misaligned expectations. I don't think I was really meant to come out of the course a better developer. I was meant to be pointed in a direction, given some tips as preparation for my journey, then sent off on my way to become a better developer.

Since the course ended I've had a number of surprises. First up was just how much stuff I managed to learn during the course, which I only realised as I worked through posting summaries of each day (which incidentally is the reason I blog -- it helps me learn stuff and collect my thoughts :)). Sure, I couldn't figure out how to apply much of it, but the sheer amount of really important revelations I got was amazing.

The second surprise took me completely off guard. One week after the course, after the sleep deprivation and system shock were beginning to wear off, I started on a new bit of code at work. And I started writing tests, top-down. And I could divide the responsibilities fairly nicely. The abstractions weren't perfect, but the design was neat. The classes were small. It all just started to flow. All the stuff I failed at during the course had started to fall into place. I could apply some of this stuff! A colleague of mine who had also been on the course started pairing with me, and he started churning out some awesome code. We started talking about the design in terms of the stuff we had learnt, and it was making sense to us.

Now this was just a small win, but a win nonetheless. I know I'm going to keep struggling with this, but less than two weeks after the course I felt I had started to make some progress -- a single, small step -- on my journey to become a better developer.

And it wasn't until that point that I truly understood what JP was saying on Day 1 about competing with and comparing yourself to other developers. He was stressing that you shouldn't look at a developer and think "I want to be able to do that!". By the time you can eventually do "that", the developer you admired has moved on to a better way of doing it. And even if you do overtake him or her, there will always be someone else to try and catch up to. If you look at it like this, then you'll always be behind. What you do will never seem good enough. And that's a fantastic way to sap all the joy out of coding. And let's face it, the reason most of us are developers is because we love to code, so once you lose that you're in trouble.

Instead, JP recommended trying to improve yourself just a little bit each day, and focus on enjoying coding. No comparisons, no competitions. The sum of all these little improvements over the years can be great, but more importantly the enjoyment and the satisfaction you get from continually improving can be life changing.

Now I'm probably one of the least competitive people you're ever likely to meet, but the one thing I do compete mercilessly against is my own expectations for myself. When I felt like I was failing to perform in the course as I expected, that was rough. But now I can see that what JP was really getting at was that I shouldn't expect to be able to do all the stuff he was showing us. I just needed to start the journey.

So my initial disappointment from the course has been turned around now. I was shown this amazing new world of OO design. It's possible I could have stumbled across it myself I guess, but the course pointed out, in fairly dramatic fashion, a direction in which to head. I learnt some really important coding tips that can help me along the way. And, now the course is done, I've finally started my journey -- and it feels awesome. :)

Wednesday, 12 August 2009

Nothin' but .NET, Sydney 2009: Day 5

The fifth and final day of Nothin' but .NET kicked off a bit later than normal (once the breakfast chats were over it was closer to 10am, but wound up around 1:45am (I think, my brain was well and truly fried by that point).

Tell, don't ask

We talked for quite a while on this. The idea is to push responsibilities on to the class that owns the data related to that responsibility. The most common violation of this principle is an object acting on data it asked another object for. For example, instead of having if (game.genre == "Platform") { /* do something */ }), we should push the do something code into the game itself. That way we are telling the game to do something, rather than asking it for information and doing it ourselves. Violating tell don't ask strips our objects of behaviour and leads to things like anaemic domain models.

Testing

I think I've rolled most of the testing stuff from the course into previous posts, but I'll rehash some of the stuff that came up on day 5 about using tests to drive design.

Again I saw that the context/SetUp of the scenario/test seems to drive most of the design. The way the context is setup determines the responsibilities and API for the SUT's collaborators and dependencies. By the time they become the SUT and their assertions are being written most of those decisions are made. In that case the only remaining decisions are the design of dependencies' dependencies. :) It is this relationship that really lets us use the tests to drive the design from the top down.

The "because" block is the API generated from the assertion and the context of the higher level component's tests. For example, if a test for our FrontController asserts that command.AssertWasCalled(c => c.Run());, then the "because" blocks of our tests around Command implementations become command.Run();. We get a similar result when we stub return values for dependencies while setting up the test context: we need to write scenarios around those calls.

Here's a pseudo-code example. Say we have a test for an HttpRequestHandler class which asserts that frontController.AssertWasCalled(fc => fc.Process(request)). We can then write tests for the FrontController around that scenario:

When the FrontController is told to process a request:
  It should run the command able to handle this request:
    command.AssertWasCalled(c => c.Run());
  Because:
    sut.Process(request);
  Context:
    request = MockRepository.GenerateStub<Request>();
    command = MockRepository.GenerateStub<Command>();
    commandRegistry = MockRepository.GenerateStub<CommandRegistry>();

    commandRegistry.Stub(c => c.GetCommandFor(request)).Returns(command);
    sut = new FrontController(commandRegistry);

This scenario is telling us that when our FrontController is told to process a request, it needs to Run() a command. This is its sole responsibility. Note how well this responsibility is summarised by the scenario and test name, "When the FrontController is told to process a request, it should run the command able to handle this request". In our context/SetUp method, we've made a design decision to have a CommandRegistry responsible for mapping a request to the right command. We'll then need to drive out the behaviour of what happens when the CommandRegistry.GetCommandForm(Request r) method is called.

I also noticed there a two different styles of TDD: incrementally driving the design vs. incrementally driving the implementation. JP's "simplest thing that makes sense" approach (see Day 3 wrap up) tends to focus the developer on driving the design, whereas sometimes the "simplest thing that works" can lead me to procedural thinking (first it should do this, test, refactor, then it should do that, test, refactor...). I'm sure this is due to me misusing TDD in this manner, but I'm reasonably confident I'm not the only TDD novice to fall into this trap of driving implementation from the tests instead of design.

Other notes from Day 5

  • With a FrontController architecture Commands are similar to MVC actions.
  • Pipelines (Pipes and Filters pattern) can be a good way to get into event-driven and message-passing architectures.
  • Moving away from layered and onion architectures to component layers that are loosely affiliated. The direction of communication can be guided by the Dependency Inversion Principle and specific requirements (like having Query objects accessible from anywhere via their interface, but consumed only below the service layer).
  • Query object pattern (as used for NHibernate's Criteria and DetachedCriteria). Related to the Specification pattern.
  • Collaborators for a test can be injected into the SUT, retrieved from another dependency, or retrieved from a Static Gateway (although be careful with that last one).
  • Don't bother creating interfaces for DTOs.
  • The Service Layer is responsible for unwrapping DTOs packaged by higher layers. DTOs shouldn't go lower down than that, they are strictly for communication between the Service Layer and higher levels.
  • Had a whirlwind tour of some Domain Driven Design (DDD) concepts.
  • Separating data updates using repositories from queries using Query Object pattern. Don't just dump create/read/update/delete functions on a repository. Enforce Command Query Separation (CQS).
  • Command and Visitor design patterns are really under-used and under-appreciated.
  • Introducing Pure Fabrications over primitives to help encapsulate behaviour and make writing aggregates and entities easier. An example, instead of using an IDictionary<Product,int> to map products to quantities in a shopping cart, introduce a CartItem fabrication that can be used to track both and encapsulate useful behaviour for the Cart aggregate.
  • Concept of Shu Ha Ri for describing the stages of learning.
  • if and for (loops and conditions) are a tad evil. Getting rid of them makes code nice. :)
  • Problem decomposition is more important than patterns.
  • Always look for the higher level of abstraction. Step back from the details of the problem, and tackle the abstraction instead.

Monday, 10 August 2009

Nothin' but .NET, Sydney 2009: Day 4

Day 4 of Nothin' but .NET started before 9am and finished after 2:45am (including eating breakfast and dinner in the conference room while still discussing stuff).

The day's activities

We started out talking about IoC containers, and their role in controlling object creation, lifecycle, autowiring dependencies, and also in dynamic interception for AOP (using Reflection.Emit, RealProxy from System.Runtime.Remoting.Proxies, Dynamic Proxy etc). Then it was up to us to implement a simple container. The class started off trying to convert the poor man's dependency injection approach we had used to quickly hack everything together. Xerx made a great suggestion that we could simply use Func<object> delegates as factory methods, and just map requested types to those methods.

Once we had configured all the mappings using our container and removed all the no arg constructors we had used for poor man's DI, the next challenge was to drive out a fluent interface for application startup. Here I had my first and only success of the week, where I actually managed to test drive out a few classes without doing my normal trick of becoming hopelessly stuck. JP needed to make a few changes to it but it seemed like I ended up fairly close to a reasonable design. I finally felt like I might be making some progress.

We were going to end the day by chatting about and implementing the domain and the service layers, but we started to lose some attendees due to illness and exhaustion, so we decided to defer the domain stuff until the morning (or, technically, later that morning). We did end up having a quick chat about service layer styles as described in Martin Fowler's PoEA, and contrasted the Transaction Script approach (i.e. a big ol' procedural method or script) to keeping a thin service layer over a domain. A couple of us worked 'til the end of class implementing some of the front controller stuff we had skipped from earlier in the course.

In contrast to previous days, after Day 4 I felt like I was finally on track to becoming a better developer. (Spoiler alert: it wasn't to last :-\)

Writing tests

I continued to build on the testing concepts I had started uncovering on previous days, although I still struggled to apply all them. I covered a lot about trying the "simplest thing that makes sense" instead of the "simplest thing that works" in my summary of Day 3, and this continued to be an important theme for Day 4.

I finally started to get an appreciation of the impact which each part of a test definition has on design. The scenario name, test case name and assertion became the fundamental behaviour and purpose for the SUT's existence. The "because" block showed why the SUT was exhibiting this behaviour (a call to a particular method). The context/setup was then used to drive the design of the SUT's collaborators and dependencies and dole out their responsibilities, as required for the SUT to do its job. This then helped us get down to the next level of abstraction.

JP seemed to use the SUT's dependencies, as setup in the test definition, as axes by which the overall design could be varied. By decomposing the problem the SUT is trying to solve into sub-responsibilities, then pushing these responsibilities down into dependencies, the SUT stayed very clean, small and simple. Finding the right abstraction for these responsibilities (in particular, programming to the API you would like them to have) made it easier to design these dependencies once they became the SUT. Any pain, duplication or smells detected while writing tests became a clear sign that we needed to look for a different abstraction around the SUT's dependencies. For example, instead of injecting in a dependency with a required behaviour, we might need to inject a factory or other dependency that would figure out the behaviour needed and return the relevant dependency.

I quizzed JP to try and find out the process he used to make all these design decisions which he seemed to effortlessly drive out while writing the tests. A lot of it seemed to come down to the context he has built up over the years by experience, and (somewhat unfortunately for me) down to an amazing talent for spotting and thinking in abstractions. Still, here is the best approximation of his approach that I could come up with:

  1. What responsibility does this SUT have? This becomes the scenario name.
  2. Decompose this into sub-responsibilities. If the SUT is responsible for running a command, then it cannot also be responsible for creating or locating that command. This becomes a responsibility of the next level down the abstraction chain.
  3. Identify collaborators/dependencies required so we can push these other responsibilities to them, rather than burdening the SUT with doing too much. If the other responsibility is creating a command, we might make a design decision to use a factory to do this.
  4. How should the SUT react under this scenario? The description of this becomes the test title. JP often used long, descriptive titles loaded with design implications. For example, the SUT "should call the run method on the command returned by the command factory". This became a broad overview of the design intention and design decisions made, with the details fleshed out in later steps.
  5. How do I assert this? Write the test case body in one logical assertion, as simply as possible. It should reflect the fundamental purpose of the SUT's existence.
  6. Why does this happen? Write the "because" block, which is basically the method call that triggers the behaviour.
  7. Work out the context/test setup. Setup the collaborators. During this time you'll be making design decisions about the responsibilities and behaviours of the dependencies.

Today's tidbits

Here's a quick round up of some other miscellaneous things I picked up on Day 4:

  • I need to write more code. Lots of code. Tonnes of code! Anything to get some practice in and get more context for my design decisions. Try tackling the same problem with completely different techniques. Try functional style. Try not using dependencies. Try different patterns. Try writing tests differently. Try having no tests. Regardless of the result it will help grow your context on which to base future design decisions.
  • Top down design is incredibly powerful for driving down from the required behaviour to the lowest levels of abstraction. Because test-driving each SUT in turn drives out the design of its dependencies, JP ended up with the problem broken down into incredibly simple abstractions. I'm not sure how you could come up with that using bottom-up design, as then your tests don't really give you any feedback on what the design of the higher level code should be.
  • Three essential abilities for OO design: problem decomposition, finding abstractions, and segregation of responsibilities. I have no idea how to learn the first two, but GRASP can probably help with the third.
  • Using dependencies as axes to vary the design of the SUT.
  • Keep to one logical assertion per test.
  • Test context/SetUp can hold design decisions and indirect assertions (e.g. stubbing a return value which is used in the test assertion is a design decision).
  • JP said not to think too far ahead, because "it will kill you". Keep at the current level of abstraction, and break the problem down. Don't think "what if?", think "what now?".
  • Concentrate on writing tests around the "happy" path. This helps to focus on the SUT's single responsibility, and defer exception handling etc. to different levels of abstraction.
  • Focus on outside-in testing using dependency injection. By passing dependencies in we can assert their state or calls made to them by the SUT, therefore making previously untestable internal state testable.
  • Static gateways may need direct access to Service Locator (IOC, or abstraction over our container).
  • Orchestrator pattern, an object that takes care of the sequence of operations in a pipeline.

Saturday, 8 August 2009

Nothin' but .NET, Sydney 2009: Day 3

Day 3 of Nothin' but .NET was a bit more laid back than previous days as we spent most of it in teams trying to apply some of the stuff we had learned. We had an early finish (10:45) in preparation for a longer day on Thursday.

I actually found this day immensely frustrating. After seeing so much cool stuff the previous two days and feeling like I was starting to understand the main concepts, it was completely demoralising to be given a fairly basic problem and utterly fail to make even the vaguest hint of progress with it. The only reason I didn't feel completely incompetent was because of the overwhelming feeling of stupidity I had, and it didn't feel quite right for someone so obviously stupid to think of a big word like "incompetence". :\ :)

This wasn't helped by the fact that as soon as JP started demonstrating how to proceed to the next stage, he made the solution seem so obvious and effortless. At the start of the course JP went to great lengths to encourage us not to compare ourselves to or compete with other developers, and that instead we should just aim to better ourselves one small step at a time. But in this case comparing JP's work to mine wasn't like comparing the work of two developers. It was more like comparing the works of Leonardo da Vinci to those of a small, under-watered cabbage. It was fairly difficult not to notice the difference.

Despite this the under-watered cabbage did manage to pick up a few things from this day. First up I got to see an end-to-end, test-driven development of a Front Controller architecture for processing web requests using Commands. The Front Controller itself was more of a component -- it consisted of several classes all grouped together to perform the front controller related tasks. It's a bit like a "layer", although the architecture wasn't really layered in a traditional sense. It was just a bunch of components working together in a pretty loosely coupled way. Not having a more traditional layered approach seemed to make the design much more flexible.

Note: As a source of examples for the following sections I'll refer to a specific class within the Front Controller component -- the slightly-confusingly named FrontController implementation (confusing because it is just one part of the entire Front Controller component). This class's responsibility is to receive an IncomingRequest (an abstraction of an HTTP get or post) and run a Command that will do whatever this request is asking.

I also picked up a couple of TDD tips. The first was to start with a test case that reflects the simplest, most fundamental description of the subject under test's (SUT's) behaviour, rather than asserting little facts about the SUT. For example, when writing a test for when our FrontController is handling a request, we shouldn't start by asserting it gets a non-null Command from its CommandFactory, or assert that CommandFactory.Create() was called on a mock object. Instead our test was that it "should tell the command that can process the request to process it".

As I started picking up on later in the week, the former is really focussed on the mechanics of an implementation, while the latter is about the required behaviour. By being very descriptive about the behaviour, we end up driving out a lot of design in that one statement (in this case, the design decision is that we need a collaborator that can return a command that can process a request, and that our SUT will run that command). This technique also encourages the use of a very simple, targeted assertion in code, which lets us defer design decisions about the SUT's dependencies until we start writing the context/SetUp method for the test.

This really comes down to the identification and segregation of responsibilities (as per the Single Responsibility Principle (SRP)). This process is really helped by attempting to identify the most abstract responsibilities beneath the SUT, while ensuring the SUT is still responsible for adding some behaviour. Or, put another way, the SUT should have only one small part to play in achieving it's overall reason for existence -- this is its single responsibility. Everything else is deferred to the implementation of its collaborators.

Another TDD technique JP used was doing the "simplest thing that makes sense", instead of the "simplest thing that could possibly work". For the FrontController example, the simplest thing that could work when initially coding it would be injecting a single Command into the FrontController and asserting that the command was run. Then what? Move on to another SUT and leave a very defficient implementation of our FrontController? In this case, we know with absolute certainty that this class will need to process more than one type of Command, so the simplest thing that makes sense is for the FrontController to get a relevant command implementation from a CommandRegistry or similar type, rather than hard-wiring in a single command. This not only gives our FrontController less reasons to change (as per the Open Closed Principle (OCP)), but it also points out the next SUT we can drive out, the CommandRegistry implementation.

Aside: In retrospect, we could potentially get the same design in a more incremental fashion by still doing the simplest thing that will work but being especially diligent during the refactoring step of TDD. Just say our first attempt was to inject a single Command and assert it's run method was called. The test passes, and the next step is to look for refactoring opportunities. We notice the OCP violation, and refactor to introduce the CommandFactory, all under the protective cover of our passing test. I have a suspicion that it might be more reliable to think about the problem in abstract terms and have a SOLID design naturally fall out than to do the simplest thing that could work, and then run it through the gauntlet of SOLID principles, GRASP patterns etc. Still, I find it comforting that if I miss the abstraction up front I still have a chance to get there via refactoring.

Other tidbits

Here is some other stuff that came up during day 3:

  • A long context/SetUp for a test is a test smell that indicates we're probably doing too much. Push some of it down into other collaborators so we can defer decisions about it.
  • Creating something is a responsibility. If a SUT needs to create something and act on it, then the creation should probably be pushed out into a Factory. The SUT is then only responsible for using the factory and its output (i.e. mediating between the two types).
  • When designing and writing tests, focus on abstraction and intention rather than focussing on implementation and mechanics. Yes, I've written this already in this post. No, I probably haven't stressed it enough.
  • If stubbed values are required for a test to work then these are tested implicitly when the test runs. In our FrontController example where it uses a CommandRegistry to get a Command and call its Run() method, we don't need to explicitly test that CommandRegistry.Create() was called. Instead we can stub out Create() to return a specific Command instance, and assert that its Run() method was called. We don't need to explicitly assert that the factory was used if the test's assertion already depends on it. This is a side-effect of identifying the simplest, fundamental assertion for a SUT rather than thinking about the implementation mechanics.
  • By constraining ourselves to one ViewModel per View we can use convention over configuration to wire everything up.
  • Went through the concept of a Service Layer, which is a type of Facade for operating on the domain.
  • Went over Command Query Separation (CQS)
  • Covered Separated Interface, where an implementation lives at a lower level than the interface itself. An example is a Query interface which is defined somewhere with visibility from all layers/components, but the implementation within the domain or by the ORM or persistence layer/component. This is why its ok to use NHibernate's criteria interfaces from pretty much anywhere within the application, provided the implementation is abstracted appropriately into a lower layer. The Dependency Inversion Principle (DIP) is used as a guide for applying the Separated Interface pattern (i.e. higher levels should not depend upon lower levels, but instead on interfaces using the Separated Interface pattern).

Wednesday, 5 August 2009

Nothin' but .NET, Sydney 2009: Day 2

Day 2 of JP Boodhoo's Nothin' but .NET bootcamp has come and gone. JP's been aiming for an 11pm stop, but we went through until about 12:30am. Which was awesome, because we had started covering some really interesting stuff. Apologies in advance for the rambling nature of this post, but most of it was done after 2am. :-\ I'll try and distil this stuff into some decent posts after the course.

The main highlight I took away from the day was finally identifying the source of and solution to a lot of the problems I have when designing OO systems. Whenever I've tried test-driving a solution from top-down I've commonly found my tests seem to raise more questions than they answer. I'd struggle through writing one test, and have to almost-arbitrarily whack in a number of dependent classes that I'd need to contort in some strange way via mocking to get the assertion to pass.

Turns out that this was on the right track (driving out dependencies), but for the wrong reason. I had been looking for any form of collaborator for my subject under test (SUT) in a vain effort to find something (anything!) to test. It all felt pretty contrived. The reason I was struggling is for the same reason I identified on day 1: I suck at segregation and assignment of responsibilities. When you start thinking of things you want to test on your SUT, they should all relate to one behaviour/concern/responsibility (SRP). Anything else, no matter how trivial it seems, really needs to be pushed into a collaborator. (The GRASP patterns can really help in identifying what these collaborators should be.) This collaborator will most of the time need to be accessed via an interface, not a concrete class (DIP). This let's you drive out the intention behind each collaborator, without needing to fill in the shape, structure or implementation of them.

This approach gives lots of very small, very focussed classes. It also produces very focussed interfaces for each collaborator (ISP). When it's done right it also means that when a responsibility needs to be added, it can generally be done without modifying existing classes, but instead producing another implementation of one of your collaborator interfaces (OCP).

You'll notice I've littered a whole lot of TLAs (Three Letter Acronyms ;)) through those last couple of sentences -- that's my effort to tie these things back to the fundamental SOLID principles. Now I felt I had a really good understanding of SOLID, and I am very careful to consider any code I write in light of those principles, but I'd never taken them to their full, logical conclusion.

Now there is one big downside to the designs that come out of thinking like this. The object model is very, very abstract and complex. You can't just hold the entire model in your head at once. It is not immediately obvious how some input at the top layer of the application works its way through the web of myriad of incredibly simple objects to give you some output at the other end. And you know what? I'm dead certain that this is the absolute entire point of Object Oriented development.

Understanding the entire flow of an operation or application is the point of procedural programming. We know that if this value is x then it will go down this branch of an if condition, then that will call a method with this argument that will check some argument to call some other method... this is done in OO languages all the time. Sure there might be some OO niceties sprinkled around like polymorphism, composition etc but classes and objects can still end up being little more than glorified namespaces for organising functions.

By contrast if you use OO programming to its fullest, you lose that immediately-apparent result you get from a more procedural style, and instead you get a whole bunch of almost-endless abstraction. The point of abstraction is that you don't need to understand the entire model, just the bit you are working on. But the benefit you get is that it becomes trivial to understand that small piece and affect how it works via its collaborators, whose implementations are equally easy to understand once you have stepped down into the next level of abstraction.

Sure, it is hard to maintain context during huge leaps through abstraction layers, but again you don't really need the entire context. It seems to me that I need to let go of that procedural safety net of knowing all the complete paths through my application to truly start doing OO design right. And doing OO right means getting a new safety net -- having each piece of my application being trivial to understand, and almost as trivial to change its behaviour. As an aside, having these trivial components also makes your design very easy to test.

Patterns and principles

We also covered a few more patterns and principles today:

  • Static Gateway as a static entry point to a DSL/fluent interface.
  • A simple version of the Event Aggregator pattern for decoupling listeners and publishers. Basically you just have system-wide events published to and subscribed via an Event Aggregator, and then let multiple components respond independently to events of system-wide significance. This also makes it easier to handle exceptions that occur during callbacks than with traditional .NET event handling (which will generally halt an invocation of a MulticastDelegate). Also learned a cool way of using attributes to tie into the Event Aggregator (so members can be decorated to get called on a system-wide event). And found out about the System.ComponentModel.EventHandlerList class.
  • Collecting Parameter pattern.
  • Registry pattern for lookups.
  • Object Mother for creating and setting up unit test data.
  • The Front Controller pattern, the forerunner (?) to MVC and other separated presentation patterns. The Front Controller object becomes the entry point that maps inputs (e.g. from UI) into Commands, and uses these Commands to coordinate between Models and Views. I've probably got the details wrong, but we'll be doing more on this tomorrow.
  • Null Object pattern for eliminating null checks, including on basic things like event invocation.
  • The Highlander Principle: there can only be one. Translated, this means that related methods should all delegate to the method with the biggest number and specificity of parameters. This is particularly important for overridding.
  • Decorator pattern. I've sometimes been hesitant to decorate classes where it only does something very trivial to the calls it delegates too, but this is really the entire point. Sure you have to reproduce several methods, but the power it gives you to augment behaviour while conforming to OCP has made me love this pattern again. :)
  • We went through the Dependency Inversion Principle.

Miscellaneous stuff

Some other things I jotted down from today:

  • You can put generic constraints on delegate definitions (this is pretty obvious in retrospect, but I hadn't thought about it before).
  • We went through the difference between the Delegate type and the delegate keyword.
  • The idea of closing a lambda/anonymous method down to a known delegate type.
  • Const fields are copied to other assemblies when compiled, so updating the original assembly won't change the values in the other assemblies until they are recompiled.
  • Closures can be replaced by a class with state and then pointing to a function of that class (which is what the compiler generates for closures anyway).
  • Referring to passing delegates around as "passing behaviours".
  • Naming test contexts very simply helps to isolate responsibilities. Any behaviour not specified in the tests for a SUT generally means that those concerns are delegated to dependencies.
  • When writing tests/specs, focus on the happy path where everything goes right. We can then choose to defer exceptions to this path to a dependency, or to flesh out the behaviour in later tests/specs.

Tuesday, 4 August 2009

Nothin' but .NET, Sydney 2009: Day 1

Wow. Just finished day one of JP Boodhoo's Nothin' but .NET bootcamp. I've found it pretty tough so far. I can generally follow along ok but start to flounder as soon as we have to pick up the code and start adding to it. The exercises have been pushing me far beyond the level I normally work at, but that's pretty much the idea. :) I don't think it would help much to cover everything we did (you kind of have to go through it), but I'll try and outline the main points that stick out in my memory.

First, generics and delegates. Turns out I know nothing about them. I mean, I know what they are, how they work, and even some ways to use them, but I don't really know them. Generally it's been a LINQ call here, a simple function pointer there, and an occasional generic class or method. Until today I'd never seen generics and delegates used and combined in such a way as to permeate through an entire design as off-handedly as most of us use int and string. JP just slung this stuff together like it was Hello World, while the rest of us hung on for dear life. Each individual part seemed to make perfect sense once I had seen JP's examples, but trying to code it for myself left my undersized brain desperately trying to hold on to information like the type of a method that had a generic type PropertyValue in a class that had a generic type Item that took a Func<Item,PropertyValue> and a Criteria<PropertyValue>.

Hot on the heals of revealing my inadequacy in that area, I also discovered my knowledge of responsibility assignment and problem decomposition is sorely lacking. JP made it look effortless as he stepped through implementing a fluent interface, starting with creating a static method that returned a factory or builder for our DSL, then driving down into separate classes that would handle each phrase in our grammar. I got some good tips on how to get better at this: look back over the GRASP patterns and focus on identifying and separating out responsibilities. One interesting technique I'll have to try out is over-zealously applying the Single Responsibility Principle (SRP), then potentially "de-factoring" some of the code if it is overly abstracted.

We also got some practice identifying a whole raft of design patterns which cropped up while going through the exercises. We had Adapter for adapting a Predicate<T> to a Criteria<T>, Factory for creating Criteria instances, Composite for creating potentially infinite chains of IComparator<T> references, Iterator both as a custom implementation and as built into .NET with the yield keyword, and Decorators for negating Criteria. It was great to see these naturally emerging from the application of the SRP, rather than as a deliberate refactoring toward a pattern.

I also had a long-held suspicion all-but-confirmed today. TDD/BDD can help inform your design decisions, but if you don't know good design principles up front then you are pretty much stuffed. I was sort of holding out hope that TDD can somehow teach good design, but I think I've given up on that. TDD helps point out some inadequacies in design approaches, like not using dependency injection, not adhering to SOLID principles etc., but you really need to know this stuff before TDD becomes really effective. So what makes me think that? Well, today we didn't use TDD at all, and I'm almost positive I would not have come up with the approach we did had I been using TDD. My design-fu just is not strong enough. The goal is clean code that works. This is a result of the good application of design principles. TDD can help you apply the principles, but you need to know them first.

Even the small amount we've done so far (today was apparently the "easy day") has opened my eyes to some areas I really need to improve upon, and given me some ideas on how to start doing so. It's definitely given me lots to think about for the next time I'm tackling a problem, especially around responsibility assignment. Tomorrow we start on the real work, including introducing TDD/BDD. I'm really interested to see the how TDD affects how we go about segregating responsibilities.