Friday, 16 December 2011

What do you enjoy about programming?

Over the last couple of weeks I think I've identified one of the main things I enjoy about programming. For me, it seems to be the puzzle-like aspect of it; finding ways to slot small bits and pieces together to create/achieve something bigger. When I'm working on a problem I can approach in this way I find it much easier to get into a good flow where I'm productive, creative and happy.

This idea started to dawn on me while I was in Joshua Kerievsky's Refactoring workshop at YOW 2011. We were going through refactoring exercises, but trying to take extremely small steps to keep the tests green. The final goal could be achieved fairly easily by moving a few chunks of code around while keeping the tests broken for a while, but I really enjoyed the challenge of finding small steps that would keep the tests passing all the way to the final state.

This was further reinforced at Tony's excellent Functional Programming workshop, where we spent quite a bit of time trying to fit together various functions (and partial applications of them) according to their type signatures. I thoroughly enjoyed this process, despite really struggling to get the solutions much of the time. I also realised I get an extra kick out of these puzzles when there is an element of elegance or simplicity to the way these pieces fit together. For example, implementing map with folds rather than recursion and pattern matching.

I think identifying what you enjoy about programming (or any activity really) can be an important part of maintaining motivation, and improving productivity and creativity as a result. I'm going to put this idea to the test by trying to break down problems in ways that I can use approaches I enjoy, and see if that helps me get better results, both in terms of solution and my happiness and enthusiasm.

This knowledge also helps me better understand some of my weaknesses. I have spent a lot of time over the years trying to piece together various patterns and practices to try and find a process I can continually apply to create great software. In other words, trying to solve the puzzle of how to develop software. My propensity to approach things as puzzles has probably led me astray here, as the unlikelihood of a silver bullet for software development has been demonstrated time and time again. Recognising this, I'll try be more careful to make sure I'm solving the right problems, and not just inventing problems to puzzle-solve (although that can be educational as well, provided it's deliberate).

So, what is it about programming you enjoy? Can you use that knowledge to improve the way you work?

Tuesday, 13 December 2011

Dave's not so excellent typing adventure

A few weeks ago I decided to depart from my usual QWERTY use and try an alternate keyboard layout. Rather than go the moderately rare Dvorak layout, I decided to go full-hipster and try Colemak. This post describes the ensuing hilarity and (mis)adventure.

Why not QWERTY?

I type reasonably well with QWERTY: normally between 90 - 110 wpm. I started doing some practice to try and improve my accuracy and technique and noticed a few things. First, my right hand tended to drift all over the keyboard (away from the home position), which would result in me sometimes getting lost and making a mistake. A mistake at 110 wpm means typing around 10 characters in the second it takes you to realise it. Backing out those characters, correcting the mistake, correcting your positioning, and continuing gives your typing speed a pretty big hit and is quite frustrating.

Second, my hands had to do all sorts of gymnastics while typing. Common English character combinations like 'st', 'ion', 'ce', 'ed' require some big shifts that take you away from the home position (slow and error prone), or require using the same finger for consecutive characters (slow and encourages shortcuts which shift your position).

Third, QWERTY is left-hand biased (at least for English). This can become quite fatiguing. There are also words like 'minimum' and the frequently-typed 'sweaterdresses' (pointed out on the Colemak site) that rely on a single hand; again, quite fatiguing.

I started to wonder if improving my QWERTY was worthwhile. Maybe I should try an alternate keyboard layout like Dvorak, which was meant to address many of these comfort and accuracy issues (my main goal), and also improve overall speed (a distant second goal).

Why Colemak?

It was around this time I saw @TheColonial mention something about the Colemak layout. I'd never heard of it, but in trying to research what it was I found some promising signs. Colemak was designed to be efficient and ergonomic by keeping commonly used keys on the home row and on the strongest fingers (check out the heatmaps for Colemak et al.), as well as being easy to learn by not deviating too far from QWERTY (the entire bottom left hand is unchanged from QWERTY, which means common shortcuts like Ctrl+Z/X/C/V are unchanged).

If you've ever experienced programmer hands, you'll appreciate why it is important to look after yourself and avoid RSI. I happened across PAT or JK's keyboard layout analyser, plugged in a few pieces of code and blog posts, and found that, had I typed these using Colemak instead of QWERTY, my fingers would have moved in the order of 50% less to type the same text. Figures like 192m for QWERTY, 97m for Colemak were not uncommon. It also tended to beat out Dvorak. How could anyone interested in efficiency not try that out?

Three weeks with Colemak

I started out doing Colemak exercises with aTypeTrainerForMac. Largely due to its commonalities with QWERTY, the layout itself was quite easy to learn. I had the key positions memorised by the end of the first day, but using it effectively was another matter entirely.

No matter how well I knew the individual keys, I found my brain wanted to think about typing a whole word or pattern at a time. I could start typing the first few characters in Colemak, get to a familiar pattern like 'ion', and my brain would leap ahead and get me to complete the word (incorrectly) in QWERTY. Getting my reason to fight these reflexes was very uncomfortable, but at the same time a fascinating insight into how the brain works.

After 1 week of fairly intense practice, I could type steadily at 20 wpm with Colemak, but my QWERTY had dropped a bit to 80.

Shortly after that (with the help of autocomplete) I was able to use Colemak pretty effectively. After three weeks of almost exclusive use I could type between 40-50 wpm and it felt very comfortable. My technique and accuracy were improving.

It was then I found my QWERTY was down to 15 wpm. That is not a typo; a measly 15 wpm. I was completely unable to use a standard keyboard.

Return of the QWERTY

I was a few days away from attending a Code Retreat in Brisbane, and was really concerned about being able to pair with people when I couldn't type properly. I decided to jump back into QWERTY full time. It took a solid day, but after being completely useless for an hour or so the reflexes started to come back. By the end of the day I was back to 90 wpm (provided I didn't think too hard about it :)). Surprisingly I could still type ~30 wpm on Colemak.

One thing that really surprised me is how clumsy QWERTY felt to me. I could feel the added strain on my fingers, wrists and shoulders due to the increased movement. Accuracy seemed to come much easier with Colemak, although that could have been a product of how I had trained myself up on all the characters, rather than on real text. I also really started to notice the bad touch typing habits I had formed with QWERTY, like leaving the home row and using wrong fingers to hit keys due to the key positions of many words.

Current state

I'm really undecided on how to proceed from here. One one hand Colemak really seems a lot more comfortable, and I'm pretty confident I could get up to comparable levels of speed with it.

On the other hand, almost every other keyboard in the English-speaking world uses QWERTY. It's obviously going to take me a bit of work to maintain proficiency with both QWERTY and Colemak. The alternative, being unable to use most other computers effectively, is very unappealing. If I was only ever going to use my own computer I'd almost definitely stick with Colemak, but as I do a lot of pair programming it really comes down to exactly how much effort it's going to take to be bi-typal.

If we end up getting QIDO for Colemak it may help for pair programming, but I'm still concerned about being stuck at some random computer and being unable to use it.

My other concern is that my perceived increase in comfort with Colemak is at half the speed of my QWERTY typing. I'm not sure whether Colemak would still be comfortable once I hit 100 wpm, or whether I would get the same discomfort as I do with QWERTY.

So there you have it; a bit of a non-conclusion. Colemak seems to have a lot of things going for it, but it's hard to recommend it in the face of the ubiquity of QWERTY and with only anecdotal evidence of it's ergonomic benefits (and there's some counter-anecdotes too). It's definitely an interesting experiment to try, just for some insight in to how your brain works with typing, but unless you only use your own hardware I'd suggest keeping up speed on QWERTY as well.

As for me, I think I'll try and develop both QWERTY and Colemak in parallel for a while and see what happens.

Saturday, 26 November 2011

Documentation via automation

There are lots of benefits to automating common, recurring tasks like builds and deployments. We gain reliable, repeatable results and remove the cost of duplicated effort. However I've just come across a related benefit that makes it worthwhile considering automation for less common tasks, and that's as a form of reliable documentation.

I'm currently battling some driver signing issues. We have some incomplete documentation from a previous occasion we've been through a related problem, but no one remembers the exact steps used. Because this previous occasion was reasonably considered a one-off, it was never automated. If it had been, it would now provide reliable, repeatable, unambiguously documented steps for getting me out of my current jam. Even if not an exact match for my current requirements, having a working example I could use as a basis to work from would be invaluable.

This has made me think that it is probably worth automating a whole bunch of tasks I hadn't previously considered due to their infrequent nature. What better documentation of a procedure than working, executable steps? Sure, there may be a need for accompanying rationale, but given a blind choice between a Word/wiki document and something that actually runs and works, I'm picking the latter.

Tuesday, 22 November 2011

Some mocking opinions

I've been thinking through how I use test doubles (mocks, stubs, test spies, etc) recently, and thought I'd write down a snapshot of my current opinions on the subject.

Don't mock types you don't own

I've written about this before, and I still think it is good advice. Test down to your lowest level of abstraction, then integration / contract / acceptance test over the boundary. By mocking a type you don't own that dependency starts bleeding in to your code and pushing your own abstractions in potentially unhelpful ways. You're also not really testing much; unless you also have good contract tests then checking you've called a specific external method is not going to tell you much about whether your code does what it needs to.

Contrived example: say I was faking out Math.Round() (let's pretend it's an instance method or you are using a framework that can mock statics via the profiler API):

[Test]
public void Calculate_with_rounding() {
    math.Round(2.5).Returns(3); //Fake math.Round
    var subject = new Calculation(math);

    Assert.AreEqual(subject.AddAndRound(1, 1.5), 3);
}

Perfectly reasonable? Well, except for the fact .NET uses banker's rounding and rounds 2.5 to 2 (and 1.5 to 2 for that matter). If you were using Python (which rounds away from 0), you would have been spot on. If you care about the rounding method, you've now got a bug.

If something as simple as rounding can trip us up, imagine what we could do if we start mocking ORM or ADO.NET calls.

Try to avoid mocks in acceptance tests

In my experience this tends to result in too much behaviour being pushed into the mocks. My first preference is to use real pieces, second is to hand-code fakes that have enough logic to work as required, and convenience methods to help tests configure them appropriately. As per "don't mock types you don't own", it is also a good idea to test your fakes match the real behaviour.

Learn mocking before a mocking framework

I've often heard developers new to automated testing say things like "I really need to learn (Rhino Mocks | Moq | Mocking Framework X)". I think this is the wrong emphasis; before learning a framework for creating test doubles it's important to understand how test doubles work and how to use them.

For me, a great way to learn was to hand-roll all my fake objects for my tests to act as required. Manually stubbing out values and/or recording calls gave me a good understanding of the different types of test doubles (mocks, spies, stubs etc.) and how they work. Once this got old (very quickly) it was fairly simple to take the behaviour I knew how to hand-code and translate that into the syntax required by a mocking framework. It just became a matter of automating what I was already doing. (It also helped me understand common difficulties like trying to mock non-virtual members.)

Don't explictly test intermediate steps or inconsequential details

If we assert on details of an implementation we tend to get tight coupling and brittle tests. An example I have seen fairly frequently is:

[Test]
public void Should_get_the_widget_from_the_factory() {
    var factory = MockRepository.GenerateMock<IWidgetFactory>();
    var subject = new Foo(factory);
    subject.DoStuff();
    factory.AssertWasCalled(x => x.GetWidget());
}

[Test]
public void Should_turn_the_widget() {
    var widget = MockRepository.GenerateMock<IWidget>();
    var factory = MockRepository.GenerateStub<IWidgetFactory>();
    factory.Stub(x => x.GetWidget()).Return(widget);
    var subject = new Foo(factory);
    subject.DoStuff();
    widget.AssertWasCalled(x => x.Turn());
}

Here the first test is a completely redundant. The second test covers that entire code path (how else could the widget from the factory get turned, if the subject did not call the factory?). Now you could argue that you prefer the extra, explicit specification the first test provides, to which I'd respond that I don't think it's worth the pain from the additional friction it causes when you want to change this implementation detail.

Besides, what do we really care about for our subject? That it uses a factory? Or that it turns the widget? Focus on how you want the object to behave, not how it implements that behaviour.

This approach can help lead us to better abstractions, as we start identifying roles and responsibilities separately from implementation details. And it will definitely make your code easier to change without the friction of over-specified tests.

Mock interaction with the contract, not the specific implementation

On a highly related point, the aim of abstraction is decoupling from the implementation. If we are configuring our test doubles with lots of behaviour that our unit tests are relying on then our object is coupled to that particular implementation, not to a contract of behaviour or a role. For an abstraction to be effective we should be able to drop in a completely new implementation that fulfils the required role. This is not the case if we need to set up a test double's method to call another dependency and return some rearrangement of the result. If we're relying on that in our test then our abstraction has failed.

Sometimes you just get stuck with having to perform a callback from a stub, but in general if you are pushing behaviour into your mock, re-think the design or consider using a hand-coded fake before you go contorting your mocking framework.

Beware over-abstraction

It is quite easy to churn out layers of useless abstractions when using mock frameworks. Abstractions have a cost. Feedback from tests is great, but pay close attention to SOLID and the rules of simple design and call out to meaningful abstractions, rather than putting in a dependency just for testing. I wrote some guidelines on abstractions a while back that I don't entirely disagree with yet.

Some unanswered questions

Mocks vs. stubs, tell vs. ask.

I've tended to prefer stubs over mocks (stubbing out the results of calls rather than checking they were received), as per the widget factory example above. This flies in the face of the "Tell, don't ask" principle, which recommends we don't ask a collaborator for some state and act on the result, but instead give the collaborator the state it needs from us and tell it what to do with it. This seems to suggest I should be using mocks (checking received calls) a whole lot more than I stub them out.

Avoiding "Yet Another Factory"

If an object news up something, our unit test will typically have almost no ability to affect that object. If we want to check our subject news up a view model and calls Activate() on it, we have no way of asserting this was done without exposing IsActivated and relying on that implementation detail. Leaky abstraction. Bad.

One solution is injecting a factory into our object. We can stub out what this returns, make it return another test double, and then check it received the Activate() call. Just introduce a factory. Yet another factory. Searching through files matching *Factory becomes an exercise akin to reading War and Peace. And they're generally not even real factories! They don't choose a particular implementation, they are just a glorified wrapper over a single constructor.

Sure, I sometimes try to ease my conscience by injecting a factory method as a Func<T> which my IoC container helps me with. But deep down I know its still YAF, and a small part of me dies.

I'm hoping choosing "better" abstractions will help me with this, but I've had limited success to date.

Interface explosion

C# seems to make it difficult to do testing without using interfaces. And so I end up pulling out yet another interface that will never see another implementation. I've heard Shannon refer to them as "the new header files". Every class has its interface documented in its header/interface file. It's easy to say just choose better abstractions where the interface can be reused, but this is still something I struggle with.

Mocking in dynamic languages

This post has been written primarily from the perspective of static languages; I not sure how much (if any) applies to dynamic languages. From my limited experience testing and mocking seem to be done quite differently in languages like Ruby and Python. I'm keen to learn more about how mocking is done in these languages and see how much can help me improve how I test.

End of transmission

This has been a brain dump of my current opinions about mocking. If you agree, disagree, and/or can help me with some of my unanswered questions, please leave a comment.

Now if you'll excuse me, I'm off to code up yet another factory...

Saturday, 22 October 2011

Odd problem with Git, Windows and virus-checkers

Had a really odd git problem this week, with an even odder solution, so am posting in the hope of helping the next poor dev who has to try and track this down via Google.

We had a branch checked out with 2 new commits on it, and we wanted to squash it into a single commit using git rebase -i (basecommit). This would start the normal interactive rebase, then get into a loop of printing the following error to console:

mv: cannot move '.git/rebase-merge/git-rebase-todo.new' to '.git/rebase-merge/git-rebase.todo'

Looking at the .git/rebase-merge folder, I could see the git-rebase-todo.new file getting repeatedly created, then deleted. This was happening on two different machines.

Some googling lead me to this post which suggested a virus checker may be locking the file.

Sure enough, turning off Microsoft Security Essentials' Real-time protection, doing the rebase, then turning it back on again, resolved the problem. I've never had this problem before so must have just gotten "lucky" with this particular file matching some property the virus checker was looking for.

I guess this is probably worth trying whenever getting strange file IO errors from any software ported to Windows.

Tuesday, 20 September 2011

Unhelpful flaming is unhelpful

I recently read an excellent post on patience and respect from Andrew Sharp, suggesting the radical idea of trying to empathise with the much-maligned whoever-wrote-this-steaming-pile-of-code-that-I-now-have-the-misfortune-of-having-to-work-with instead of the usual cursing of the previous author's name, professionalism, family, and very existence in a bout of barely contained nerd-rage.

This is often quite embarrassing when you realise the previous author was you, but this isn't the only reason to stop the practice of flaming the authors of rotten code.

Yes, this post is basically a rehash of Andrew's post, but as it resonated with me I felt compelled to write down my perspective on the topic. His is concise and beautifully written, but mine briefly teases SharePoint, so take your pick. :D

The situation

You open the code. It's worse than you thought. Your face falls as you flick through the files you need to change, and see one or more of the following:

  • Classes with thousands of lines.
  • Methods with thousands of lines.
  • Nested ifs so deep you need to scroll horizontally to find the end of the arrow.
  • Constructors with 17 arguments.
  • Regions.
  • No trace of automated testing. Or of even having been manually tested.
  • Code that only works due to some obscure side-effects.
  • Code that only works on Tuesdays.
  • Liberal sprinklings of Thread.Sleep.
  • Object Oriented code where all object members are static.
  • No.respect_for.the_law().of.demeter()
  • SharePoint involved somewhere.
  • Incredibly clever code featuring an abstract factory pattern to produce strategies which are injected into commands applied with double dispatch using the visitor pattern, proxying through to WCF calls batched into units of work and persisted using an EF entity data model. You can't help but think this is overkill to print FizzBuzz.

The initial reaction

  • WTF!?!?
  • No seriously; WTF?!!?!?
  • What were they thinking?
  • What an unprofessional jerk!
  • How do they sleep at night?!?!
  • I've got to post this to twitter. Everyone will get a good laugh at this.
  • Oh, and I'll blog it! There's another couple of hundred people to pour scorn and ridicule on this deserving schmuck!
  • Even better, I'm posting this to thedailywtf.com!

Stop! Hammertime!

What exactly are we doing here? We're all puffed up with self-righteous indignation, but what is this vehement response going to accomplish? Shame an incompetent person out of programming, thereby ridding the glorious profession of software development of one of those rare dullards that somehow slip into the ranks of the brilliant, intellectual master-craftsmen and women that dominate the industry? (In case it isn't clear, that sentence carries a <sarcasm/> tag :P).

Probably not. All we're doing is stroking our own egos, delighting in someone else screwing up for a change, and further fostering the culture of competitiveness, elitism and macho-attitudes that seem to keep popping up in our industry.

Sure, short-term you may feel better for venting, but adding to the bile on the internet is a bit like peeing in the dam that supplies your own drinking water.

So what could we do instead?

Consider the context

Let's take a deep breath, then start thinking about why this code stinks. It is important to realise that people are generally trying to do good work. There are probably valid reasons the code you are currently staring at seems wrong, and most of them probably have nothing to do with trying to spite you.

  • The author is inexperienced.
  • The author does not have experience with that particular problem.
  • The author is trying a new approach in an attempt to learn.
  • The author doesn't know about a language feature, pattern, or project convention.
  • The author was working to an insane deadline that would cost the company millions if the feature wasn't out the door in 33 minutes.
  • The author was actually confronted with worse code, and did substantial tidy up of a whole bunch of classes and extracted the most ugly bit to a place where it could be refactored later (i.e. where you currently need to modify ;)). The reason you are not currently fighting another 3 fires is because of the work they did.
  • The author knew the code wasn't perfect, but it wasn't obvious at the time how else to factor it.
  • The author just seperated from their long term partner, they have a parent in hospital, their car was smashed by an uninsured driver, their bank just increase its mortgage rate and their pet just died in a tragic canoeing accident. They had trouble concentrating on the code they wrote that week.
  • The author implemented a bunch of awesome classes, and this one.
  • The author wrote the code in a way that was perfect at the time, it's just that times have changed.
  • The code is actually really good. We just don't see the beauty of it.
  • The author made a mistake. You may remember making one or two of these yourself in your younger days.
  • The author was forced at gunpoint to write the code by a rogue PM who was also holding the author's family hostage, dangling them precariously over a vat of potent hydrochloric acid while screaming "add another responsibility to the class or they all get it! Add more regions and superfluous comments! And CLOSE THAT CLASS FOR EXTENSION!!! DO IT!!!1!"

None of this means the author is a bad person, and yet us programmers seem to take perverse delight in opening some code and sniggering "WTF?!!? What were they thinking?" while firing up our favourite blame tool. Maybe it just makes us feel better about all the cruddy code we've written?

Opportunities

Instead of dwelling on the negatives, let's look at the opportunities the code has given us to do some more constructive things than suggested by our initial reactions.

  • If this is public code, we could contact the author, thank them for being brave enough to share the code in the first place, and constructively suggest alternate approaches, explaining the rationale for doing so and being careful to keep the person and the code separate (e.g. favour "this class could use a factory to create this instance" or "we could use a factory..." over "you shouldn't create this here, use a factory". This helps get past people's natural tendency to become defensive, and prevents you from slipping into flame mode.)
  • If you work with the person, maybe consider how collective ownership applies, and think about how the team can improve things in future.
  • If you can see how the code ended up in its present state and what to do about it, take the general concepts (not the personal attack) and blog that, post it to a programming mailing list, conduct a brownbag session at work, present the general principles at your local usergroup, talk at a conference, get on a podcast, start a podcast. Teach it. Educate. Play a small but valuable part in lifting the community and advancing the practice of programming.
  • Think about how APIs or patterns could change or be used to prevent the problem. Release an open source project that makes getting it right easy and obvious (as well as an explanation of why this approach works).
  • Practice giving criticism in a constructive, respectful way. This is a skill worth developing.
  • Recognise that there is no perfect, tech debt-free solution, and think up of lots of ways to fix the problems you see in the code. This will help you to improve both your coding and your perspective. Combine this with aforementioned blogging, presenting, teaching etc.
  • Encourage the author. Programming is hard; praising anything good about the code and encouraging them to try improvements can give them more confidence to tackle future problems more creatively. It's amazing what being freed from the fear of failure can do.
  • We could, hmm, I dunno, maybe FIX THE DARN CODE AND MOVE ON WITH OUR LIVES!!! ;)
Tip: While explaining a techinical concept, avoid condescending words or words that diminish the task like "simply", "basic", "trivial", and "obvious". It may be to you, or it may be in hindsight. Recognise that if it were any of these things to the author, then they would have done it that way in the first place.

Why bother?

Because programming culture seems filled with large egos, worthless competitiveness, negativity, and cults around celebrities which discourage both creative thinking and challenges to the status quo. And yet we also have many examples of selflessness and generosity: people pouring hundreds of hours into OSS projects, donating their time to usergroups and mailing lists, and blogging stuff they learn so that others may benefit. Let's spend more time on these positives within the dev community. By focusing on the opportunities finding "bad" code gives us, I believe we can get a community where:

  • People can ask honest questions without being told to "RTFM n00b".
  • People spend more time discussing different approaches than defending previous ones.
  • People can post sample code or OSS projects without being ripped apart and mocked, but instead given constructive advice and congratulations for having the guts to risk failing in public.
  • All people can contribute; not just the loudest, most extroverted, most confident personalities. A more diverse group of people will provide more diverse ideas and innovations.
  • Failure starts to be considered a wonderful thing, as even a single failure is an opportunity for many people to learn and improve as developers.

Please leave your flames and ad hominem attacks in the comments. ;)

Tuesday, 16 August 2011

Functional programming newbie and something something monad something

Let me get one thing straight: I know absolutely nothing about monads. I have never intentionally used something I've recognised as a monad. I am dangerously unqualified to enhance your understanding of monads in any way. In fact reading this may damage you and prevent you from ever learning what a monad actually is!!!

The first reason I'm posting anything about monads at all is that I watched one of Robert "Uncle Bob" Martin's entertaining NDC 2011 talks titled "WTF is a monad" (video available from the NDC site). I'm unsure how approximated or mathematically correct he was intending the presentation to be, but I found it really interesting and was able to implement something I can only hope was vaguely monadic based on my interpretation of the information he presented. So I thought I'd share it with you in case you could correct me (it should go without saying, but any mistakes here are mine and have nothing to do with Bob or his presentation). Worst case is it gets you interested enough to look into the topic and find out all the stuff I got wrong. (Was that alright OJ? ;))

The second reason is that I like writing words like monad, monadic, and monoid because for a brief, shining moment it makes me feel like a real computer scientist. This moment generally comes crashing down as soon as I realise I have no idea what any of these terms mean, but it is a good couple of milliseconds. :)

Did I mention I don't know what I'm talking about? For this post especially I mean. Yes? Good, you should be safe to read on then...

Something something monad something

As far as I can gather, a monad is a structure that will let you use functions that take arguments of a certain type, and apply it to values from an another type (I'll call this the monadic type, but I could be misusing the term). We need to be able to map back and forth between these types. Bob roughly approximates a monad to an adapter; a monad is a way of adapting one type to another.

It is the form of this adapter that makes it a monad. A monad can be expressed as two functions: the unit function, normally called return or result, that takes an argument of the original type and returns the monadic type; and a bind function that takes the monadic type and a function that works on original types. (Technically monads should also obey the monad laws. I'm sure I've missed other important points about them too, but let's run with this for now.)

This structure has a few useful properties, mainly to do with being able to chain a sequence of functions that take arguments of the original type, then apply arguments of the monadic type to that chain. I think.

A dot monad?

Uncle Bob's first example was using a monad to manipulate a dots type using functions that normally work with integers. The dots type is simply a representation of an integer using '.' characters, so 5 maps to '.....' and back again. We'd like to be able to be able to use dots with standard integer operations like add, so that '..' + '...' gives '.....'.

Let's look at an example in Python:

class DotMonad:
    def result(self, i):
        return '.' * i
    def bind(self, dots, f):
        return f(len(dots))
Aside: If you haven't used Python before, the self arguments to the functions is required due to how instance methods work in Python. You can safely ignore them for this post, but if you know C# or Java self basically becomes like this in the context of an instance method.

Here our result function just translates integers (i) into dots. The bind function takes some dots and a function f that takes an integer. First it converts dots to integers (using the length of the string of dots) then calls f using the result.

This means that if we have an add function which takes integers, we can use our monad to adapt that function to take dots.

# Integer add function
def add(a, b):
    return a+b

# Monadic add function for dots
def addM(dotsA, dotsB):
    m = DotMonad()
    return m.bind(dotsA, 
        lambda a: m.bind(dotsB, 
        lambda b: m.result(a+b)
        )
    )

I've used a and b as the plain integer types, and dotsA and dotsB to represent our monadic dots type. We can now call addM('..', '...') and get '.....'.

So how's this work? Well remember that bind takes a dot for a first argument, and calls the function provided as a second argument after converting the dot to an int. The function we provide will be called with dotsA converted to integer a, then recursively call bind to convert dotsB in the same way. The last function in the chain is to the monad's result method which will convert the result of a+b back to dots.

Let's expand out and trace through the addM('..', '...') example to make sure we've got a handle on this:

return m.bind(dotsA,    # dotsA is '..', which is converted to int and passed to fn in 2nd arg
    lambda a:           # bind calls function with a = len('..'), which is 2 
        m.bind(dotsB,   # dotsB is '...', which is converted to int and passed to fn in 2nd arg
    lambda b:           # 2nd bind calls function with b = len('...'), which is 3 
        m.result(a+b)   # a+b is 2+3=5. m.result converts this back to '.....'
)

I think this is called lifting the add (+) function to work with our monad.

Lifting functions using monads

So we've now got a version of the basic integer add function that can work with our monadic dots type. But we'd like to be able to apply all integer functions to work with dots. In fact, we can generalise our addM function from before to lift any function which takes two arguments using a monad that can bind to that function's argument type .

Aside: We could also generalise to support functions with any number of arguments, but I'm struggling to keep up as it is. :\ :)

def liftm(m, op):
    return lambda a,b: m.bind(a,
            lambda ax: m.bind(b,
            lambda bx: m.result(op(ax, bx))
            )
    )

This is pretty much identical to our addM function, but we can now do some neat stuff. Let's import some standard Python operators and dot-erise them:

import operator

addM = liftm(DotMonad(), operator.add)
subM = liftm(DotMonad(), operator.sub)
divM = liftm(DotMonad(), operator.div)
mulM = liftm(DotMonad(), operator.mul)

#Interactive python session
>>> addM('..', '.')
'...'
>>> subM('....', '...')
'.'
>>> divM(mulM('..', '...'), subM('...', '.'))
'...'

Should we try again? Maybe...

Let's try another monad (again, from one Bob showed in his talk). This time we're going to try and represent a type that can either have or be missing a value as a monadic type. So something very similar to .NET's nullable types, Nullable<T>. The difference with the monadic form is that, because of the way we chain sequences of bind operations, we can actually perform operations involving missing values without throwing null reference exceptions everywhere.

class MaybeMonad:
    def result(self, x):
        return x
    def bind(self, maybe, f):
        if (maybe is None):
            return None
        else:
            return f(maybe)

Here our result function just returns whatever value it is given. If it has a value it will return that value; otherwise it will return None (Python's null or nil value).

Now we can lift our standard operators to work with our Maybe type:

>>> addm = liftm(MaybeMonad(), operator.add)
>>> mulm = liftm(MaybeMonad(), operator.mul)
>>> addm(2, 3)
5
>>> addm(4, None)
>>> mulm(6, 7)
42
>>> mulm(None, None)

Or we can lift null-safe versions of other functions:

def string_lens(a, b):
    return len(a) + len(b)

#Interactive python session
>>> string_lens("Hello", "World")
10
>>> string_lens("Hello", None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in string_lens
TypeError: object of type 'NoneType' has no len()

>>> safe = liftm(MaybeMonad(), string_lens)
>>> safe("Hello", "World")
10
>>> safe("Hello", None)

Here string_lens throws when we pass in None, but our safe lifted version takes them in its stride.

Real-world monads

Monads can actually be spotted out in the wild. They particularly enjoy frolicking with pure functional languages, where they can be used for (among other things) getting around the pesky limitation of not allowing side-effects in functions. Mutable state can be simulated by passing a State monad between functions. The I/O monad is used to encapsulate the side-effects of reading and writing from input and output.

Reading through the examples in the Wikipedia entry shows some collections can even be regarded as monads (for example, result can return a list from a single item, bind can map a function to each element in a list). In some instances LINQ statements can also be used as monads. I've even seen JQuery accused on monadishness (yes, I just made up a word).

So where's this leave us? If you're like me: dazed, confused, craving a cup of tea, and also quite eager to resume working through the excellent Learn you a Haskell tutorial. :)