How test driven are you?

Standard

Database design

For the past few days I’ve been struggling with a moral dilemma: I’m attracted to the idea of test driven development (TDD) and writing unit tests before actual code, and it’s great whenever I do excercises to try it out. My problem is that although I’d like to work in a completely test driven way, I have problems actually practicing the methodology to the letter. So, “good that I have a blog”, I thought and decided to ask my readers (that’s you!) for help.

The ideal procedure presented in TDD literature and for example eXtreme Programming is that you start with a minimal piece of functionality that you want to add to your code, then you write a test for that. First the test of course fails because there is no code available. But once you’ve written enough code to make the test pass, you move on to the next test. And so on.

I get stuck at the first step. When I start a new application and need to write that first test for it, I just don’t know what to write. I don’t want to do big up-front design because it’s against the Agile principles, so how do I know what my first test should be?

My solution so far has been to start with some design and a prototype of some kind. For example a mock up of the UI, some screens laid out in HTML or even some Rails controllers. Slowly I start to see the application and how it should be designed, and then I write my first test. That’s when my prototype starts to evolve into a real application. One test at a time, I create the model class and move functionality where it should be instead of the place where I dumped it in the first place when I didn’t know what I should be doing.

This violates a few principles: first, some people say that you should only use your prototype during design and then start from scratch again. Instead of doing that I’m evolving my prototype into this fully functioning piece of software. Second, as I said, it’s not really test first: I’m doing kind of a coding phase before I know what I should start testing. And sometimes it’s not only in the beginning. And I’m not prototyping only at the beginning of the project: When I’m done with the clear part and again start to work on something I’m not sure how it should be done I go back to prototyping mode until things get clear again.

So, I have a few questions that I hope you could help me solve:

  1. Is it OK to go to a prototyping mindset every now and then and still think that you’re being test driven?
  2. How strictly are you, other developers, following practices like TDD? Do you think they should be practiced like a religion, or should we be using our common sense and create our own mixes of practices? Why?
  3. And if you’re able to do completely test driven development, what’s your secret? Where do you find the first test? And so on…

And now, the floor is all yours.

18 thoughts on “How test driven are you?

  1. Don’t follow anything religiously, not even religion. Religion often disables thought. How do you know God exists? You have faith. How do you know that Agile is the best tool for your job?

    Some of the driving factors that make Agile useful might not be present in your environment, or part of your environment. Refactoring mercilessly is harder when the thing that needs refactoring is your database schema. Writing tests before code is counterproductive when you’re not sure until you write the code how you want it to behave.

    Test-first could be called Little Design Up Front – you’re designing the next unit by specifying how it should behave given certain inputs. In that sense, it has all the problems of Big Design Up Front, but scaled down, possibly enough that they disappear, depending on the context.

    Personally, I write tests when I am a little hazy about how the code works, or should work, and I write them when I encounter a bug that survives my first burst of debugging (say, 10 minutes or so). Sometimes I write a test after solving the bug, to make sure the bug doesn’t reappear. But then, as not all refactoring is automated, I can find myself spending longer applying a refactor to the tests than to the production code, so I try not to have more test code than actual code. I think when I have more test code than production code, I’m subconsciously trying to construct a proof through testing, which is usually impossible.

    Everything in software should be a tool, not forcibly immersive. Immersion can be good, but you need to be able to break out too.

    I mean *everything* there, from static typing, through Agile, through IDEs, OOP, FP, etc. Any time you stop thinking, because you’re following something religiously you’re likely to negatively impact the outcome. You don’t need a framework to think in, you need a way of externalising your thoughts. That’s not to say your tools/paradigms shouldn’t influence you, but they shouldn’t stop you from thinking.

  2. Hi,

    An interesting blog entry.

    “1. Is it OK to go to a prototyping mindset every now and then and still think that you’re being test driven?”

    I call it spiking. I only do it when I’m not sure how to solve some problem. I throw the result away once I’m sure what’s the best design for the requirement.

    “2. How strictly are you, other developers, following practices like TDD? Do you think they should be practiced like a religion, or should we be using our common sense and create our own mixes of practices? Why?”

    I follow TDD very strictly but that’s just my personal choice. You have to find your way. For me TDD is great for 3 reasons:
    - Tests, so that I don’t break anything later, during refactoring
    - Design, that’s probably the most important part
    - Documentation, so that I know which classes are responsible for what.

    I just invented a new definition for TDD = Tests, Design, Documentation :D

    “3. And if you’re able to do completely test driven development, what’s your secret? Where do you find the first test? And so on…”

    My secret is to always write a user story for the feature I’m adding. Then it’s easier, because you know what’s the scope. After writing a user story, I write a functional test (Selenium-like). Then I start writing unit tests and my code. That’s all. At the end, the whole functional test passes and I’m done.

    Cheers,
    Andrzej

  3. funny I saw this…
    Ricky makes excellent points. I don’t often get to write brand spanking new code. However when I do I try my best to follow TDD principles. When I do manage to follow it I find my code is just so much more loosely coupled and testable. I also just love seeing that red to green bar. Helps my motivation :)

  4. For me, TDD is a means to an end, not an end in and of itself. I find it to be a useful tool for producing tested and testable objects, but when the test is written in relation to the rest of the code is of secondary importance. Personally, I tend to use TDD after I’ve identified the need for an object that will do something that might have a bug in it. “Test the things that might break.”

    Like you, I usually have to do some prototyping to get a rough idea of what objects I need! I’ll sometimes use CRC cards as a tool for identifying objects and their responsibilities, and this gives me an idea of what tests I’ll need. I tend to use TDD for the “meaty” objects.. those objects that have may have non-trivial behavior and form the core of your program (some call them “domain objects” or “business objects”). I want to be sure that those objects are testable, and actually covered in unit tests. But I might not even use TDD for infrastructure code like factories and data transfer objects.

    I do strive to have most if not all of my code covered by at least some kind of test, because the tests give me confidence that my code does what I think it does. As I see it, as long as the test gets written, and the design of the program lends itself to being testable, it doesn’t really matter when the tests are written.

  5. Wow! Thanks for your comments, guys. It was great to read them and I got new ideas and insight from all of you.

    I like Ricky’s idea that nothing (not even religion) should be followed religiously just because someone tells us to. But I do believe in test driven development, at least to some extent.

    Some time ago I finished a big mobile game project without any unit testing. During the beta testing period it seemed that every time we fixed a bug, we accidentally caused a new one. The code wasn’t testable, and there was no test to show us if the code was still doing what we thought it should do (and most of the time it wasn’t). So, I believe in TDD, because I’ve seen the alternative and it wasn’t a beautiful one.

    As Jim mentioned, the results from doing TDD make TDD useful: loosely coupled, testable code, and the green bar to guarantee that the code still pretty much works the way I want it to make TDD something I want to be doing as much as I can.

    Andrzej has a great point about writing functional tests first. It might actually be true that if I started from functional tests, I might be able to skip the prototyping phase in most of the cases. Or do that phase with support from tests as well. Because that’s what I’m actually doing most of the time when prototyping: controllers, views and other things closer to the user interface than the actual model level. I think I’ll try that approach in my future work.

    And finally, it’s great to hear that Margtholomew is working in quite a similar way as I am. After all what matters is that in the end the code is covered with tests and the customer gets what he wants: a high quality product which can be maintained without creating more problems.

    Also maybe it doesn’t make sense to start writing tests without “any design”. Little design is needed, and we need to find out what we’re doing before we can start doing it. User stories and functional tests might be a good place to start, and I guess prototyping isn’t that bad either, every now and then.

  6. You seem to assume that chaos is the only alternative to writing tests before code. Firstly, you can write tests after code, at the same time as code or before code, and I think none of those necessarily cause chaos.

    Secondly, there are other approaches besides testing – many programmers use (usually static) type systems to construct proofs of correctness.

    TDD specifically talks about unit testing – you could have had integration testing in your game project and caught your new bugs just as well.

  7. Carl

    “My secret is to always write a user story for the feature I’m adding. Then it’s easier, because you know what’s the scope. After writing a user story, I write a functional test (Selenium-like). Then I start writing unit tests and my code. That’s all. At the end, the whole functional test passes and I’m done.”

    I agree with this wholeheartedly but this does seem to be leaning towards BDD and that is why I am trying to transition from TDD to BDD.

  8. Justin

    The reason I test is so I can be confident in my code. On my first Rails app, I developed the whole thing without one piece of test code. I could have launched it, but I had no confidence in the stability of the app. Then, I rewrote the entire app using TDD and when I launched, I had complete confidence in my code and myself.

  9. Daniel Watkins

    Hey,
    Just read a blog post about behaviour-driven development[0], which is a process derived from test-driven development. It basically started from the premise that writing your tests in the form ‘shouldDoX’ helps you outline your tests easily.

    That said, I always start by writing a bit of code that I know to be correct, writing tests to cover it and then move on with TDD…

    [Footnote 0: http://dannorth.net/introducing-bdd/

  10. Ricky:
    Actually I don’t think the order matters that much – it’s more about whether I write tests or not. In the project I’m refering to all testing was done by using the product in a way an end user would use it. So, there was no automated tests like unit tests present at all. I think that any kinds of unit tests would have made it easier for us to do the changes and thus get the project completed faster.

    This approach works pretty well in the sense that we actually did find the problems eventually. But the problem was that it was the QA department who had to look for the bugs and report them to us. So it was quite time consuming. With unit test (or any kind of automated tests) we could have found the problems even before giving the new code to our testers.

    To me it seems that TDD’s could be that by following that practice you actually do write the tests, and make your code more testable. And that’s how it can help you avoid chaos.

    And well… To be fair, the project I’m talking about wasn’t complete chaos. After all, we managed to get the bugs closed and the product ended up pretty good. But at some point it sure felt like a never ending story. :)

    Carl:
    I’ve been also looking at BDD and it looks interesting. I’ll have to try it sometimes in the near future as well.

  11. Hey Daniel, thanks for the link! I’ve been reading some blog posts about behavior driven development, but this one cleared many things to me about what’s the idea behind that approach – and how it could help me in the situations described in this blog post.

  12. I usually use follow TDD when extending existing programs. When I build completely new software I first create a prototype, then I either toss it, and start TDD from scratch, or build on it gradually adding tests and turning it into fully functional software. Maybe you will feel better about it knowing that this pattern has a name ;-). The basic difference between Spike and Big Design Upfront is that the former is “depth-first” and tries to get “something” done as soon as possible (the “something” can then be tossed or not, it’s there mostly to help you explore the problem), while the latter is breadth-first and will invariably end up in a disaster if the problem isn’t well known.

  13. Jake McArthur

    I get the urge to prototype a lot, so I do. To keep safe, here is my process:

    1. Fork a new branch of the project.
    2. Modify it until it has the new functionality I want, then refactor it a bit if necessary. No tests are strictly necessary here.
    3. Go back to the original branch and begin the normal TDD/BDD process. Make a test that fails, make it pass, etc. All the while, use your other branch as a reference.
    4. Once you are done integrating the functionality into the main branch, trash your prototype. Never look at it again.

    I have chosen to slightly modify the process to safely support my natural tendencies. This is the path I think most people should follow. Like Ricky Clarkson says, don’t follow anything religiously. Different methods are better for different situations. However, sometimes it is just better to slightly evolve a process to suit you.

  14. Carl

    Jake: I never thought about doing it that way, but that is indeed the best of both worlds. Thanks for the insight! i Have heard of developers using VM’s to do this sort of thing but that seems more complicated and less reusable than what you suggest.

  15. Jim Corbeil

    I’m totally a sucker for prototyping my way into test driven development. I take it one step further in that I use a prototyping annotation tool like Protonotes (http://www.protonotes.com) to add notes to my prototypes that eventual evolve into my test cases.

  16. I’m writing up labs for a seminar I’ll present to my staff soon on doing TDD in JSF. My approach since JSF is just another MVC framework I started at the service layer. I assumed I was going to have authentication so I needed a SecurityService. So I wrote tests like testLoginWithGoodUser() and testLoginWithNullUser(). That forced me to implement some sort of user class. When I felt like that was fleshed out enough I dropped into the persistence layer and tested things were being persisted using an in-memory HSQLDB and Hibernate. Then I went back to the controller and wrote tests to actually perform a login. Finally I created a Selenium tests for the front end to test the navigation between pages.

    This approach has allowed me to stay almost 100% TDD. Of course the reason I’m writing up the seminar is to convince my developers there is a way to do hardcore TDD in J2EE land even though they often struggle with it. Mock frameworks can help a lot to since you can take out dependencies and do TDD without implementing much of the full stack upfront.

  17. Jim: Thanks for the tip. Protonotes seems like a pretty useful tool for web based applications. I’ll give it a try at some point.

  18. I tend to follow TDD very closely but I will break it when needed. I find it gratifying when I can complete the feature without even firing up a web browser.

    1. “Is it OK to go to a prototyping mindset every now and then and still think that you’re being test driven?”

    Yes. I can’t remember where I read about it (Dr Dobbs?) but there was a study that found even the most Extreme XP shops spent the first iteration planning and doing some up front design.

    2. “How strictly are you, other developers, following practices like TDD? Do you think they should be practiced like a religion, or should we be using our common sense and create our own mixes of practices? Why?”

    Not like a religion but practiced heavily. If a developer says they completed a feature without adding/changing the tests I will have them prove it to me that it works. Sometimes I will write up test that fails and have them fix it (typically a boundary condition).

    3. “And if you’re able to do completely test driven development, what’s your secret? Where do you find the first test? And so on”

    I like to build up my trust in the system. My first tests are usually “Did the test data really get loaded?” (Model) or “Is the webpage really displaying or is it redirecting?” (Controller). After that I just start to build upwards. This still takes some design up-front but not much.

Comments are closed.