Testing: disciplin, attitude and intuition (3/3)

Intuition

I had one other bug that I found really inspiring 2 days agos ("learn from your errors", they say,...). I had to create a project, open it, generate test cases, reopen the project and regenerate test cases to get the bug .

The question is: "why would you test that sequence"? Because you have testing experience? Because, it is written in the requirements? Hum, not really. I have never seen "REQ_451 : after 2 successive uses , the software should not crash". This requirement is really an implicit one.

In fact, this is actually an invariant that is worth writing down as such in a requirements document: "REQ_451 : whenever a project is opened, a test generation should not make the software crash ".

Then, you have several ways to test that requirement:

1. The "classical" way. Trust your intuition/experience and find a naughty test case to drive the application mad
2. The "LTG" way. Use LTG and define an operation that models your invariant: "if (one of my entities in the system has no parent entity) then the application is broken". Then, model valid operations that modify the parent of an entity and generate test cases, that will try to break the invariant.
   

At that point there are 2 possibilities:

1. your requirements allow a valid sequence of events that may break the invariant. LTG may find tests that show this behaviour, but then you may just conclude that your specification is not tight enough! There's a breach in the specification and it's nice to know it.
   
2. the broken behavior can only appear because of a broken implementation. Then use LTG to concentrate the fire on that specific spot. Generate more test cases that will vary the conditions where the entities parents are modified.

The second case is an automation of the tester's intuition. This makes the case for the trio "disciplin, attitude and intuition":

1. Intuition because you focus on what could go wrong
2. Attitude because you don't try to guess what could go wrong exactly but more likely you test the range of combinations that should be allowed
   
3. Disciplin because you take time to produce many different combinations (of course LTG helps with that matter,...)
   

Testing: disciplin, attitude and intuition (2/3)

Attitude

I ran across a test today that I found interesting. The tested class was one of our association containers (see previous blog entries), a List to be more precise. A list can contain several times the same element.

Believe it or not, I am not sure that writing the tests for that class I would have thought to include that kind of specificity. I am glad I didn't develop the test case for that class,... because the developer that did, did it right. He had tests that included that specificity.

I am not sure that the first implementations of the class needed that kind of test, but surely the modifications I introduced afterward failed because of that blessed test!

I think the main difference between our 2 state of minds is that I tended to do white-box testing whereas the other developer did black-box testing. That's a very different state of mind:

• when you do white-box testing, you know what could go wrong and you test it
  

• when you do black-box testing, you define what you expect but since you have no idea of the implementation, you try to expect most of it. So you add the same element twice in the List. You dont want to guess what could go wrong, you just want to test what is ok for you

To that extent, testing is also a matter of attitude. Having the "tester" hat is quite different to having the "developer" hat. Switching between the 2 may not be so easy, hence the need for disciplin mentioned in the previous entry.

Testing: disciplin, attitude and intuition (1/3)

This is the first of 3 entries that make testing look like a martial art,...

Disciplin

Test-driven development is a really wonderful idea. Start doing it and you'll catch the fever. There are many sources that establish all the benefits of this approach. However, we're still human people,...

I personaly find it difficult to do pure test-driven development when:

• In the process of implementing necessary code to make one test pass, I have to write new code in other classes that should also be tested. Sometimes, I am so much into getting that green bar, that it's not so easy to digress and start fully testing/developing something else
  

• When I develop a new idea, a new implementation across a whole set of classes, I may want to write code that could end up not being the right approach for my current objective. Should I then write the tests for something that may really not function as planned and that should be thrown away?

The "sensible" approach to those 2 questions may be: "no Eric, don't sell your soul to the Devil, you have to test each and every line of code that you produce". So I should:

• Set aside my first testing goal, clarify what I expect of each new operation I am developing and test it, even if it may cause a delay for the first green bar, or for the exploratory design

But I live in reality and I actually find it more rewarding and productive to do what I want to do to a certain extent without testing it first. The trick is,...

...to always have a tasks list for the things that should be tested:

• Once you have the first green bar, go back to the list and test everything that hasn't been tested
• Ditto for the exploratory design

I have exceptions for that process. If I need to develop something that's really not easy along the way, I switch back to the TDD mode, because I simply know that I will produce missing code faster.

"Ok, ok, Eric, it's nice to learn about your fascinating worklife, but what's the point?".

Well, the point is that one: in order to succeed, you need disciplin.

You absolutely need to get back to your list. I found that 2 nice ways to do that:

• Use your buddy! When you practise pair-programming, your partner should always have the responsability for the tasks list. He is your Master that should help you with the disciplin stuff (this also goes for coding standards respect and so on)

• If you program with Eclipse, you could use Mylar. This plugin keeps track of all the classes that you modified in the context of programming a given feature. That way, when you get your green bar, you can be confident that you got all the modified resources at hand.

It's cool to have a tool

It's the positive "hammer" effect ("when you have a hammer, everything looks like a nail"). We were facing the following problem:

We want to enforce that, when an object is composed of other objects, those objects can not have the same name. This is a strong constraint on our domain model, however it allows us to define "paths" for our entities that are truly unique (a path is composed of entities names from the object to its root parent).

However, what if the user wants to rename an object that is part of a composition? This could break the rule. There are 3 design choices :

1. have a "rename" method on the composite object doing the check and be careful never to give access to the component objetct (by cloning it when accessed through a getter method for instance)
2. let the user rename the object directly and then let him validate the composite object (what if he forgets?)
3. our design choice

I'll explain our choice, of course. Since we have Composition objects that observe their content, it is quite easy to notify them (through an Observer pattern) when the setName() method is called on any of our entity objects. Then, they can do the check (that they usually do when an object is added to them by the way).

This is one of the neat benefits of making the distinction between Compositions and Associations in a domain model (see Eric Evan's book Domain-driven design (DDD)). Many times, when associated objects evolve, the related objects need to be informed to change their state:

1. when a component changes, the composite must be able to validate that change. A possibility would be to never give access to components in an aggregate (the advice in DDD). This can be pretty restrictive for some domain models.
2. when an associated object changes (for instance is removed from the database), the related object needs to know about it (otherwise, he may have a dandling pointer). Usually this kind of rules is managed outside of the objects (through a data access layer). But why should this behaviour be separated from the domain behavior?

This is where the hammer hits: by having Composition and Association objects (and variants: Lists, Maps,...) it is really easy to control this behaviour in a very generic way. Each "association container" observes its associated entities and when they face "important" events (renaming, deletion), they notify their container. The container then knows what to do:

• a OneAssociation has no more associated object: "oops, this must be a critical mistake"
• a ZeroManyCompositionList is deleted. So should all the components inside the list
• ...

If I had a hammer,....