Coming full circle

In this post I want to show a few of the upcoming features in specs2-1.9 and also take a step back on how specs2 features and implementation unfolded since I started the project one year ago.

Why would you want to rewrite from scratch?

Good question, that seems to be very masochistic :-). Not only because of the sheer amount of features to re-implement but more importantly because of the difficulty to move users to a new version.

I explained in details the reasons why I thought the rewrite was necessary, how it would benefit users and how to migrate in a previous blog post. What I want to emphasize here is the compromise I decided to make at that time:

• more concurrency and reliability
• less practicality

The main reason for this compromise was immutability. Forcing myself to immutability had opened the gates of robust and concurrent software but at the same time I had to cut back on the syntax I had proposed for the original specs1 project. For example it wasn't possible anymore to write:

    // this example can only be "registered" if there are side-effects
    "hello must have 5 letters" in {
      "hello" must have size(5)
    }
    "world must have 5 letters" in {
      "world" must have size(5)
    }

Instead, I proposed to write:

    // examples are "linked" together with the ^ operator
    "hello must have 5 letters" ! e1 ^
    "world must have 5 letters" ! e2

    def e1 = "hello" must have size(5)
    def e2 = "world" must have size(5)

The feed-back was immediate. Some developers who were sent a preview liked it but others told me: "NEVER I would use this horrible syntax". Ok, fair enough, I can't blame you, it's a bit ugly on the right.

I decided then that I would relax my constraints a little bit and add one, just one, var. It would only be used to accumulate examples when building the specification, to enable the good old specs1 style. This was also, at least partially, solving the migration problem since a full rewrite of the specifications was not necessary.

Not everything was as cool as in the original specs1 tough. In particular the super-easy "isolated" execution model was missing. Until...

Lightbulb

... Until it struck me, just one month ago, that reintroducing this mode of execution was less than 10 lines of code!

Because I had chosen the functional paradigm of "executing a Specification" <=> "interpreting a data structure" it was really easy to change the interpretation to something declaring: "for each example, execute the code in a clone of the Specification so that all local variables are seen as fresh variables".

In terms of product management, it was a "magic" feature almost for free! To me, this is the illustration of a principle of Software: "if code is a liability, maximise your return on investment".

Maximize the ROI

I'm sure you've already read something like: "features are an asset, code is a liability". But I haven't yet read the logical consequence of it: "Maximize the ROI: extract as many features as you can from your existing code". Indeed every time we write a piece of code, it's worth wondering:

• how can this be put to a better use?
• can I add something slightly different to provide a new useful feature?
• can I generalize it a bit to make sense of it in another context?
• can I make it composable with an existing feature to get a new one?

That's exactly what happened with the "isolated" feature above. Here's another example of this principle in action.

IO and serialization are not cheap

A few months ago, I developed a feature to create an index page. On this index page I had to show the status of specifications which had been executed as part of the previous run. This meant that I had to store somewhere, on the file system, the results of each specification execution.

In terms of feature price, this is not a particularly cheap one. Everytime there is some IO interaction and some kind of serialization, the number of possible issues to consider is not so small. In a way that was really an "iceberg" feature: not a lot of functionality on the surface, but a lot of machinery under the water. So I wondered how I could improve my ROI on this. Hmm... since I'm writing status information to the disk there might be a way to reuse it!

Indeed, I can use this information to selectively re-run only the failed examples, or the skipped ones, or... And so on. From there, a new "feature space" opens, and the initial investment starts making sense.

It is also possible to maximize the ROI on existing features. A feature is an "asset". Perhaps, but it's not completely free either. You have to explain it, to promote it, to show when and how it interacts with the rest of the software. This is why maximizing the ROI of features makes sense as well.

That's exactly what happened with the brand-new "isolated" feature.

All expectations

Year after year I'm looking at the specifications that people are writing with specs/specs2. Kind of my hallway usability test for open-source libraries... I usually see several "styles" of specifications and one style is pretty frequent:

    "This is an example of building/getting/processing a datastructure" >> {
       // do stuff
       val data = processBigData

       // check expectations
       data must doThis
       data must haveThat
       data.parts must beLikeThis
       // and so on...
    }

In that style of specification, there is usually one action and lots of things to check. It is very unfortunate that most of the testing libraries out there will stop at the first failure. Because it really makes sense to collect all of them at once instead of having to execute the specification multiple times to get to all the issues.

Is there a way to "collect" all the failures in specs2-1.8.2? Not really. You can try use "or" with the expectations:

       (data must doThis) or
       (data must haveThat) or
       (data.parts must beLikeThis)

And that will collect all the failures up to the first success, and then it will stop executing the rest. Besides, the "or" syntax with all the parenthesis is not so nice.

After a while I realized that the only way to make this work was to use yet another mutable variable to silently register each result. But then I'd be back to the old specs1 problem. What about concurrency? How can I prevent each concurrent example to step on another example results?

Well, that's easy, I have the "isolated" feature now! Each example can run in its own copy of the specification and the mutable variable will only collect the results from that example safely. The result is a feature that's easy to implement but also easy to use because it's just a matter of mixing a trait to the specification!

Coming full circle

Can I go further with the same thinking? Why not?!

From the beginning of specs2, I liked the fact that the so-called "Acceptance specification" style allowed me to write a lot of text about what my system behavior and then annotate it with the actual code. The price to pay was all the symbols on the right of the screen which appeared utterly cryptic to some people (to say it nicely).

Then, last week, I realized that I could rely on something which exists in most code editors: code folding! In a classical JUnit TestCase, specs1 specification or specs2 mutable specification if you fold the code, you're left with only the text, forming a narrative for your system. If you see it like that, any specs2 mutable specification can be turned into an acceptance specification, just a few features are missing:

• the ability to add an arbitrary piece of text (instead of should/in blocks) and formatting fragments (for example to give an overview of the system, before going into the details)
• given/when/then specifications
• auto-examples

Not a lot to implement really. Most of the machinery is already provided by the org.specs2.Specification trait. This means that, for a very reasonable "price", you can now use "mutable" specifications in specs2 with a great deal of expressivity (nothing's perfect though, there are small issues due to semi-column inference for example, see here for variations around the "Stack" theme).

Full circle and blurred lines

To me, it really feels that I've come full circle:

1. I rebuilt from scratch the "Specification" concept from specs1, throwing away the syntax
2. I reintroduced some mutability very carefully to get some of that syntax back
3. I built upon all the immutable code to finally end up with exactly what I would have liked to have in specs1!

My only concern now is that newcomers might feel lost because the library is not really prescribing a specific style: "should I use a 'unit' style or an 'acceptance' style? And what are the differences between the 2 anyway?". I hope that they'll realize that this is actually an opportunity. An opportunity to try out different ways of communicating and then choosing the most efficient or pleasing.