Software design is like a magic trick

There's something I love at least as much as software development: my wife (Hey!! Give me back that keyboard, will ya?. I love you *more* than software development of course,...). So, what was I saying, ah, yes. There's something I love at least as much as software development: magic.

And there are some interesting similarities between the design of a magic trick and the design of software. Let's take, as an example, the last functionality I have designed for specs (but not released yet).

The effect

When you start thinking about a magic trick, you just try to be creative, to focus on what will really make other peoples brain explode. Or, and I find this even more impressive, transport them emotionally in a real magic world. To that respect David Copperfield is an incredible magician, and there are lesser known magicians whose creativity and talent are as amazing: Jay Sankey, Juan Tamariz, Michael Ammar, Jean-Pierre Vallarino, Vito Lupo,... The list is long. All of them have spent hours just thinking: "Why is this magic?", "How can I make it more magic?". They think about the spectator and the effect first and before any consideration of how it could be done.

In my case, it's a lot less romantic,... I just want the developer to be able to:

• create a table containing data rows
  
• have a header of Strings labeling each column
• have the rows being typechecked so that the type of the elements in a given column is always the same
• apply a function to each row and have the function parameters being typechecked against the row types
  
• have a light syntax allowing the table to look like a table as much as possible (not a list of lists,...)

Something like that:
|"a"|"b"|"c = a + b"|
| 1 | 1 | 2 |
| 1 | 2 | 3 |
| 2 | 2 | 4 | {(a:Int, b: Int, c: Int) => (a + b) must_== c }

How can I use my target language, Scala, to do that? Without loosing the ideal representation above; I want to keep my effect as magical as possible!

The method

Let's say I want to make the Statue of Liberty disappear, this would be baffling, wouldn't it? But even with an incredible budget, will the New-York authorities allow me to place a huge trap below Ellis Island? No? So how did he do it ?!

Here are 3 things I noticed with great magicians:

• They have an incredible magic culture. They know every obscure method to make a coin disappear or a card change its color, including the name of the inventor, the year of publication and the thousands of tricks using it
  

• They are always open to new stuff. Anything. Okito was just playing with a pill box when he invented the "Okito box". Michael Ammar actually listened to every single "great new idea" we had at our local magic club!
  

• They can invest an incredible amount of time and energy to preserve the magic of the effect: exercise for hours for a single "invisible" move, research chemical catalogs, learn the order of 6 decks of cards by heart (you can find a Hollywood version of that in "The Prestige")
  

That's what I tried to do with the DataTable feature: use all the Scala wizardry I knew of, be opened to new ideas and invest the maximum of energy so the feature is as simple as can be for the developer.

Here was my first attempt:
("a", "b", "c = a + b") |
( 1 , 1 , 2 ) |
( 1 , 2 , 3 ) |
( 2 , 2 , 4 ) | { t => val (a, b, c) = t
(a + b) must_== c }In that version, I used the following features:

• Tuples have a litteral notation ( , , , )
• You can create new operators, like | on tuples using implicit definitions
• The definition of the | operator can make sure that I will always add Tuples having the same types for their elements
  
• A tuple can be deconstructed as 3 values using "val"
  

This was mostly what I wanted but not quite, so I searched again and eventually got to this:
"a"| "b"| "c = a + b" |
1 ! 1 ! 2 |
1 ! 2 ! 3 |
2 ! 2 ! 4 | {(a: Int, b: Int, c: Int) => c must_== calc.add(a, b) }

Yes, almost my original intention! The most notable difference is the use of ! instead of | as a separator for the cells in the data rows. This is because operators beginning with | have a lower precedence over those beginning with ! in the Scala specification (6.12.3). So if I use |, I will have difficulties delimiting rows. I learned a new trick!

Frankly, it's not as good as the ideal version but it's ok if you read | as a delimiter for the table boundaries (including the header) and ! for the actual data. Sometimes in a magic trick you change a line, bend the story a little and it is still a pretty good trick.

And what about the rest of the magic? The first version only allowed to define a function taking a tuple as a parameter. Well, I worked a lot behind the scene to obtain that new version:

• The DataTable class takes 20 type parameters. 20 is a limitation but should act like a warning anyway. Why would you seriously need more that 20 columns in your table? Can't it be refactored?
  

• There are 20 DataRow classes, having from 1 to 20 type parameters. Each DataRow object can only be joined with a similar DataRow object using the | operator
  

• The DataTable class has 20 methods to apply functions having 1 to 20 parameters. Applying each row to the function parameters is done in each of these methods. A nice side-effect of that is that I can use a function with less parameters than the row size
  

• All that code is produced with a Ruby script to ease the work of the magician developer
  

You can have a look at the current code, which may actually be clearer than the points above.

Design and magic

This comparison with Magic can add more credits to the "Design is a craft" theory. I'm pretty sure that you could do the same kind of comparison with some craft you're fond of. So what's next: "Software design is like Origami", "Software design is like Wood carving"?

PS: for the Statue of Liberty trick, just Google for the explanation, you should get a glimpse of the kind of creativity those darn magicians can have!