Testing a complex API

We all know the value of testing, and especially the importance of unit testing. However, it is sometimes quite a challenge to properly test some pieces of code.

What about testing classes that access an API that uses complex data structures as input and output?

Testing the Eclipse Modeling Framework

One example of this is the Eclipse Modeling Framework, but I am sure you will find plenty of examples of your own.

This kind of API offers points of entry to modeling data structures such as "PackageRegistry". A PackageRegistry has "getAllPackages" method returning Package objects.

Then, each Package object can be browsed for its content: Class objects, Diagram objects, nested Packages,...

Package, Class, Diagram,... are all interfaces to a complex data structure that can be navigated in order, for instance, to generate code or to create HTML reports. But how do you test this transformation code?

An "out-of-the-box" answer is to use mock objects to create the objects accessed through the API. A fine Mock object framework such as jmock is one of the best for that purpose.

However, setting up the mock structure is:

1. extremely painful (such as in "extracting-my-teeth-one-by-one")
2. near to unreadable

Here's what it looks like:

Mock packagesRegistry = mock(PackagesRegistry.class);
List packages = new ArrayList();
packagesRegistry.expects(once()).method("getAllPackages").will(returnValue(packages));

// now create one mock package
Mock package = mock(Package.class);
List classes = new ArrayList();
package.stubs().method("getName").will(returnValue("pack1"));
package.stubs().method("getClasses").will(returnValue(classes));

// and so on,...

This code is so awful that it is almost useless. We need a fresh approach for this kind of unit testing: a mocking DSL!

A mocking DSL

Well, it may be far fetched to call that a DSL, but the idea is to use operation calls to represent the mocks structure and the expected stubs (with real-life code sample):

topPackage = (Package) create(
mock(Package.class),
stub("getFullName", "model"),
stub("getOwnedMembers",
elist(createClassOne())),
stub("getNestedPackages",
elist(create(
mock(Package.class),
stub("getFullName", "model.pack1"),
stub("getOwnedDiagrams", elist(createDiagram())),
stub("getOwnedMembers", elist()),
stub("getNestedPackages",
elist(create(
mock(Package.class),
stub("getFullName", "model.pack1.pack2"),
stub("getNestedPackages", elist())))))))));I let you imagine the same mock structure with "classical" declarations,...

Even here, this is not a revolutionary improvement:

1. it is still quite verbose: more complex data structure may not be readable at first glance (but do you need them for unit testing?)
2. instead of having "create", "stub", "mock" methods, "package", "klass", "method"may be closer to the domain. If you are to test heavily the complex API, I would suggest to do so.
   

The MockBuilderTestCase class

The "create", "mock" and "stub" operations are offered by a MockBuilderTestCase class subclassing the MockObjectTestCase (see the jmock doc).

This is fairly simple code:
public class MockBuilderTestCase extends MockObjectTestCase {
private Stack mockStack = new Stack();

public Mock mock(Class klass) {
mockStack.push(super.mock(klass));
return currentMock();
}

private Mock currentMock() {return mockStack.peek();}
protected Mock callOnce(String methodName, Object value) {
currentMock().expects(once()).method(methodName).will(returnValue(value));
return currentMock();
}

protected Object create(Mock mock) {
mockStack.pop();
return mock.proxy();
}

protected Object create(Mock mock, Mock...) {
return create(mock);
}

protected Mock stub(String methodName, Object value) {
currentMock().stubs().method(methodName).will(returnValue(value));
return currentMock();
}
}
As "mock" operations are encountered when java executes the topmost "create" function, a new mock object is created and pushed on a Stack. Then all subsequent "stub" operations are applied to that mock. Eventually, when the "create" operation has all its arguments evaluated, the mock is ready and can be removed from the stack.

I wish I could turn the "eager/lazy" knob for evaluating arguments in java. I often find that using nested operation calls in java can provide an easy internal DSL, however, having the operation arguments evaluated first is not always practical.

Don't forget the Law of Demeter!

A last tip for an efficient and easy testing of classes that uses a complex API: don't let them access a too deep level in the API hierarchy.

That is, instead of allowing modelBrowser.getOperationNames() be like

List result = new ArrayList();
for (klass: getAllClasses()) {
for (operation: klass.getOperations()) {
result.add(operation.getName);
}
}
aim for delegation to classes dedicated to lower levels:

List result = new ArrayList();
for (klass: getAllClasses()) {
result.addAll(classBrowser.getOperationNames(klass));
}