Shine Technologies
Experiences with Mock Objects
Experiences with Mock ObjectsThis page assumes some knowledge of DynaMock and Mock Objects. For more information, see here. As mentioned earlier, I recently used DynaMock and the Mock Object pattern on a medium-sized project. In particular, they were used extensively on a batch-processor component that processed XML data into a database. During the development of this component I had to grapple with a number of issues relating to the best way to apply the Mock Object pattern. The batch-processor only did one thing, but the one thing that it did was reasonably complex. It had many levels of method calls and whilst some recurring patterns emerged during development, there were limited opportunities for obvious, easy reuse. In some ways it could be thought of as a 'deep' application as opposed to a 'broad' one like a typical web application (which could do many things, but did probably did not need to do as much in each case). Everything By InterfacesAs was discussed here, any object interaction you want to mock must be done via interfaces. Whilst this can be a pain, design-by-interface is also a proven strategy for good software design. The only debating point with regards to mock objects is whether their usage requires design-by-interface to be taken to a point where it is no longer useful - ie, whether the cost outweighs the benefits. Whilst I cannot comment decisively on this right now, one thing I do advise is that you clearly document why you are using interfaces all the time. Some programmers get frustrated by what they see as excessive use of interfaces - this would at least give them some explanation. I found it a bit tedious putting this documentation in every interface, so in the end I just defined a 'Mockable' interface and instituted a policy of making any new interfaces extend it. The comments of the 'Mockable' interface explained that any sub-interfaces were intended to be mocked up at some stage - and hence the use of an interface. Note that this interface was just a 'marker' interface (in a similar manner to java.io.Serializable or java.lang.Clonable) and did not contain any methods. There are some technical tricks that can be used to get around the everything-by-interfaces restriction but they are rather advanced. JMock includes a facility that allows you to create mocks of classes as well as interfaces Dodgy Service LocatorsThis section assumes that you are familiar with the concepts of Dependency Injection and the Service Locator pattern. If not, Martin Fowler's article on the subject As already discussed, one of the big implications of using Mock Objects is that the object being tested has to be 'given' its collaborators. At times dependency injection was the easiest way to do this. However, when collaborators were only going to be used way down the call chain, it becomes a nuisance to have to be continually passing them through from one method to another when they were not actually being used en route. This is when it seemed appropriate to use a Service Locator. The problem is that Service Locators use Singletons The problem was that if, for the purposes of unit testing, I overrode the implementation of some service with a mock implementation and then, after the test, forgot to set it back to the original implementation, then subsequent tests that did require the real implementation would fail (often in quite an obscure manner). To make matters worse, this would only happen if the tests were all being run in the same Virtual Machine (for example, using the Eclipse JUnit runner). Tests running in separate VMs (for example, using the Maven JUnit support) would not experience this problem. The way around this was that I always had a 'tearDown' method in my tests that reset the Service Locator to its original value. Note that this problem isn't exclusive to mock objects; it could occur under any circumstances where you're using the Service Locator pattern. Mocking Legacy APIsOne of the big assumptions about using something like DynaMock is that all of your collaborators can be defined with interfaces. Sometimes this is the case - the JDBC API is defined with interfaces - but sometimes it is not; for example, most of the java.io package is defined by classes. To get around this, interface definitions of common class-based APIs have been built and are available via http://www.mockobjects.com For example, the alt.java.io.File interface defines an interface that is identical to that of the java.io.File class. You can use this interface to create mock implementations of a File that your objects can interact with. When, at integration time, you need to use a real implementation of a java.io.File, you can wrap it in an alt.java.io.FileImpl class and pass that to your objects instead. This was useful when I was testing code that needed to navigate files and directory structures. I could write tests that my program would respond correctly to files that didn't exist, or files that were actually directories, without having to actually set up real (or non-real Object ConstructionOne of the biggest problems I encountered concerned if and how to mock object construction. Regularly I would need to test some method that, during its operation, happened to create some non-trivial object. Problems would occur if the constructor of that object:
Under such circumstances it could be difficult to continually construct the object in a manner that would allow testing to occur. Furthermore, from the perspective of testing the method, I didn't care about what was happening inside the object constructor; all I cared about was that the object was created, the constructor was being passed the right parameters, and that other 'mock' calls could be made to the object after construction. If I actually wanted to test the constructor properly, I'd write a separate unit test to do it. So in this sense, I wanted to set an expectation on a mock object that its constructor would be called. To get around this problem, I began using factories for object creation. The factories were configurable singletons, so that mock implementations could be plugged in at runtime: As an example, consider that we are testing the following method: void someMethodBeingTested()
{
...
ComplexObject complexObject = new ComplexObject(1);
...
}
To test object creation, we would do define the following factories: interface Factory
{
ComplexObject
createComplexObject(int value, String string);
}
class FactorySingleton
{
Factory getInstance();
void setInstance(Factory factory);
}
And then modify the original method so that it used the factory: void someMethodBeingTested()
{
...
ComplexObject complexObject =
FactorySingleton.createComplexObject(1);
...
}
The test of the method would then look like: void test()
{
// Set up the factory and its expectations.
Mock factoryMock = new Mock(Factory.class);
factoryMock.expectAndReturn(
"createComplexObject",
1,
(ComplexObject) new Mock(ComplexObject.class).proxy()
);
FactorySingleton.setInstance(
(Factory) factoryMock.proxy()
);
// Now exercise the method.
someMethodBeingTested();
// And check that it interacted with the factory in the
// right way.
factoryMock.verify();
}
Whilst this can be a bit fiddly, I found it allowed me to confidently keep my tests fine-grained and focused. The need to separate the Factory interface from the Singleton class is also a nuisance. A possible way around this could be to use JMock and leverage its ability to mock classes Often, if there were a whole bunch of different object creations I wanted to mock, I'd put them all in the same factory. If the objects were only visible within a particular package, then I would only make factory visible within that package. Essentially, the factory can be thought of as putting a layer of indirection between the object being tested and the use of the Java 'new' keyword that is required to create a new instance of a collaborator. The 'new' keyword is a fundamental and indispensable part of the language. Intercepting it is a non-trivial matter and factories were the best way I could see to go about doing it using Java only. For the more adventurous, there is at least one possible alternative: use AspectJ to intercept instantiations of a particular class, check that the parameters are as expected, and then return a JMock-generated instance instead of a real one. This eliminates the use of intrusive factories. However, I've only experimented with this in my own time and haven't tried it on a real project. Feel free to contact me Testing Methods that Call Other MethodsAnother problem I had was testing a method that called another method within the same class. For example: void method()
{
...
int result = method2("hello");
...
}
private int method2(String string)
{
...
}
If I want to test the details of 'method', I might not want to call the real 'method2'. All I want to do is make sure that 'method2' is called with the right arguments, and that 'method' deals correctly with all of the different things that can be returned (or thrown) by 'method2'. If I want to test 'method2' properly, I'll write another test to call it directly (being a private method, the TestCase would have to live in the same class). The way around this was to use an interface to access 'method2'. To keep it private, I could make my test case a member class of 'method's class. The test can override the default 'real' implementation of the method with a mock implementation. Testing Big MethodsConventional wisdom has it that methods shouldn't really be larger than one screen. The use of Mock Objects for unit testing compounded the need for this practice, as methods that are larger than one screen proved especially difficult to test in detail. In fact, I was finding some piece of functionality difficult to write a test for, it was usually a pretty good sign that the functionality should be broken into parts. Patterns of TestingAs I wrote more and more unit tests using DynaMock, I became aware of recurring patterns of usage. Say that I was testing the Processor.doStuff(int) method described in the DynaMock example above. Suppose that the first thing that I expected is that if it the value passed in was less than zero, it would throw an exception with the message. To Be Continued Missing FunctionalityOne problem I had was that I would get so focussed on testing a particular piece of functionality that I would completely forget about implementing something else. So whilst what I had coded was thoroughly tested and had a high likelyhood of working correctly, there would be items in my requirements document that I had totally forgotten. Whilst they were not very major, this felt rather embarrassing Fortunately, in my particular case, such omissions became apparent quite early. Peer review helped, as did early system testing. Regular 'sanity checks' by me of the requirements against the completed code helped, although after a whilst I would become 'blind' to sections of the documents I had regularly been over. As has already been discussed, it also helped to have at least a few end-to-end tests of key functionality - be they automated or non-automated. The Limit of Unit TestingHaving employed mock objects to test at a very fine level of detail I reached a rather disconcerting point: if I carefully broke my code into classes and methods, and then broke my methods down further into chunks that could be easily tested, all I was really doing was testing for-loops and if-statements. Sure, I was checking that objects interacted with one another correctly, but ultimately these interactions were just occurring within primitive control structures. Furthermore, I found that when I got down to this level of detail, the real code and the test code began to mirror one another. And perhaps most disturbingly, it became apparent that I could write more code to test an if-statement or for-loop than would actually be required to write the statement itself. This rapidly led to the question: is it really worth testing at this level of detail? Whilst I'm not %100 sure of the answer to this, a couple of things spring to mind:
Moving On...Having been over the experiences I had with Mock Objects on a real project, now is probably a good time to wrap up and talk about the conclusions I came to. |
) files. Of course, it was still worth writing a couple of tests that worked with real files, but only for testing the overall health of the system.