Unit Testing with State Machines

Hello again, readers. Welcome once more to a consideration of just some of the many advantages of writing software using explicit state machines. In the first post in this series I introduced the idea of an explicit state machine and explained some of the benefits of using them. In the second post, I expounded on some approaches to defining explicit state machines in Python.

In this post I’ll discuss some of the advantages of state machines with respect to automated unit testing. When the system under test is implemented using an explicit state machine certain useful assertions become easy to generate automatically. This has multiple benefits including reducing the overall cost of developing and maintaining the test suite as well as improving the overall quality of the test suite.

Recall that in the last example the states and inputs of the state machine were explicitly enumerated as objects:

from twisted.python.constants import Names, NamedConstant

class States(Names):
    LOCKED = NamedConstant()
    UNLOCKED = NamedConstant()
    ACTIVE = NamedConstant()

class Inputs(Names):
    FARE_PAID = NamedConstant()
    ARM_UNLOCKED = NamedConstant()
    ARM_TURNED = NamedConstant()
    ARM_LOCKED = NamedConstant()

One thing we can easily infer from these two enumerations is that there are at least twelve cases to test: (LOCKED, UNLOCKED, ACTIVE) × (FARE_PAID, ARM_UNLOCKED, ARM_TURNED, ARM_LOCKED). Through some simple metaprogramming it’s possible to automatically verify that all of these tests have been written. For example, if you’re using Python’s unittest module you might start off with a tool like:

def testsFor(states, inputs):
    """
    Construct a mixin class to be used with a L{TestCase}
    subclass which defines a minimal set of tests that
    must be written.
    """
    cases = (
        (state, input)
        for state in states.iterconstants()
        for input in inputs.iterconstants())

    template = "test_%s_%s"
    tests = dict(
        (template % (state.name.lower(), input.name.lower()),
         lambda self, state=state, input=input:
             self.fail(
                 "Did not test input %s in state %s" % (
                     input, state)))
        for (state, input) in cases)
    return type("StateMachineMixin", (object,), tests)

class TurnstileTests(TestCase, testsFor(States, Inputs)):
    pass

Running this test suite produces twelve failures because I haven’t written any of the tests that are required:

$ trial test_turnstile.py                                                                                                              
test_turnstile
  TurnstileTests
    test_active_arm_locked ...    [FAIL]
    test_active_arm_turned ...    [FAIL]
    test_active_arm_unlocked ...  [FAIL]
    test_active_fare_paid ...     [FAIL]
    test_locked_arm_locked ...    [FAIL]
    test_locked_arm_turned ...    [FAIL]
    test_locked_arm_unlocked ...  [FAIL]
    test_locked_fare_paid ...     [FAIL]
    test_unlocked_arm_locked ...  [FAIL]
    test_unlocked_arm_turned ...  [FAIL]
    test_unlocked_arm_unlocked ...[FAIL]
    test_unlocked_fare_paid ...   [FAIL]

This is a great starting place for development. I now have a test suite telling me twelve specific things to test and then implement. When I finish implementing those twelve tests I may not be completely done but I’ll have at least one test for each input in each state. And if I’m very careful about representing all inputs and states explicitly then it may turn out that I am done.

There are many more testing helpers like this one that become possible when you have an explicit state machine. One I’ve found useful is a collection of APIs for creating a new instance of the class and putting it into a specific state. With these in hand, testsFor above might be extended to include a setUp method that creates an instance of the class in the right state. Something like:

    def setUp(self):
        self.machine = machineInState(Turnstyle, state)

(suitably parameterized) so that each test method can skip the boilerplate of creating the object to be tested and just start testing using self.machine.

This is just the beginning of the possibilities for enhanced unit testing. At ClusterHQ I’ve been exploring how explicit state machines can simplify the implementation of and improve the test coverage for some of our core replication functionality – with an eye towards expanding this implementation style throughout the project.

Get Involved

Sign up for email updates about Flocker