Acceptance Testing C++ with Cucumber and the Wire Protocol

Acceptance Test Driven Development has become a popular practice for establishing and validating what done means for a feature. Cucumber has gained much popularity in validating Ruby code, especially for Ruby on Rails web applications. The Wire Protocol was added to Cucumber to support testing different languages by implementing a simple Wire Server in the target language. The Wire support allows step definitions to be written in an arbitrary language with communication over a TCP socket.

The following example shows how this works with C++ using Cucumber-Cpp (thanks Paolo Ambrusio) and an example that comes bundled with it.

Features are written in Gherkin, just like standard Cucumber tests. Gherkin is a business-level DSL (Domain-Specific Language) that allows test to be written in high level terminology. This allows tests to be written and understood by non-developers, thus facilitating the conversion of what done should mean for a given feature. Below is a feature definition and scenario for a simple calculator application to validate division:

Feature: Division
  In order to avoid silly mistakes
  Cashiers must be able to calculate a fraction
  Scenario: Regular numbers
    Given I have entered 3 into the calculator
    And I have entered 2 into the calculator
    When I press divide
    Then the result should be 1.5 on the screen

The implementation of the step definitions for all features and the corresponding support for setting up the test fixture are contained in a C++ file. Below is the source file that implements all the needed step definitions for the feature file above, though it also includes step definitions for the rest of the feature files in the test suite. At the top of the file is the implementation of the Context class that is instantiated prior to executing each feature scenario, and is destroyed when the scenario terminates. In the current implementation of Cucumber-Cpp, exceptions (including those in test assertion frameworks) cannot occur in the Context constructor nor destructor. There is a request to in to resolve this limitation.

Wire communication is done via a simple protocol encoded in JSON. Executing the feature file above yields the output below, which is the same output you would get from a true Cucumber test. Although, the C++ step definitions  get annotated with the source file and line number for cross referencing.

Feature: Division
  In order to avoid silly mistakes
  Cashiers must be able to calculate a fraction
  Scenario: Regular numbers                     # division.feature:6
    Given I have entered 3 into the calculator  # CalculatorSteps.cpp:11
    And I have entered 2 into the calculator    # CalculatorSteps.cpp:11
    When I press divide                         # CalculatorSteps.cpp:22
    Then the result should be 1.5 on the screen # CalculatorSteps.cpp:27
4 scenarios (4 passed)
16 steps (16 passed)
0m0.073s

Adding some debug output to Cucumber shows how the execution happens across The Wire:

Scenario: Regular numbers # division.feature:6
> ["begin_scenario"] # Cucumber sends this message to the Wire server to start the scenario
< ["success"] # Each message sent to the Wire server is acknowledged, and may include response data as well
> ["step_matches", # Cucumber sends the first step of the scenario
    {
      "name_to_match":"I have entered 3 into the calculator"
    }
  ]
< ["success", # The Wire server responds with the regular expression that matches the specified step, including the extracted parameter and location of the step definition
    [
      {
        "regexp":"^I have entered (\\d+) into the calculator$",
        "args":[{"val":"3","pos":15}],
        "id":"1",
        "source":"CalculatorSteps.cpp:11"
      }
    ]
  ]
> ["invoke", # Cucumber now tells the server to go ahead and execute the step
    {
      "args":["3"],
      "id":"1"
    }
  ]
< ["success"] # The server responds with the result
 Given I have entered 3 into the calculator # CalculatorSteps.cpp:11 (This is the normal formatted output that appears on the console after a step is executed)

As you can see, the protocol and data exchange is pretty straight-forward. In addition to Cucumber-Cpp, there are Wire implementations for other languages as well.