Adding Integration Tests to Gradle

It’s well understood that unit tests alone may not be sufficient to release new code with confidence. Integration/functional tests are used to automate testing for workflow regressions. As part of setting up a new Java REST API server, I wanted to embed integration testing into our regular workflow via Gradle.

Gradle supports additional groups of Java source code (such as the integration test code) via source sets. Although there are plugins that simplify configuration for new source sets, I hesitated to use them. Libraries and plugins maintained by a single developer in their spare time are always at risk of abandonment, and support can be sporadic.

My first, optional, step was to create a file, integration-test.gradle, to encapsulate my integration test Gradle configuration. I then added apply from: "$rootDir/integration-test.gradle" to build.gradle.

Inside integration-test.gradle, I set up the following:

sourceSets {
    integrationTest {
        compileClasspath += sourceSets.main.output + configurations.testCompile
        runtimeClasspath += output + compileClasspath + configurations.testRuntime
    }
}

idea {
    module {
        testSourceDirs += sourceSets.integrationTest.java.srcDirs
        testResourceDirs += sourceSets.integrationTest.resources.srcDirs
        scopes.TEST.plus += [ configurations.integrationTestCompile ]
    }
}

task integrationTest(type: Test) {
    description = 'Runs the integration tests.'
    group = 'verification'
    testClassesDirs = sourceSets.integrationTest.output.classesDirs
    classpath = sourceSets.integrationTest.runtimeClasspath
    outputs.upToDateWhen { false }
    mustRunAfter test
}

check.dependsOn integrationTest

2: Defines the name of the new source set.
3: Includes the compiled main code and (unit) test compilation dependencies when compiling integration tests.
4: Includes the compiled integration test code, the previous inclusions, and test runtime dependencies when running integration tests.
10: Correctly marks the integration test’s Java source directories for Intellij IDEA’s configuration.
11: Correctly marks the integration test’s resource directories for Intellij IDEA’s configuration.
12: Configures Intellij IDEA to scope the integration tests as TEST.
16: Creates a new Gradle task called integrationTest. You can think of this task as an instance of the Gradle Test class.
17: (recommended) Describes this task for reports and user interfaces such as when running ./gradlew tasks.
18: (recommended) Groups this task under Verification for reports and user interfaces such as when running ./gradlew tasks.
19: Sets the code to test as the compiled code from the integrationTest source set.
20: Sets the runtime classpath to be as defined in the integrationTest source set.
21: Forces Gradle to always run the integration tests when asked to. By default, Gradle attempts to optimize task execution by not re-running tasks whose inputs have not changed. Since integration tests may fail due to external systems, we want to run them even if no code has changed.
22: (optional) Enforces task ordering, not task dependency. Unit tests run fast, so we want to avoid running integration tests if unit tests fail. However, we use mustRunAfter test rather than dependsOn test because we do not always want to run unit tests when we run integration tests.
With mustRunAfter test:

• ./gradlew integrationTest test runs unit tests, then integration tests.
• ./gradlew integrationTest runs integration tests.

With dependsOn test:

• ./gradlew integrationTest test runs unit tests, then integration tests.
• ./gradlew integrationTest runs unit tests, then integration tests.

25: (optional) Enforces that integration tests will be run when ./gradlew check is run. Semantically, I expect the check task to verify all systems (including integrations) are working.

You should now be able to run ./gradlew integrationTest and see your integration tests run successfully.

Note

If you intend to use this for a project that will run on Heroku, you may want to add an additional step.

By default, Heroku runs ./gradlew build -x test when new code is deployed. Since this command does not skip integration tests, it may do much more than you or Heroku expect. Heroku allows you to override this by providing a Gradle stage task. Although you could be precise with the stage task, I simply chose to alias the assemble task with task stage(dependsOn: ['assemble']) and skip checking altogether.