CircleCI – Running Parallel iOS Test Jobs with Workflows

With the release of CircleCI 2.0, a new feature called Workflows is now included with the platform. Workflows allow you to specify how each of your individual jobs are related and define criteria about which jobs should be executed, and when.

Workflows also have the ability to share data between jobs using a shared workspace. For most projects, this is pretty straightforward, but I recently had an issue using this feature on an iOS app that I figured I should document for others who might encounter it.

In this post, I plan to share an example of using Workflows with a sample iOS app. All of the code that I share here can be found at this GitHub repository.

Running an iOS Test Job

In my sample app, there are five unit tests that I want to run on both iOS 11.4 and iOS 12.2. We could accomplish this with the following CircleCI config that uses the xcodebuild tool to build the app and run the tests.

# Running tests twice within the same job

# Make sure we're using CircleCI 2.0
version: 2
jobs:

  build:
    macos:
      xcode: "10.2.1"

    shell: "/bin/bash --login -eo pipefail"

    steps:
      - checkout
      - run: 
          name: Run Unit Tests For iOS 11.4
          command: |
            xcodebuild test \
            -scheme Hello_World \
            -sdk iphonesimulator \
            -destination "platform=IOS Simulator,OS=11.4,name=iPhone 7"

      - run: 
          name: Run Unit Tests For iOS 12.2
          command: |
            xcodebuild test \
            -scheme Hello_World \
            -sdk iphonesimulator \
            -destination "platform=IOS Simulator,OS=12.2,name=iPhone 7"

This will run both sets of tests (one for iOS 11.4 and another for iOS 12.2), but the execution of the tests for iOS 12.2 depends on the tests for iOS 11.4 passing. Even if all of the tests were able to pass on both OS versions, we would have to wait for the entire test suite to execute twice before we could get the feedback that we need.

Running iOS Test Jobs in Parallel

We can speed things up by splitting the single job into two. This will allow our tests to run in parallel between the two OS versions. What’s more, neither set of tests relies on the other passing before it can execute (unless we wanted to configure our Workflow that way for some reason).

# Using a workflow to run 2 jobs in parallel

# Make sure we're using CircleCI 2.0
version: 2
jobs:

  test_ios_11:
    macos:
      xcode: 10.2.1

    shell: /bin/bash --login -eo pipefail

    steps:
    - checkout
    - run:
        name: Run Unit Tests
        command: |
          xcodebuild test \
          -scheme Hello_World \
          -sdk iphonesimulator \
          -destination "platform=IOS Simulator,OS=11.4,name=iPhone 7"

  test_ios_12:
    macos:
      xcode: 10.2.1

    shell: /bin/bash --login -eo pipefail

    steps:
    - checkout
    - run:
        name: Run Unit Tests
        command: |
          xcodebuild test \
          -scheme Hello_World \
          -sdk iphonesimulator \
          -destination "platform=IOS Simulator,OS=12.2,name=iPhone 7"

workflows:
  version: 2
  test:
    jobs:
      - test_ios_11
      - test_ios_12

This certainly helps shorten the feedback loop with each change, but we’re spending additional time to build the app for every job in our Workflow. MacOS images on CircleCI aren’t cheap, especially compared with their Linux images. Anything that can be done to limit the total execution time of the Workflow is strongly encouraged.

One Build, Multiple Parallel Test Jobs

The ideal scenario would be to have one job that utilizes the build-for-testing action from xcodebuild to build the app for testing, persist the necessary files to the shared workspace for the Workflow, and copy those folders from the shared workspace to be used in any corresponding test jobs. These test jobs should be able to use the test-without-building action to prevent rebuilding the app when the tests are run. You can read more about these actions here.

Patrick Star is correct. When the argument is stated this way, it seems like it should be a simple series of tasks. Unfortunately, on CircleCI, MacOS images are executed as a user that doesn’t have the necessary file system permissions for the default location of the DerivedData folder. To get around this, we need to utilize the -derivedDataPath argument from xcodebuild to specify a different folder of our own making that will store the build sources.

# Using a workflow to run 2 jobs in parallel with only one build

# Make sure we're using CircleCI 2.0
version: 2
jobs:

  test_build:
    macos:
      xcode: "10.2.1"

    shell: "/bin/bash --login -eo pipefail"

    steps:
      - checkout
    
      - run:
          name: Setup New DerivedData Folder
          command: mkdir DerivedData

      - run: 
          name: Build For Testing
          command: |
            xcodebuild build-for-testing \
            -scheme Hello_World \
            -sdk iphonesimulator \
            -destination "platform=IOS Simulator,OS=11.4,name=iPhone 7" \
            -derivedDataPath DerivedData

      - persist_to_workspace:
          root: DerivedData/
          paths:
            - "**/*"

  test_ios_11:
    macos:
      xcode: "10.2.1"

    shell: "/bin/bash --login -eo pipefail"

    steps:
      - checkout

      - attach_workspace:
          at: /tmp/workspace

      - run:
          name: Setup new DerivedData folder
          command: |
              mkdir DerivedData
              cp -r /tmp/workspace/ DerivedData/

      - run: 
          name: Run Unit Tests
          command: |
            xcodebuild test-without-building \
            -scheme Hello_World \
            -sdk iphonesimulator \
            -destination "platform=IOS Simulator,OS=11.4,name=iPhone 7" \
            -derivedDataPath DerivedData

  test_ios_12:
    macos:
      xcode: "10.2.1"

    shell: "/bin/bash --login -eo pipefail"

    steps:
      - checkout

      - attach_workspace:
          at: /tmp/workspace

      - run:
          name: Setup new DerivedData folder
          command: |
              mkdir DerivedData
              cp -r /tmp/workspace/ DerivedData/

      - run: 
          name: Run Unit Tests
          command: |
            xcodebuild test-without-building \
            -scheme Hello_World \
            -sdk iphonesimulator \
            -destination "platform=IOS Simulator,OS=12.2,name=iPhone 7" \
            -derivedDataPath DerivedData 

workflows:
  version: 2

  build_and_test:
    jobs:
      - test_build 
      
      - test_ios_11:
          requires:
            - test_build

      - test_ios_12:
          requires:
            - test_build

As you can tell from the screenshot at the top of this section, our total execution time is technically longer than it would be if we allowed both jobs to build the app themselves. For this small sample application, that is certainly the case. However, as the app becomes more complex and the build times increase, the difference is clear. The project that I’m currently working on takes 12 minutes to compile from a clean environment, which I believe makes this approach worthwhile.


I hope that you’ve found this helpful in some way, even if you don’t necessarily work on an iOS project. Have you tried Workflows on CircleCI?