Android Emulator on CircleCI’s MacOS Executor

Article summary

The Android Emulator, unfortunately, doesn’t work in CircleCI’s conventional (Docker-based) Android build environment. With a little tinkering, though, we can make it work in another environment!

##What Doesn’t Work

Since CircleCI 2.0, the recommended build environment for most projects is the [Docker Executor][executor types]. Overall, it’s great: Docker images are fast, portable, and cacheable. Chances are you can start with a [prebuilt one][prebuilt images].

One of the jobs in our current workflow boots up the `circleci/android:api-29-node` image in about four seconds with all the build tools we need. For building and publishing, this is fantastic.

Unfortunately, when you begin configuring your tests, you’ll soon realize that this environment [can’t run the Emulator](https://circleci.com/docs/2.0/language-android/#overview).

###Why?

To achieve reasonable performance, the Android Emulator needs
[hardware acceleration][hypervisor], which depends on supporting capabilities from the processor and operating system. We can use the Emulator’s `-accel-check` flag to interrogate a system’s compatibility. Here’s what it says in a CircleCI Docker environment:


circleci@b52cf4976e06:~$ uname -a
Linux b52cf4976e06 4.15.0-1043-aws #45-Ubuntu SMP Mon Jun 24 14:07:03 UTC 2019 x86_64 GNU/Linux
circleci@b52cf4976e06:~$ emulator -accel-check
accel:
3
KVM requires a CPU that supports vmx or svm
accel
circleci@b52cf4976e06:~$

(That means “no.”)

But wait! Docker is but one of several [executors][executor intro] available on CircleCI. What if we use a conventional Linux VM instead of Docker? (This is called the [machine executor]).


circleci@default-aa5e8f01-b158-409c-864d-3c5048959c7f:~/repo/App$ uname -a
Linux default-aa5e8f01-b158-409c-864d-3c5048959c7f 4.15.0-1027-gcp #28~16.04.1-Ubuntu SMP Fri Jan 18 10:10:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
circleci@default-aa5e8f01-b158-409c-864d-3c5048959c7f:~/repo/App$ emulator -accel-check
accel:
3
KVM requires a CPU that supports vmx or svm
accel

That doesn’t work either. Bummer.

At this point, you might heed CircleCI’s [advice][circle android] and pursue a third-party service like [Firebase Test Lab] or [AWS Device Farm], but I wasn’t ready to give up yet.

##What Works

We were already using CircleCI’s [MacOS support][circle ios] to build and test our React Native app for iOS. I had one last [wacky idea] to try: could we run the Android Emulator on MacOS?


bash-3.2$ uname -a
Darwin static.162.252.208.208.cyberlynk.net 19.0.0 Darwin Kernel Version 19.0.0: Wed Sep 25 20:18:50 PDT 2019; root:xnu-6153.11.26~2/RELEASE_X86_64 x86_64
bash-3.2$ emulator -accel-check
accel:
0
Hypervisor.Framework OS X Version 10.15
accel

It works!

##Configuration

Without the convenience of an externally-maintained Docker image, it’s on you to install the Android tools. If you want to try Android testing on MacOS, hopefully our configuration can save you some time:


  android-test:
    macos:
      xcode: "11.2.0"
    working_directory: ~/repo/App
    steps:
      - checkout:
          path: ~/repo

      - run:
          name: set ANDROID_SDK_ROOT
          command: |
            echo 'export ANDROID_SDK_ROOT=$HOME/android-tools'  >> $BASH_ENV

      - restore_cache:
          key: android=tools-v1-{{ checksum "scripts/install-android-tools.sh" }}-{{ arch }}

      - run:
          name: install android tools
          command: |
            sh scripts/install-android-tools.sh
            echo 'export PATH=$ANDROID_SDK_ROOT/tools/bin:$PATH'  >> $BASH_ENV
            echo 'export PATH=$ANDROID_SDK_ROOT/tools:$PATH'  >> $BASH_ENV
            echo 'export PATH=$ANDROID_SDK_ROOT/platform-tools:$PATH'  >> $BASH_ENV
            echo 'export PATH=$ANDROID_SDK_ROOT/emulator:$PATH'  >> $BASH_ENV
            source $BASH_ENV
            sdkmanager --list

      - save_cache:
          key: android=tools-v1-{{ checksum "scripts/install-android-tools.sh" }}-{{ arch }}
          paths:
            - /Users/distiller/android-tools

      - run:
          name: create AVD
          command: make create-avd

      - run:
          name: start AVD
          command: emulator-headless -avd android-tablet
          background: true

      - run:
          name: wait for emulator
          command: adb wait-for-device shell 'while [[ -z $(getprop dev.bootcomplete) ]]; do sleep 1; done;'

      - run: adb shell screencap -p > screenshots/before.png

      # (insert testing here)

      - store_artifacts:
          path: screenshots

And here’s `install-android-tools.sh`:


if [ -d $ANDROID_SDK_ROOT ]
then
    echo "Directory $ANDROID_SDK_ROOT already exists so we're skipping the install. If you'd like to install fresh tools, edit this script to invalidate the CI cache."
    exit 0
fi

mkdir -p $ANDROID_SDK_ROOT
cd $ANDROID_SDK_ROOT
curl https://dl.google.com/android/repository/sdk-tools-darwin-4333796.zip -o sdk-tools.zip

unzip sdk-tools.zip

mkdir -p "$ANDROID_SDK_ROOT/licenses"

echo "24333f8a63b6825ea9c5514f83c2829b004d1fee" > "$ANDROID_SDK_ROOT/licenses/android-sdk-license"
echo "84831b9409646a918e30573bab4c9c91346d8abd" > "$ANDROID_SDK_ROOT/licenses/android-sdk-preview-license"
echo "d975f751698a77b662f1254ddbeed3901e976f5a" > "$ANDROID_SDK_ROOT/licenses/intel-android-extra-license"

SDKMANAGER=$ANDROID_SDK_ROOT/tools/bin/sdkmanager

$SDKMANAGER "platform-tools"
$SDKMANAGER "platforms;android-29"
$SDKMANAGER "build-tools;29.0.2"
$SDKMANAGER "ndk-bundle"
$SDKMANAGER "system-images;android-29;google_apis;x86_64"
$SDKMANAGER "emulator"

Conclusion

It’s unorthodox, but this approach has worked reasonably well so far for our small React Native project. One set of [Appium][molly spin] tests can run against both iOS and Android, and they run the same way in CircleCI that they do locally.

I’d be interested to hear about your experiences with Android UI tests in CircleCI, whether via a third-party service, a CI host that supports the Emulator, or another approach altogether.

[molly spin]: https:/ui-testing-web-views/
[lock file]: https://myers.io/2019/01/13/what-is-the-purpose-of-a-lock-file-for-package-managers/
[caching]: https://circleci.com/docs/2.0/caching/
[circle ios]: https://circleci.com/docs/2.0/testing-ios/
[circle android]: https://circleci.com/docs/2.0/language-android/#overview
[acceleration_so]: https://stackoverflow.com/questions/26455756/how-can-i-run-android-emulator-for-intel-x86-atom-without-hardware-acceleration
[MacOS environment]: https://circleci.com/docs/2.0/testing-ios/#macos-build-containers
[accel-check]: https://developer.android.com/studio/run/emulator-acceleration#accel-check
[emulator architecture]: https://medium.com/androiddevelopers/android-emulator-project-marble-improvements-1175a934941e
[android emulator]: https://developer.android.com/studio/run/emulator
[circle_2]: https://circleci.com/blog/say-hello-to-circleci-2-0/
[executor types]: https://circleci.com/docs/2.0/executor-types/#section=configuration
[executor intro]: https://circleci.com/docs/2.0/executor-intro/#section=configuration
[prebuilt images]: https://circleci.com/docs/2.0/circleci-images/#section=configuration
[vm restrictions]: https://developer.android.com/studio/run/emulator-acceleration#vm-accel-restrictions
[hardware acceleration]: https://developer.android.com/studio/run/emulator-acceleration
[hypervisor]: https://developer.android.com/studio/run/emulator-acceleration#hypervisors
[machine executor]: https://circleci.com/docs/2.0/executor-types/#using-machine
[firebase test lab]: https://firebase.google.com/products/test-lab/
[aws device farm]: https://aws.amazon.com/device-farm/
[wacky idea]: https:/wacky-ideas/

Conversation
  • Jason Hagglund says:

    This looks amazing! At my workplace we are not entirely pleased with how our tests are performing using Firebase Test Lab and somebody brought up executing directly on CircleCI. I ended up finding this and it was a major lightbulb moment. I look forward to trying this out.

    One question: is `make create-avd` a script on your end that creates a particular emulator device?

    • John Ruble John Ruble says:

      Thanks for the comment, Jason! I look forward to hearing how it goes if you try this out.

      `make create-avd` is a Makefile rule that, yes, creates an emulator device:

      “`
      create-avd:
      avdmanager create avd –device “10.1in WXGA (Tablet)” –name android-tablet -k “system-images;android-29;google_apis;x86_64” –force

      “`

  • Dragos says:

    I got really excited when I saw your post and followed the steps but unfortunately I cannot get it to work :(.

    I even tried installing different versions of the sdktools, via brew or manual download & install but the result is the same. For example

    “`
    static:~ distiller$ sdkmanager “platform-tools”
    Warning: Failed to download any source lists!
    Warning: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    Warning: Failed to find package platform-tools
    static:~ distiller$ ] 10% Computing updates…
    “`

    A command like `sdkmanager –list` yields
    “`
    Warning: Failed to download any source lists!
    Warning: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    Installed packages:=====================] 100% Computing updates…
    Path | Version | Description | Location
    ——- | ——- | ——- | ——-
    tools | 26.1.1 | Android SDK Tools 26.1.1 | tools/
    “`

    Any ideas on this?

    Thank you,
    Dragos

  • Dragos says:

    Ok so I managed to start the emulator if java 8 is installed but even with caching, its really slow and feels unstable. I think I will integrate a service like genymotion.

    Thank you for the above article, it has helped me a lot!

  • Comments are closed.