Automating an Android Instrumentation Test Run

I recently spent some time working through ways to automate running an Android test suite on my MacBook Pro. I found helpful bits and pieces all over the place—from Stack Overflow answers to blog posts talking about how to get Android into various CI servers (Travis, Jenkins, etc.)—but the information was scattered. In this post, I’m going to document what I learned while writing an Android test script.

Android Instrumentation Tests

My unit test suite uses Robolectric, which means the tests don’t need an emulator to run. There is a separate suite of tests that employs Espresso to test the app using the user interface, and these need an emulator (or real device) to run on. These types of test are known as Android Instrumentation Tests.

Conveniently, there is an out-of-the-box gradle task called connectedAndroidTest that can be used to run Instrumentation Tests. This task builds the app and tests and runs them on all connected devices (real or emulated).


./gradlew connectedAndroidTest -i

But the gradle task is limited in that it only runs the tests. You need to have an emulator already running. And while it will print out test failures, it doesn’t provide any access to the logcat output during the test run.

Fortunately, the Android ecosystem has a variety of command line tools making it possible to script pretty much anything that’s needed.

Start the Emulator

The following command can be used to start a clean emulator (wiping its contents completely):


$ANDROID_SDK/tools/emulator -avd Nexus_5X_API_23 -wipe-data

This command runs in the foreground until the emulator is closed. Obviously, that’s not going to work when we want to run a gradle task (connectedAndroidTest) while the emulator is running.

One way to handle this is to run the emulator command in the background, then kill it when the tests are done. To do this, you’ll need to hold on to the PID of the emulator process:


$ANDROID_SDK/tools/emulator -avd Nexus_5X_API_23 -wipe-data &
EMULATOR_PID=$!

Wait for the Emulator

It can take a while for an Android emulator to launch, boot, and be ready to run a test suite. To prevent the issues than arise when you try to run tests on an emulator that’s still booting, you’ll have to wait until the emulator’s fully booted before proceeding.

The adb tool comes with a built-in wait-for-device argument that waits until the emulator has launched before proceeding, but that doesn’t mean the OS is fully booted. While Android is booting, it sets its init.svc.bootanim property to “running.” When it’s done booting, the property is set to “stopped” (source: Detect when Android emulator is fully booted). Using this property, it’s possible to script waiting for the emulator to be launched and booted:


WAIT_CMD="$ANDROID_SDK/platform-tools/adb wait-for-device shell getprop init.svc.bootanim"

until $WAIT_CMD | grep -m 1 stopped; do 
  echo "Waiting..."
  sleep 1
done

Unlock the Lock Screen

If the emulator you’re using starts up with a lock screen, you’ll need to unlock it before running the tests. As found on Handy adb commands for Android, there’s a command for that:


$ANDROID_SDK/platform-tools/adb shell input keyevent 82

Capture logcat

When trying to find out why a test is failing, it can be very useful to have access to any log output, which on Android, comes from logcat. The following snippet will clear the current logcat butffer and start a background process that sends the logcat output to a file.


$ANDROID_SDK/platform-tools/adb logcat -c
$ANDROID_SDK/platform-tools/adb logcat > build/logcat.log &
LOGCAT_PID=$!

Run the Tests

As I mentioned above, there is a built-in gradle task called connectedAndroidTest that will compile the tests, send them to the emulator, and run them.


./gradlew connectedAndroidTest -i

Cleanup

After the tests finish, we need to stop capturing the logcat output and kill the emulator. Assuming the process IDs were captured, kill will do the job:


kill $EMULATOR_PID
kill $LOGCAT_PID

Put It All Together

Combining all of these pieces results in the following script that will start a wiped emulator, wait for it to boot, unlock the lock screen, run the Android Instrumentation tests while capturing logcat output, and stop the emulator.


#!/bin/bash

#Start the emulator
$ANDROID_SDK/tools/emulator -avd Nexus_5X_API_23 -wipe-data &
EMULATOR_PID=$!

# Wait for Android to finish booting
WAIT_CMD="$ANDROID_SDK/platform-tools/adb wait-for-device shell getprop init.svc.bootanim"
until $WAIT_CMD | grep -m 1 stopped; do
  echo "Waiting..."
  sleep 1
done

# Unlock the Lock Screen
$ANDROID_SDK/platform-tools/adb shell input keyevent 82

# Clear and capture logcat
$ANDROID_SDK/platform-tools/adb logcat -c
$ANDROID_SDK/platform-tools/adb logcat > build/logcat.log &
LOGCAT_PID=$!

# Run the tests
./gradlew connectedAndroidTest -i

# Stop the background processes
kill $LOGCAT_PID
kill $EMULATOR_PID

Saving that to a script named “androidtests,” it can be run like so:


ANDROID_SDK=~/Library/Android/sdk androidtests

Summary

Most of the Continuous Integration tools out there try to handle a lot of this automation for you. But if you want to be able to run the tests in a CI-like fashion on your development machine (or have more control over how the CI machine runs your tests), it can be useful to have some scripts that automate the process for you.

So grab a copy of the script above, tweak it to suit your needs, fire it up, and let it run while you’re getting more coffee.

Conversation
  • Martin says:

    Hi,

    I tried your script and have 2 problems:
    1. “Wait for android to finish booting” ends earlier than my android emulator is fully loaded. So script related to unlock screen is executed to early and doesn’t unlock the screen.
    2. When I start emulator with option -wipe-data after unlock screen I have next “Make yourself at home” welcome screen and I don’t know how to disable it.

    Thanks

    • Patrick Bacon Patrick Bacon says:

      1. After writing this post I also ran into issues with the “wait for booting” portion being called to early and not actually waiting. I ended up needing to add a few second sleep before that so the emulator would actually be running (although not yet booted) so the script could wait properly.

      2. I haven’t run into that particular issue so I’m not sure how to handle it.

  • Sean says:

    I currently have the same logcat solution you have listed but it doesn’t work for me. Every time I start the tests, gradlew kills logcat and I end up with incomplete logs. I am running a LOT of log-dense tests and need more than the 256KB log cache size.

  • Sam says:

    Hey, Thanks a lot. This really works, you are awesome! :)

  • craj says:

    Can this script be used for running espresso Ui automation tests as well ? I am trying to run UI
    espresso tests on an emulator locally .

    • Patrick Bacon Patrick Bacon says:

      craj,

      I haven’t tried Espresso UI Automation tests so I can’t say for sure. Sorry.

  • Comments are closed.