UI Testing Windows Apps with WinAppDriver

UI testing is a great way to ensure that your app works as intended from the perspective of a user. If you’re working on a web project, you likely have access to tools such as Cypress or Playwright. But, what do you do if you’re working on a Windows desktop app?

Here are a few tips and tricks that I’ve picked up while working on a recent Windows desktop project.

About Windows Application Driver

Windows Application Driver (WinAppDriver) is a service to support Selenium-like user interface test automation on Windows applications. It’s essentially a locally-running REST API that helps automate interactions with other Windows applications. In a C# project, you’ll likely want to include the Appium NuGet package to help interact with the WinAppDriver server.

From a high-level, using WinAppDriver looks like this:

1. Build your application
2. Start the WinAppDriver process
3. Run your tests
4. Stop the WinAppDriver process

This might seem fairly straightforward, but there are several obstacles to overcome to keep these tests running reliably. For example, while your tests are running, you will need to avoid using the keyboard/mouse. Attempting to do other tasks while the tests are running on your machine will likely result in test failures.

Let’s take a look at some of the other common issues when testing desktop applications.

Files and User Config

If the application persists data to disk, you’ll likely want to remove those files after each test. This ensures that your tests remain isolated from one another. I would recommend doing this at the beginning of each test. That way, if you want to inspect the state of your app after the test, the files should still be there at the time of failure.

Another common item to clean up is the AppData folder, which likely contains some user config you’ll want to reset after each test. This is especially important when testing a settings screen that will likely change the user’s config.

Your function might look like this:

private void ResetAppFiles() 
{
    var dataPath = @"C:\AppName\data";
    var userConfigPath = Path.Combine(
        Environment.GetFolderPath(
            Environment.SpecialFolder.LocalApplicationData
        ), 
        "AppName"
    );

    if (Directory.Exists(dataPath))
        Directory.Delete(dataPath);

    if (Directory.Exists(userConfigPath))
        Directory.Delete(userConfigPath);
}

Opening the App

This one might seem obvious, but it can be tricky if your app also has a splash screen. Creating a WindowsDriver in your test will prompt the WinAppDriver server to attach to the newly-opened app session. This can sometimes lead to WinAppDriver incorrectly focusing on the splash screen instead of the main window.

I’ve found that adding an explicit wait and switching to the “first” window is the most effective way to fix this.

[SetUp]
public void BeforeEach()
{
    // other test setup stuff

    _driver = new WindowsDriver("http://127.0.0.1:4723", appOptions);

    Thread.Sleep(1000);
    _driver.SwitchTo().Window(_driver.WindowHandles[0]);
}

With this in place, if WinAppDriver accidentally attaches to the splash screen the WindowsDriver will switch focus back to the main window once the splash screen is closed.

Closing the App

At the end of each test, you’ll want to ensure that the current session of your app is closed before starting the next test. Calling the Quit() method on the WindowsDriver object should do the trick. However, there are a handful of cases where this isn’t sufficient.

The main issue is with the test opening sub-windows while testing the app. Calling Quit() will close the current window, but not all the windows in the current session. You might be able to figure out a reliable method for closing all of the window handles programmatically. But, I’ve found it easier to ensure there is only one remaining window at the end of each test by throwing an exception if that is not the case.

private void CloseApp()
{
    var windowCount = _driver.WindowHandles.Count;
    if (_driver != null && windowCount > 1) 
    {
        throw new IOException($"Ensure only the main window is open at the end of each test. There are currently {windowCount} windows open");
    }

    _driver.Quit();
}

This puts a bit more pressure on me as the developer to ensure only the main window is open at the end of each test, but in my experience, this is the most reliable way to ensure the app is closed properly at the end of each test.

Screenshots and Logging

Running the tests locally is a good place to start, but you’ll likely want these tests to run on a CI service like Azure Pipelines as well. The issue with this is that you don’t have much context for why a test failed when they are run in a CI job.

Not being able to find a particular element on the page is a common reason for test failure. It can be helpful to determine if the test made it to the correct page or is even focused on the correct window when the failure occurred.

Capturing a screenshot when the test fails is a great way to uncover some of that information:

using NUnit.Framework;
using NUnit.Framework.Interfaces;

[TearDown]
public void AfterEach()
{
    if (TestContext.CurrentContext.Result.Outcome != ResultState.Success)
    {
        var screenshotPath = $@"path\to\screenshots\{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
        _driver.GetScreenshot().SaveAsFile(screenshotPath);
        TestContext.AddTestAttachment(screenshotPath);
    }

    // The method from earlier
    CloseApp();
}

Introducing some form of logging is another way to add more context for why a test failed. More specifically, I recommend adding logs for each of the steps taken during the test. I typically do this by creating my own utility methods that are essentially wrappers around the WindowsDriver methods:

public void ClickElement(string accessibilityId, WindowsDriver driver)
{
    Console.WriteLine($"Clicking element with ID {accessibilityId}");
    driver.FindElementByAccessibilityId(accessibilityId).Click();
}

The result of using utility methods like this results in a clear log of which actions were taken up until the point of test failure:

Clicking element with ID menu
Clicking element with ID menuSettingsSubMenu
Entering "This is test input" into element with ID txtNotesField
Clicking element with ID btnSave

You could also use some form of ILogger here. However, for now, I’ve kept it simple since the results of standard output are automatically attached to the TestContext when using NUnit.

WinAppDriver for Reliable Integration Testing of Windows Apps

I hope this has helped you add some reliable integration tests to your Windows desktop application. Please let me know if you have any tips/tricks of your own!