The Android for Work (AfW) Test Harness is a test suite for validating the AfW compatibility of Android devices. It includes support apps, test cases, configuration files, and a test runner (afw-test-tradefed) built on cts-tradefed. You should setup and run the AfW Test Harness after completing Provisioning for Device Administration.

Note: Building and running the AfW Test Harness is similar to building and running the Android Compatibility Test Suite (CTS).

Setting up a development environment

The development environment for the AfW Test Harness is similar to Android OS. Follow the steps in Requirements to set up a development machine.

Downloading source code

Download the AfW Test Harness source code using the steps in Downloading the Source. The AfW Test Harness source code is in the ./test/AfwTestHarness project. The branch name determines the version of AfW Test Harness to download (each Android platform has a separate version of AfW Test Harness). For Android 7.0, the branch name is afw-test-harness-nougat-dev. To initialize the repo and download source code for this branch, use:

mkdir WORKING_DIRECTORY
cd WORKING_DIRECTORY
git config --global user.name "Your Name"
git config --global user.email "you@example.com"
repo init -u https://android.googlesource.com/platform/manifest -b afw-test-harness-nougat-dev
repo sync -j24

To check out the source code for a different version, specify the branch with the corresponding tag. Available branches include:

Other dependency projects required to build the harness are also downloaded with the source code.

Viewing in Android Studio

To view and edit AfW source code in Android Studio:

1. Run the following commands

   make idegen
   development/tools/idegen/idegen.sh
   

2. In Android Studio, open android.ipr.

The AfW Test Harness source code is in test/AfwTestHarness.

Configuring the AfW Test Harness

You can customize the harness by configuring test/AfwTestHarness/afw-test.props. To run the harness successfully, complete the following steps:

1. Configure the Wi-Fi network in afw-test.props with the following properties:

   wifi_ssid
   wifi_password (optional)
   wifi_security_type (optional, available options are: NONE, WEP or WPA)
   

2. Obtain at least one account from a domain that is bound to Test DPC as its device policy controller. Specify the details in afw-test.props with the following properties:

   work_account_username
   work_account_password
   

   The AfW Test Harness uses Test DPC to test provisioning flows, so accounts must bind to Test DPC to run the test harness.

Building the AfW Test Harness

Initialize the build configuration using:

source build/envsetup.sh
lunch

Select a device type and press Enter.

Build the harness using:

make afw-test-harness -j32

This creates a directory (out/host/linux-x86/afw-th/android-cts) with all necessary binaries, configuration files, and tools to run the test harness. This directory is also zipped into a file (out/host/linux-x86/afw-th/android-afw-test-harness.zip) for distribution.

Running the AfW Test Harness

Use the following steps to run the AfW Test Harness:

1. In your build environment, launch the test runner using:

   afw-test-tradefed
   

   This starts the cts-tf console, loads test plans, test cases, and afw-test.props from out/host/linux-x86/afw-th/android-cts.
2. From the unzipped folder of android-afw-test-harness.zip, launch the test runner using:

   cts-tf> ./android‐cts/tools/afw-test‐tradefed
   

   This loads test plans, test cases, and afw-test.props from android-cts directory. Ensure ./android‐cts/repository/testcases/afw-test.props has the work account and Wi-Fi configuration.
3. Run a test plan. Each test plan is an XML file that contains a set of test packages from the AfwTestHarness/tests test package directory. Common plans include:
   • afw-userdebug-build. Contains all test packages that require a userdebug build.
   • afw-user-build. Runs on a user build but requires the test device to be set up properly, including completing the initial setup and enabling USB debugging.
   
   To run the test plan afw-userdebug-build, use:

   cts-tf> run cts --plan afw-userdebug-build
   

   To see all test plans, use the command list plans. To view plan definitions, refer to out/host/linux-x86/afw-th/android-cts/repository/plans.
   
4. Run a test package. To run a single test package, use

   cts-tf> run cts --package com.android.afwtest.NfcProvisioning
   

   To view all packages, use the command list packages. For more options, use the command run cts --help.

Debugging the AfW Test Harness

Run all commands in the afw-test-tradefed console (cts-tf), which you can launch by running afw-test-tradefed.

• Display more information with the -l INFO or -l DEBUG flags. Example:

  cts-tf> run cts ‐‐plan afw-userdebug-build -l DEBUG
  

• Run the test harness on a specific device with the -s flag. Example:

  cts-tf> run cts ‐‐plan afw-userdebug-build -l DEBUG -s device_sn
  

• Run test harness on all connected devices with the --all-devices flag. Example:

  cts-tf> run cts ‐‐plan afw-userdebug-build -l DEBUG --all-devices
  

• View current running executions using list invocations or l i.
• View summary of past test executions using list results or l r.
• View other list commands using help list.
• Monitor real-time logcat with filter using afwtest, then open another terminal and start logcat using: adb logcat | grep afwtest. After a test completes:
  • View logs in out/host/linux-x86/afw-th/android-cts/repository/logs/start-time. The full device logcat and host log (afw-test-tradefed logs) are saved in separate zip files.
  • Find relevant information by searching the device logcat for afwtest. Example: zless out/host/linux-x86/afw-th/android-cts/repository/logs/start-time/device_logcat_random-number.zip | grep afwtest
  • To view the full afw-test-tradefed log, use: zless out/host/linux-x86/afw-th/android-cts/repository/logs/start-time/host_log_random-number.zip
• A test package automates an AfW provisioning flow by going through UI pages and recording a navigation log in the device logcat file for each page. Example: afwtest.AutomationDriver: Navigating:com.android.afwtest.uiautomator.pages.gms.AddAccountPage
  UI pages for test package com.android.afwtest.NfcProvisioning include:
  • com.android.afwtest.uiautomator.pages.managedprovisioning.NfcProvisioningPage
  • com.android.afwtest.uiautomator.pages.PageSkipper
  • com.android.afwtest.uiautomator.pages.LandingPage
• If a test failed during the provisioning process, logcat contains an error similar to:

  TestRunner: java.lang.RuntimeException: Failed to load page: com.android.afwtest.uiautomator.pages.packageinstaller.DeviceAccessPage
  

  This is typically caused by errors in a previous UI page or the page that failed to load, so try to find other error messages in logcat before this error, then try to reproduce it manually following the provisioning flow.
• If a test package fails:
  • A screenshot is saved to out/host/linux-x86/afw-th/android-cts/repository/logs/start-time using the following syntax: screenshot-test_test_class_full_name_test_case_name-random_number.png. This information is also logged in the host log.
  • A bug report is saved to out/host/linux-x86/afw-th/android-cts/repository/logs/start-time as: bug-test_class_full_name_test_case_name-random_number.zip.
• After all test packages execute, a screenshot is taken and saved to out/host/linux-x86/afw-th/android-cts/repository/logs/start-time as: screenshot-random_number.png. This information is also logged in the host log.

FAQ

For help with questions not answered below, contact afw-testharness-support@google.com.

Can I run test plan afw-userdebug-build on a device flashed with user build?

No. Test packages in the afw-userdebug-build plan factory reset the testing device before running the actual test flow and require adb debugging to be auto-enabled. With a user build, adb debugging can be enabled only by manually changing the setting in Developer options.

Can I run test plan afw-user-build on a device flashed with userdebug build?

Yes, but we recommend that you run this test plan on a user build.

Sometimes my test fails because UI loading takes too much time. How can I fix this?

Configure the timeout_size setting in ./android-cts/repository/testcases/afw-test.props. Valid settings are: S, M, L, XL, XXL.

The test package com.android.afwtest.NfcProvisioning (or SuwDoProvisioning) fails on my device because the installed initial setup (i.e. Setup Wizard) shows customized UI (such as Term & Conditions) after provisioning is complete. How can I skip this customized UI?

There should be minimal UI after the provisioning process. The test harness will automatically skip such UI if the UI has a button that has meaningful text or content description that contains any of the following words: Skip, Finish, Done, Accept, Agree, Next, Continue, or Proceed. Alternatively, you can define a button in afw-test.props to configure the test harness to skip your UI. Example:

oem_widgets=your_btn
your_btn.text=your_customized_text
your_btn.package=your_package
your_btn.action=click

To define multiple widgets, separate using commas.

The test package com.android.afwtest.NfcProvisioning (or SuwDoProvisioning) failed and the last UI screen is "Verify your account." Why does this happen and how can I recover the testing device?

This failure occurs because the previous test package failed to clear Factory Reset Protection at the end of the test. You must manually enter the account to unlock the device.

My device needs more time to factory reset. Can I extend the factory reset timeout?

Yes. Configure the factory_reset_timeout_min setting in afw-test.props. Valid settings are in minutes; you can set to any number of minutes that works with your device.