Follow the instructions
to get and build the Android source code but specify a particular CTS branch
name, for example-b android-5.0_r2
, when issuing the repo
init
command. This assures your CTS changes will be included in the
next CTS release and beyond.
Execute the following commands to build CTS and start the interactive CTS console:
Note: You may supply one of these other values
for TARGET_PRODUCT
to build for different architectures:
aosp_x86_64
or aosp_mips
cd /path/to/android/root
make cts -j32 TARGET_PRODUCT=aosp_arm64
cts-tradefed
At the cts-tf console, enter e.g.:
tf> run cts --plan CTS
CTS tests use JUnit and the Android testing APIs. Review the
Testing and Instrumentation
tutorial while perusing the existing tests under the
cts/tests
directory. You will see that CTS tests mostly follow the same
conventions used in other Android tests.
Since CTS runs across many production devices, the tests must follow these rules:
Most CTS test cases target a specific class in the Android API. These tests
have Java package names with a cts
suffix and class names with the
Test
suffix. Each test case consists of multiple tests, where each
test usually exercises a particular method of the class being tested.
These tests are arranged in a directory structure where tests are grouped into
different categories such as "widgets" and "views."
For example, the CTS test for the Java package
android.widget.TextView
is
android.widget.cts.TextViewTest
with its Java package name as
android.widget.cts
and its class name as
TextViewTest
.
android.widget.cts
.
TextView
, the class name should be
TextViewTest
.)
widget
), although it does not have to be.The directory structure and sample code depend on whether you are using CTS v1 or CTS v2.
For Android 6.0 and earlier, use CTS v1. For CTS v1, the sample code is at
cts/tests/tests/example
.
The directory structure in CTS v1 tests looks like this:
cts/ tests/ tests/ package-name/ Android.mk AndroidManifest.xml src/ android/ package-name/ SampleDeviceActivity.java cts/ SampleDeviceTest.java
For Android 7.0 and later, use CTS v2. For details, see the sample test in AOSP.
The CTS v2 directory structure looks like this:
cts/ tests/ module-name/ Android.mk AndroidManifest.xml src/ android/ package-name/ SampleDeviceActivity.java cts/ SampleDeviceTest.java
When adding new tests, there may not be an existing directory to place your test. In those cases, you'll need to create the directory and copy the appropriate sample files.
If you are using CTS v1, refer to the example under
cts/tests/tests/example
and create a new directory. Also,
make sure to add your new package's module name from its Android.mk
to CTS_COVERAGE_TEST_CASE_LIST
in
cts/CtsTestCaseList.mk
. This Makefile is used by
build/core/tasks/cts.mk
to combine all the tests together to create
the final CTS package.
Use the sample test
/cts/tests/sample/
to quick start your new test module with following steps:
mkdir cts/tests/module-name && cp -r cts/tests/sample/* cts/tests/module-name
cts/tests/module-name
and substitute all instances of
"[Ss]ample" following the recommended naming convention from above.
SampleDeviceActivity
to exercise the feature you're testing.
SampleDeviceTest
to ensure the activity succeeds or logs its
errors. Other Android directories such as assets
, jni
,
libs
and res
can also be added. To add JNI code,
create a directory in the root of the project next to src
with the native
code and an Android.mk
in it.
The makefile typically contains the following settings:
LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libCtsSample_jni # don't include this package in any target LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := list of source code files LOCAL_C_INCLUDES := $(JNI_H_INCLUDE) # Tag this module as a cts test artifact LOCAL_COMPATIBILITY_SUITE := cts LOCAL_SHARED_LIBRARIES := libnativehelper LOCAL_SDK_VERSION := current include $(BUILD_SHARED_LIBRARY)
Finally, the Android.mk
file in the root of the project will
need to be modified to build the native code and depend on it, as shown
below:
# All tests should include android.test.runner. LOCAL_JAVA_LIBRARIES := android.test.runner # Includes the jni code as a shared library LOCAL_JNI_SHARED_LIBRARIES := libCtsSample_jni # Include for InstrumentationCtsTestRunner LOCAL_STATIC_JAVA_LIBRARIES := ctstestrunner... LOCAL_SDK_VERSION := currentinclude $(BUILD_CTS_PACKAGE) #Tells make to look in subdirectories for more make files to include include $(call all-makefiles-under,$(LOCAL_PATH))
Besides adding new tests there are other ways to contribute to CTS: Fix or remove tests annotated with "BrokenTest" or "KnownFailure."
Follow the Submitting Patches workflow to contribute changes to CTS. A reviewer will be assigned to your change, and your change should be reviewed shortly!
CTS releases follow this schedule.
Note: This schedule is tentative and may be updated from time to time as CTS for the given Android version matures.
Version | Branch | Frequency |
---|---|---|
9 | pie-cts-dev | Monthly |
8.1 | oreo-mr1-cts-dev | Monthly |
8.0 | oreo-cts-dev | Monthly |
7.1 | nougat-mr1-cts-dev | Monthly |
7.0 | nougat-cts-dev | Monthly |
No releases are planned for 6.0, 5.1, 5.0, 4.4, 4.3 and 4.2. |
CTS development branches have been set up so that changes submitted to each
branch will automatically merge as below:
nougat-cts-dev -> nougat-mr1-cts-dev -> oreo-cts-dev -> oreo-mr1-cts-dev -> pie-cts-dev -> <private-development-branch for Android Q>
If a changelist (CL) fails to merge correctly, the author of the CL will get an email with instructions on how to resolve the conflict. In most of the cases, the author of the CL can use the instructions to skip the auto-merge of the conflicting CL.
Additionally, if an older branch requires the change, then the CL needs to be cherry-picked from the newer branch.