aboutsummaryrefslogtreecommitdiff
path: root/en/compatibility
diff options
context:
space:
mode:
authorMark Hecomovich <mheco@google.com>2019-01-28 21:29:33 +0000
committerGerrit Code Review <noreply-gerritcodereview@google.com>2019-01-28 21:29:33 +0000
commit56cd845b72cebfc74b8c527936ed2e4aff4b51dd (patch)
tree94e1ae7edb82c95a5da83468b6ea104d651850f3 /en/compatibility
parentd8c39787c5a4061686695502e62c0477b939c633 (diff)
parent964c5956c98fb20ab44ead57e59b5c0ed9f0ab11 (diff)
downloadsource.android.com-56cd845b72cebfc74b8c527936ed2e4aff4b51dd.tar.gz
Merge "Docs: Changes to source.android.com"
Diffstat (limited to 'en/compatibility')
-rw-r--r--en/compatibility/_toc-tests.yaml24
-rw-r--r--en/compatibility/_translation.yaml30
-rw-r--r--en/compatibility/cts/camera-hal.html167
-rw-r--r--en/compatibility/cts/camera-its-box-assembly.html4
-rw-r--r--en/compatibility/cts/camera-its-box.html77
-rw-r--r--en/compatibility/cts/camera-wfov-box-assembly.md2
-rw-r--r--en/compatibility/cts/development.html9
-rw-r--r--en/compatibility/cts/downloads.html100
-rw-r--r--en/compatibility/cts/run.html315
-rw-r--r--en/compatibility/cts/setup.html57
-rw-r--r--en/compatibility/tests/development/blueprints.md7
-rw-r--r--en/compatibility/tests/development/index.md8
-rw-r--r--en/compatibility/tests/development/test-mapping.md35
-rw-r--r--en/compatibility/vts/codelab-video.html61
-rw-r--r--en/compatibility/vts/index.html4
15 files changed, 540 insertions, 360 deletions
diff --git a/en/compatibility/_toc-tests.yaml b/en/compatibility/_toc-tests.yaml
index 6d9d575f..8002d3c2 100644
--- a/en/compatibility/_toc-tests.yaml
+++ b/en/compatibility/_toc-tests.yaml
@@ -56,15 +56,19 @@ toc:
- title: Camera HAL Testing
path: /compatibility/cts/camera-hal
- title: Camera ITS-in-a-Box
- path: /compatibility/cts/camera-its-box
- - title: Wide Field of View ITS-in-a-Box Assembly
- path: /compatibility/cts/camera-wfov-box-assembly
- - title: Regular Field of View ITS-in-a-Box Assembly
- path: /compatibility/cts/camera-its-box-assembly
- - title: Sensor Fusion Box Quick Start
- path: /compatibility/cts/sensor-fusion-quick-start
- - title: Sensor Fusion Box Details
- path: /compatibility/cts/sensor-fusion-box-assembly
+ section:
+ - title: Overview
+ path: /compatibility/cts/camera-its-box
+ - title: Regular Field of View Box
+ path: /compatibility/cts/camera-its-box-assembly
+ - title: Wide Field of View Box
+ path: /compatibility/cts/camera-wfov-box-assembly
+ - title: Sensor Fusion Box
+ section:
+ - title: Sensor Fusion Box Quick Start
+ path: /compatibility/cts/sensor-fusion-quick-start
+ - title: Sensor Fusion Box Details
+ path: /compatibility/cts/sensor-fusion-box-assembly
- title: Secure Element
path: /compatibility/cts/secure-element
- title: Interpret Results
@@ -147,6 +151,8 @@ toc:
path: /devices/tech/debug/jank_jitter
- title: Using GDB
path: /devices/tech/debug/gdb
+ - title: Dumping User & Kernel Stacks
+ path: /devices/tech/debug/native_stack_dump
- title: Native Memory Use
path: /devices/tech/debug/native-memory
- title: Network Connectivity Tests
diff --git a/en/compatibility/_translation.yaml b/en/compatibility/_translation.yaml
deleted file mode 100644
index 320f9ed7..00000000
--- a/en/compatibility/_translation.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-ignore_paths:
-- /compatibility/1.6/...
-- /compatibility/2.1/...
-- /compatibility/2.2/...
-- /compatibility/2.3/...
-- /compatibility/4.0/...
-- /compatibility/4.1/...
-- /compatibility/4.2/...
-- /compatibility/4.3/...
-- /compatibility/4.4/...
-- /compatibility/5.0/...
-- /compatibility/5.1/...
-- /compatibility/6.0/...
-- /compatibility/7.0/...
-- /compatibility/7.1/...
-- /compatibility/8.0/...
-- /compatibility/8.1/...
-- /compatibility/images/...
-- /compatibility/source/...
-- /compatibility/android-cdd
-enable_continuous_translation: true
-title: Android Open Source Project Compatibility tab
-description: Translations for SAC compatibility tab
-language:
-- zh-CN
-cc:
-- sac-doc-leads+translation@google.com
-reviewer:
-- gdimino@google.com
-product: Android
diff --git a/en/compatibility/cts/camera-hal.html b/en/compatibility/cts/camera-hal.html
index f6db85a9..72da83db 100644
--- a/en/compatibility/cts/camera-hal.html
+++ b/en/compatibility/cts/camera-hal.html
@@ -24,40 +24,49 @@
<p>This document lists all tests available for evaluating the Android camera
-hardware abstraction layer (HAL). It is intended for OEMs and application
+hardware abstraction layer (HAL). It is intended for original equipment
+manufacturers (OEMs) and application
processor (AP) vendors so they may ensure proper implementation of the camera
HAL with minimum defects. Although this is a voluntary addition to the Android
Compatibility Test Suite (CTS), it greatly increases camera test coverage and
will certainly identify potential bugs.</p>
-<p>By passing these tests, original equipment manufacturers (OEM) validate
-whether they have properly integrated the latest Android camera hardware
-abstraction layer (HAL) 3.2 interfaces. When conforming with all items in the
+<p>By passing these tests, OEMs validate
+whether they have properly integrated the Android
+camera hardware
+abstraction layer (HAL) 3 interfaces. When conforming with all items in the
checklist, a device implementation may be considered <em>full</em> with respect
-to the new Android Camera HAL interfaces. This will in turn enable a device to
-properly support the new <code>android.hardware.camera2</code> package that
+to the Android Camera HAL interfaces. This will in turn enable a device to
+properly support the
+<a href="https://developer.android.com/reference/android/hardware/camera2/package-summary" class="external">
+ <code>android.hardware.camera2</code></a> package that
camera apps build upon.</p>
-<h2 id=camera_hal_3_2_specification>Camera HAL 3.2 specification</h2>
+<h2 id=camera_hal_3_2_specification>Camera HAL3 specification</h2>
-<p>The Android Camera HAL 3.2 specification is the authoritative source of
-information on what devices must satisfy; the document here provides a summary
+<p>The <a href="/devices/camera/camera3">Android Camera HAL3</a> specification
+is the authoritative source of
+information on what devices must satisfy; this page provides a summary
of all tests that can be used as a checklist. Camera HAL implementers (e.g. AP
-vendors) should go through the HAL 3.2 specification line-by-line and ensure
+vendors) should go through the Camera HAL3 specification line-by-line and ensure
their devices conform to it.</p>
-<p>The current HAL 3.2 specification is defined in these files within the L
-generic Android Platform Development Kit (PDK):</p>
+<p>The current HAL specification is defined in these files within the Android
+5.0 and later generic Android Platform Development Kit (PDK):</p>
<ul>
<li><em>Camera HAL 3.x interface and spec</em>: <code><a
-href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h">hardware/libhardware/include/hardware/camera3.h</a></code>,
+href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h"
+class="external">hardware/libhardware/include/hardware/camera3.h</a></code>,
<code><a
-href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera_common.h">hardware/libhardware/include/hardware/camera_common.h</a></code>
+href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera_common.h"
+class="external">hardware/libhardware/include/hardware/camera_common.h</a></code>
<li><em>Camera HAL 3.x metadata spec</em>: <code><a
-href="https://android.googlesource.com/platform/system/media/+/master/camera/docs/docs.html">system/media/camera/docs/docs.html</a></code>
+href="https://android.googlesource.com/platform/system/media/+/master/camera/docs/docs.html"
+class="external">system/media/camera/docs/docs.html</a></code>
<li><em>HAL pixel format interface and spec</em>: <code><a
-href="https://android.googlesource.com/platform/system/core/+/master/include/system/graphics.h">system/core/include/system/graphics.h</a></code>
+href="https://android.googlesource.com/platform/system/core/+/master/libsystem/include/system/graphics.h"
+class="external">system/core/libsystem/include/system/graphics.h</a></code>
</ul>
<h2 id=camera_test_types>Camera test types</h2>
@@ -103,14 +112,17 @@ HIDL interface level. For more information on using VTS, see
<h2 id=cts_tests>Compatibility Test Suite (CTS) tests</h2>
<p>Camera Android Compatibility Test Suite (CTS) tests focus upon device
-compatibility. They do not require a specific test environment (the field of
-view or FOV CTS Verifier test being the lone exception). </p>
+compatibility. For more information on the recommended test environment for camera CTS, see <a href="/compatibility/cts/setup#camera">Set up CTS</a>.</p>
<p>The starting path for Camera CTS tests is: <code>platform/cts</code>.</p>
<p>When running Camera CTS for devices that support external cameras (such as
USB webcams), you must have a device plugged in when running CTS or the tests
-will automatically fail.</p>
+will automatically fail. Examples of external cameras include: <a
+href="https://www.logitech.com/en-us/product/hd-pro-webcam-c920"
+class="external">Logitech HD Pro Webcam C920</a> and the <a
+href="https://www.microsoft.com/accessories/en-us/products/webcams/lifecam-hd-3000/t3h-00011"
+class="external">Microsoft LifeCam HD-3000</a>.</p>
<p>See the <a href="/compatibility/cts/index.html">CTS
introduction</a> and its subpages for general instructions on running CTS.</p>
@@ -144,66 +156,95 @@ the <code>android.hardware.camera2</code> API</h3>
<h2 id=its_tests>Image Test Suite (ITS) tests</h2>
-<p>The CameraITS tests focus upon image correctness. These Python scripts are
-manually run on a workstation with the Android device connected over USB. The
-workstation can run any operating system as long as it has the requisite Python
-2.7 environment.</p>
+<aside class="Note"><strong>Note:</strong>
+<a href="/devices/camera/camera3">Camera HAL3</a> is required for all devices
+running Android 9 or higher (except for Android Go devices).</aside>
-<p class="note">Since ITS is a CTS Verifier subtest, start CTS Verifier and the
-ITS subtest before running the python scripts so they have processes with which
-to communicate.</p>
+<p>The Camera Image Test Suite (ITS) tests focus on image correctness.
+ To perform the tests, run the Python scripts on a workstation with the
+ Android device connected over USB.</p>
-<p>The CameraITS infrastructure and tests are located under:
-<code>cts/apps/CameraITS</code></p>
+<p>The Camera ITS infrastructure and tests are located in the
+<a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS"
+class="external">
+ <code>cts/apps/CameraITS</code></a> directory. Each test resides in a
+ <code>tests/scene<var>#</var></code> subdirectory.</p>
-<p>See the latest <code>README</code> file in this top-level folder for
-instructions on how to set up and run the tests. For setup: <code>make
-cts</code></p>
+To set up the test environment, run:
<pre class="devsite-click-to-copy">
<code class="devsite-terminal">extract root/out/host/linux-x86/cts-verfier/android-cts-verifier.zip</code>
<code class="devsite-terminal">cd android-cts-verifier</code>
<code class="devsite-terminal">adb install -r CtsVerifier.apk</code>
<code class="devsite-terminal">cd CameraITS</code>
+<code class="devsite-terminal">source build/envsetup.sh</code>
</pre>
-<p>See <code>tutorial.py</code> in the <code>tests</code> subdirectory for a
-walkthrough of the script's use. Each test resides in a corresponding
-<code>tests/scene<#></code> subdirectory. See the <code>README</code> file in
-each subdirectory for specific test instructions.</p>
-
-<p>To follow the recommended way of setting up and running the Camera Image Test
-Suite, see <a href="/compatibility/cts/camera-its-box">Camera
-ITS-in-a-Box</a>.</p>
-
-<p>To run ITS manually, you will need a simple physical environment with a
+<aside class="note"><strong>Note:</strong> Because ITS is a CTS Verifier subtest,
+start CTS Verifier and the ITS subtest before running the Python scripts so they
+have processes with which to communicate.</aside>
+
+<p>For more information on how to set up and run the tests, see the
+<code>CameraITS</code> PDF file in <code>cts/apps/CameraITS</code>. See
+<a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS/tests/tutorial.py"
+ class="external">
+ <code>tutorial.py</code></a> in the <code>tests</code> subdirectory for a
+guide on how to use the scripts.</p>
+
+<p>The ITS static tests (scenes 0-5) can run with any operating
+system with the required Python 2.7 environment. However, the
+<code>sensor_fusion</code>
+test with the <a href="/compatibility/cts/sensor-fusion-quick-start">Sensor
+Fusion Box</a> must be run with the
+Linux operating system.
+</p>
+
+<p>The recommended setup for scenes 0-4
+is described in <a
+href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a>. The
+recommended setup for the sensor_fusion scene is described in <a
+href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box Quick
+ Start Guide</a>.</p>
+
+<p>To run ITS manually, prepare a simple physical environment with a
specific, reusable target
-such as a white wall, grey card, and desk lamp. The Android device is mounted
-on a tripod and its camera functions are exercised by the scripts. Most tests
-are pass or fail but some offer metrics, as well.</p>
+such as a white wall, a grey card, and a desk lamp. Mount the Android device on
+ a tripod and run the scripts to test the camera functions. Most tests
+are pass or fail but some offer metrics.</p>
+
+<p>These scripts test scenarios that are not tested in CTS and are an important
+ component of the HAL 3.2 test plan.</p>
-<p>These tests are works-in-progress and are not yet comprehensive enough for
-full automated pass/fail validation of the camera HAL. However, these scripts
-do test scenarios that are not tested in CTS and are an important component of
-the overall HAL 3.2 test plan.</p>
+<p>ITS tests either pass or fail. All mandated
+tests in each scene folder must pass. Tests that are not mandated can
+fail and still count as a pass in
+<code>CtsVerifier</code>.</p>
-<h3 id=its_tests_on_scene_0_plain>ITS tests on scene 0 (plain)</h3>
+<h3 id="scene0_to_scene4_tests">Scene 0 to scene 4 tests</h3>
-<p>This test requires no specific setup. Pass all of the tests in the
-<code>tests/scene0</code> folder, for all cameras (back + front + any
-others).</p>
+<p>These scenes represent a large part of ITS testing and are included as PDF
+files in the <code>scene</code> folder. To automate these tests, use the <a
+href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a> system.</p>
-<h3 id=its_tests_on_scene_1_grey_card>ITS tests on scene 1 (grey card)</h3>
+<ul>
+ <li>Scene 0: Requires no setup.</li>
+ <li>Scene 1: Requires a grey card.</li>
+ <li>Scene 2: Requires a face scene.</li>
+ <li>Scene 3: ISO12233 chart.</li>
+ <li>Scene 4: Custom scene with a circle inside a square.</li>
+</ul>
-<p>Pass all of the tests in the <code>tests/scene1</code> folder, for all
-cameras (back + front + any others). The <code>tests/scene1/README</code> file
-describes the scene setup.</p>
+<h3 id="scene5_tests">Scene 5 tests</h3>
+<p>Scene 5 tests require a diffuser to be placed on top of the camera.</p>
-<h3 id=its_tests_on_scene_2_camera_lab>ITS tests on scene 2 (camera lab)</h3>
+<h3 id="sensor_fusion_tests">Sensor fusion tests</h3>
-<p>Pass all of the tests in the <code>tests/scene2</code> folder, for all
-cameras (back + front + any others). The <code>tests/scene2/README</code> file
-describes the scene setup.</p>
+<p>Sensor fusion tests require specific camera motion to test the timestamp
+difference between the camera and the gyroscope for AR and VR applications. This
+test is skipped if no gyroscope is included or if the <code>REALTIME</code>
+parameter is not enabled. The <code>sensor_fusion</code> test can be automated
+with the <a
+href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box</a>.</p>
<h2 id=media_framework_tests>Media Framework tests</h2>
@@ -222,11 +263,11 @@ to install the resulting .apk. Example commands are included below.</p>
<pre class="devsite-click-to-copy">
<code class="devsite-terminal">make mediaframeworktest</code>
-<code class="devsite-terminal">adb install out/target/product/<em>&lt;name></em>/data/app/mediaframeworktest.apk</code>
+<code class="devsite-terminal">adb install out/target/product/<var>name</var>/data/app/mediaframeworktest.apk</code>
</pre>
-<p>Where the <em><code><name></code></em> variable represents the directory
-containing the vendor's product.</p>
+ <p>Where the <var>name</var> variable represents the
+ directory containing the vendor's product.</p>
<p>Find all of the tests in the following directory or its subdirectories:</p>
diff --git a/en/compatibility/cts/camera-its-box-assembly.html b/en/compatibility/cts/camera-its-box-assembly.html
index f6336f10..45d2f868 100644
--- a/en/compatibility/cts/camera-its-box-assembly.html
+++ b/en/compatibility/cts/camera-its-box-assembly.html
@@ -1,6 +1,6 @@
<html devsite>
<head>
- <title>Regular Field of View (RFoV) ITS-in-a-Box Assembly</title>
+ <title>Regular Field of View (RFoV) Box</title>
<meta name="project_path" value="/_project.yaml" />
<meta name="book_path" value="/_book.yaml" />
</head>
@@ -41,6 +41,8 @@ qualified vendors.</p>
95050<br>fred@acuspecinc.com</li>
<li><em>MYWAY, Inc</em><br>4F., No. 163, Fu
Ying Rd., New Taipei City, Taiwan<br>sales@myway.tw</li>
+<li><em>West-Mark, Inc</em><br>2704 Railroad Ave, Ceres, CA
+95307<br>dgoodman@west-mark.com</li>
</ul>
<h2 id=building-box>Building an RFoV ITS-in-a-box</h2>
diff --git a/en/compatibility/cts/camera-its-box.html b/en/compatibility/cts/camera-its-box.html
index 8bdc3bb3..5027656e 100644
--- a/en/compatibility/cts/camera-its-box.html
+++ b/en/compatibility/cts/camera-its-box.html
@@ -62,8 +62,11 @@ software.</li>
</ol>
<h2 id=configure-tablet>Configuring the tablet</h2>
-<p>This section provides step-by-step instructions for setting up a Pixel C
-tablet for use with the CameraITS software.</p>
+<p>This section provides step-by-step instructions for setting up a
+tablet for use with the CameraITS software. These instructions use a Pixel C as
+ an example tablet. For information on tablet
+requirements and recommendations, see <a href="#tablet-requirements">Tablet
+requirements</a>.</p>
<p class="note"><strong>Note:</strong> The CameraITS python scripts
automatically set the following options on the tablet for you:
@@ -74,10 +77,10 @@ automatically set the following options on the tablet for you:
<ol>
<li>Charge the tablet and power it on. If prompted to set up an account, skip it
(CameraITS does not require any account paired with the tablet).</li>
-<li>Update the tablet to Android 7.0 or later (Android 6.x and earlier versions
-do not support CameraITS).</li>
-<li>Enable developer mode by going to <em>Settings > About tablet</em> and
-tapping <strong>Build number</strong> five times.</li>
+<li>Update the tablet to Android 7.0 or higher. Android 6.x and lower versions
+do not support CameraITS.</li>
+<li>Enable <a href="https://developer.android.com/studio/debug/dev-options#enable"
+ class="external">developer mode</a>.</li>
<li>Return to <em>Settings</em> and select <strong>Developer options</strong>.
<table>
@@ -247,5 +250,67 @@ adb -s FA6BM0305016 pull /sdcard/verifierReports
</ol>
</li>
</ul>
+
+<h2 id=tablet-requirements>Tablet requirements</h2>
+
+<p>Tablets must have a display size of around 10 inches with a screen resolution
+ greater than 2000 x 1500 pixels. The <code>DISPLAY_LEVEL</code> value must be
+ set in
+<code>CameraITS/tools/wake_up_screen.py</code> according to the tablet model.
+The table below lists the values for recommended tablets.</p>
+
+<p>Recommended tablets for ITS testing are shown below:</p>
+
+<table>
+ <tr>
+ <th>Device</th>
+ <th>Display size<br>(inches)</th>
+ <th>Display size<br>(pixels)</th>
+ <th>Tablet dimensions<br>(inches)</th>
+ <th>DISPLAY_LEVEL</th>
+ <th>OS</th>
+ </tr>
+ <tr>
+ <td>Asus Zen Pad 3</td>
+ <td>9.7</td>
+ <td>2048 x 1536</td>
+ <td>9.47 x 6.44 x 0.28</td>
+ <td>192</td>
+ <td>Android 6.0</td>
+ </tr>
+ <tr>
+ <td>Huawei Mediapad m5</td>
+ <td>10.8</td>
+ <td>2560 x 1600</td>
+ <td>10.18 x 6.76 x 0.29</td>
+ <td>192</td>
+ <td>Android 8.0</td>
+ </tr>
+ <tr>
+ <td>Pixel C</td>
+ <td>10.2</td>
+ <td>2560 x 1800</td>
+ <td>9.53 x 7.05 x 0.28</td>
+ <td>96</td>
+ <td>Android 8.0</td>
+ </tr>
+ <tr>
+ <td>Samsung S3</td>
+ <td>9.7</td>
+ <td>2048 x 1536</td>
+ <td>10.76 x 6.65 x 0.24</td>
+ <td>192</td>
+ <td>Android 7.0</td>
+ </tr>
+ <tr>
+ <td>Sony Xperia Z4</td>
+ <td>10.1</td>
+ <td>2560 x 1600</td>
+ <td>10 x 6.57 x 0.24</td>
+ <td>192</td>
+ <td>Android 7.0</td>
+ </tr>
+</table>
+
</body>
</html>
diff --git a/en/compatibility/cts/camera-wfov-box-assembly.md b/en/compatibility/cts/camera-wfov-box-assembly.md
index cadc0b42..64b8373c 100644
--- a/en/compatibility/cts/camera-wfov-box-assembly.md
+++ b/en/compatibility/cts/camera-wfov-box-assembly.md
@@ -16,7 +16,7 @@ Book: /_book.yaml
limitations under the License.
-->
-# Wide Field of View (WFoV) ITS-in-a-Box Assembly
+# Wide Field of View (WFoV) Box
Android {{ androidPVersionNumber }} introduces ITS-in-a-box revision 2, an
automated test system for both wide field of view (WFoV) and regular field of
diff --git a/en/compatibility/cts/development.html b/en/compatibility/cts/development.html
index 3741c35e..99069374 100644
--- a/en/compatibility/cts/development.html
+++ b/en/compatibility/cts/development.html
@@ -277,13 +277,9 @@ updated from time to time as CTS for the given Android version matures.</p>
<td>nougat-cts-dev</td>
<td>Monthly</td>
</tr>
+
<tr>
- <td>6.0</td>
- <td>marshmallow-cts-dev</td>
- <td>Monthly</td>
-</tr>
-<tr>
- <td colspan="3">No releases are planned for 5.1, 5.0, 4.4, 4.3 and 4.2.</td>
+ <td colspan="3">No releases are planned for 6.0, 5.1, 5.0, 4.4, 4.3 and 4.2.</td>
</tr>
</table>
@@ -303,7 +299,6 @@ Open Source Project (AOSP).
<p>CTS development branches have been set up so that changes submitted to each
branch will automatically merge as below:<br>
-marshmallow-cts-dev ->
nougat-cts-dev -> nougat-mr1-cts-dev -> oreo-cts-dev -> oreo-mr1-cts-dev -> pie-cts-dev -> &lt;private-development-branch for Android Q&gt;</p>
<p>If a changelist (CL) fails to merge correctly, the author of the CL will get
diff --git a/en/compatibility/cts/downloads.html b/en/compatibility/cts/downloads.html
index 6f3811d0..bbc6c108 100644
--- a/en/compatibility/cts/downloads.html
+++ b/en/compatibility/cts/downloads.html
@@ -26,107 +26,107 @@
<p>Thank you for your interest in Android Compatibility! The links below give
you access to key documents and information about the program. As CTS is
updated, new versions are added to this page. CTS versions are denoted by
-R&lt;number&gt; in the link name.</p>
+R<var>number</var> in the link name.</p>
<h2 id="android-90">Android 9</h2>
<p>Android 9 is the release of the development milestone code-named P.
The source code for the following tests, including tests for instant apps, can be synced with the
-'android-cts-9.0_r4' tag in the open-source tree.</p>
+'android-cts-9.0_r5' tag in the open-source tree.</p>
<ul>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-9.0_r4-linux_x86-arm.zip">Android
-9.0 R4 Compatibility Test Suite (CTS) - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-9.0_r5-linux_x86-arm.zip">Android
+9.0 R5 Compatibility Test Suite (CTS) - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-9.0_r4-linux_x86-x86.zip">Android
-9.0 R4 Compatibility Test Suite (CTS) - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-9.0_r5-linux_x86-x86.zip">Android
+9.0 R5 Compatibility Test Suite (CTS) - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-9.0_r4-linux_x86-arm.zip">Android
-9.0 R4 CTS Verifier - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-9.0_r5-linux_x86-arm.zip">Android
+9.0 R5 CTS Verifier - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-9.0_r4-linux_x86-x86.zip">Android
-9.0 R4 CTS Verifier - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-9.0_r5-linux_x86-x86.zip">Android
+9.0 R5 CTS Verifier - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts_instant-9.0_r4-linux_x86-arm.zip">Android
-9.0 R4 CTS for Instant Apps - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts_instant-9.0_r5-linux_x86-arm.zip">Android
+9.0 R5 CTS for Instant Apps - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts_instant-9.0_r4-linux_x86-x86.zip">Android
-9.0 R4 CTS for Instant Apps - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts_instant-9.0_r5-linux_x86-x86.zip">Android
+9.0 R5 CTS for Instant Apps - x86</a></li>
</ul>
<h2 id="android-81">Android 8.1</h2>
<p>Android 8.1 is the release of the development milestone code-named Oreo-MR1.
The source code for the following tests can be synced with the
-'android-cts-8.1_r11' tag in the open-source tree.</p>
+'android-cts-8.1_r12' tag in the open-source tree.</p>
<ul>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-8.1_r11-linux_x86-arm.zip">Android
-8.1 R11 Compatibility Test Suite (CTS) - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-8.1_r12-linux_x86-arm.zip">Android
+8.1 R12 Compatibility Test Suite (CTS) - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-8.1_r11-linux_x86-x86.zip">Android
-8.1 R11 Compatibility Test Suite (CTS) - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-8.1_r12-linux_x86-x86.zip">Android
+8.1 R12 Compatibility Test Suite (CTS) - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.1_r11-linux_x86-arm.zip">Android
-8.1 R11 CTS Verifier - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.1_r12-linux_x86-arm.zip">Android
+8.1 R12 CTS Verifier - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.1_r11-linux_x86-x86.zip">Android
-8.1 R11 CTS Verifier - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.1_r12-linux_x86-x86.zip">Android
+8.1 R12 CTS Verifier - x86</a></li>
</ul>
<h2 id="android-80">Android 8.0</h2>
<p>Android 8.0 is the release of the development milestone code-named Oreo.
The source code for the following tests can be synced with the
-'android-cts-8.0_r15' tag in the open-source tree.</p>
+'android-cts-8.0_r16' tag in the open-source tree.</p>
<ul>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-8.0_r15-linux_x86-arm.zip">Android
-8.0 R15 Compatibility Test Suite (CTS) - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-8.0_r16-linux_x86-arm.zip">Android
+8.0 R16 Compatibility Test Suite (CTS) - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-8.0_r15-linux_x86-x86.zip">Android
-8.0 R15 Compatibility Test Suite (CTS) - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-8.0_r16-linux_x86-x86.zip">Android
+8.0 R16 Compatibility Test Suite (CTS) - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.0_r15-linux_x86-arm.zip">Android
-8.0 R15 CTS Verifier - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.0_r16-linux_x86-arm.zip">Android
+8.0 R16 CTS Verifier - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.0_r15-linux_x86-x86.zip">Android
-8.0 R15 CTS Verifier - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-8.0_r16-linux_x86-x86.zip">Android
+8.0 R16 CTS Verifier - x86</a></li>
</ul>
<h2 id="android-71">Android 7.1</h2>
<p>Android 7.1 is the release of the development milestone code-named Nougat-MR1.
The source code for the following tests can be synced with the
-'android-cts-7.1_r23' tag in the open-source tree.</p>
+'android-cts-7.1_r24' tag in the open-source tree.</p>
<ul>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-7.1_r23-linux_x86-arm.zip">Android
-7.1 R23 Compatibility Test Suite (CTS) - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-7.1_r24-linux_x86-arm.zip">Android
+7.1 R24 Compatibility Test Suite (CTS) - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-7.1_r23-linux_x86-x86.zip">Android
-7.1 R23 Compatibility Test Suite (CTS) - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-7.1_r24-linux_x86-x86.zip">Android
+7.1 R24 Compatibility Test Suite (CTS) - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.1_r23-linux_x86-arm.zip">Android
-7.1 R23 CTS Verifier - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.1_r24-linux_x86-arm.zip">Android
+7.1 R24 CTS Verifier - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.1_r23-linux_x86-x86.zip">Android
-7.1 R23 CTS Verifier - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.1_r24-linux_x86-x86.zip">Android
+7.1 R24 CTS Verifier - x86</a></li>
</ul>
<h2 id="android-70">Android 7.0</h2>
<p>Android 7.0 is the release of the development milestone code-named Nougat.
The source code for the following tests can be synced with the
-'android-cts-7.0_r27' tag in the open-source tree.</p>
+'android-cts-7.0_r28' tag in the open-source tree.</p>
<ul>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-7.0_r27-linux_x86-arm.zip">Android
-7.0 R27 Compatibility Test Suite (CTS) - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-7.0_r28-linux_x86-arm.zip">Android
+7.0 R28 Compatibility Test Suite (CTS) - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-7.0_r27-linux_x86-x86.zip">Android
-7.0 R27 Compatibility Test Suite (CTS) - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-7.0_r28-linux_x86-x86.zip">Android
+7.0 R28 Compatibility Test Suite (CTS) - x86</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.0_r27-linux_x86-arm.zip">Android
-7.0 R27 CTS Verifier - ARM</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.0_r28-linux_x86-arm.zip">Android
+7.0 R28 CTS Verifier - ARM</a></li>
<li><a
-href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.0_r27-linux_x86-x86.zip">Android
-7.0 R27 CTS Verifier - x86</a></li>
+href="https://dl.google.com/dl/android/cts/android-cts-verifier-7.0_r28-linux_x86-x86.zip">Android
+7.0 R28 CTS Verifier - x86</a></li>
</ul>
<h2 id="android-60">Android 6.0</h2>
diff --git a/en/compatibility/cts/run.html b/en/compatibility/cts/run.html
index 0bd66d90..0caf4c7e 100644
--- a/en/compatibility/cts/run.html
+++ b/en/compatibility/cts/run.html
@@ -45,156 +45,27 @@ short) continuous test framework.</p>
<li>Start the default test plan (contains all test packages) by appending: <code>run
cts --plan CTS</code> . This kicks off all CTS tests required for compatibility.
<ul>
+ <li>For CTS v2 (Android 7.0 and later), enter <code>list modules</code> to
+ see a list of test modules.
<li>For CTS v1 (Android 6.0 and earlier), enter <code>list plans</code> to view a list of
test plans in the repository or <code>list package</code>s to view a list of
test packages in the repository.
- <li>For CTS v2 (Android 7.0 and later), enter <code>list modules</code> to
- see a list of test modules.
</ul>
</li>
<li>Alternately, run the CTS plan of your choosing from the command line
using: <code>cts-tradefed run cts --plan
<plan_name>
</code>
-<p class="note"><strong>Note:</strong> When running Android 6.0 (Marshmallow)
-CTS only, we recommend you use the <code>--skip-preconditions</code> option to
-skip the experimental pre-conditions feature that may cause issues for when
-executing CTS tests.</p>
+<p class="note"><strong>Note:</strong> You may save run time in Android 7.0 (Nougat) and higher, by
+ using the <code>run cts-dev</code> command instead of <code>run cts</code>. This command skips
+ preconditions, device-information collection, and all system status checkers. It also runs the
+ tests on only a single ABI. For device validation, avoid this optimization and
+include all preconditions and checks.</p>
<li>View test progress and results reported on the console.
<li>If your device is Android 5.0 or later and declares support for an ARM and a
x86 ABI, you should run both the ARM and x86 CTS packages.
</ol>
-<h2 id=using-cts-v1>Using the CTS v1 console</h2>
-
-<p>For Android 6.0 or earlier, you'll use CTS v1.</p>
-<h3 id=selecting_cts_plans>Selecting plans</h3>
-<p>The following test plans are available:</p>
-<ul>
- <li><em>CTS</em>—all tests required for compatibility. </li>
- <li><em>Signature</em>—the signature verification of all public APIs </li>
- <li><em>Android</em>—tests for the Android APIs </li>
- <li><em>Java</em>—tests for the Java core library </li>
- <li><em>VM</em>—tests for ART or Dalvik </li>
- <li><em>Performance</em>—performance tests for your implementation </li>
-</ul>
-<p>These can be executed with the <code>run cts</code> command.</p>
-<h3 id=cts_reference>CTS v1 console command reference</h3>
-
-<p class="table-caption" id="console-commands">
- <strong>Table 1.</strong> This table summarizes the CTS v1 console commands for
-various uses.</p>
-<table>
- <tbody>
- <tr>
- <th>Host</th>
- <th>Description</th>
- </tr>
- <tr>
- <td><code>help</code></td>
- <td>Display a summary of the most commonly used commands</td>
- </tr>
- <tr>
- <td><code>help all</code></td>
- <td>Display the complete list of available commands</td>
- </tr>
- <tr>
- <td><code>exit</code></td>
- <td>Gracefully exit the CTS console. Console will close when all currently running tests are finished</td>
- </tr>
- <tr>
- <th>Run</th>
- <th>Description</th>
- </tr>
- <tr>
- <td><code>run cts</code></td>
- <td>Run the specified tests and displays progress information. One of
-<code>--plan</code>, <code>--package</code>, <code>--class</code> or
-<code>--continue-session</code> needs to be specified
- <p>The CTS console can accept other commands while tests are in progress </p>
- <p>If no devices are connected, the CTS desktop machine (or host) will wait for a device to be connected before starting tests </p>
- <p>If more than one device is connected, the CTS host will choose a device automatically</p></td>
- </tr>
- <tr>
- <td><code>--plan &lt;test_plan_name&gt;</code></td>
- <td>Run the specified test plan</td>
- </tr>
- <tr>
- <td><code>--package/-p &lt;test_package_name&gt;  [--package/-p &lt;test_package2&gt;...]</code></td>
- <td>Run the specified test packages</td>
- </tr>
- <tr>
- <td><code>--class/-c &lt;class_name&gt; [--method/-m &lt;test_method_name&gt;</code></td>
- <td>Run the specified test class and/or method</td>
- </tr>
- <tr>
- <td><code>--continue-session</code></td>
- <td>Run all not executed tests from previous CTS session; the sessions testResult.xml will be updated with the new results</td>
- </tr>
- <tr>
- <td><code>--shards &lt;number_of_shards&gt;</code></td>
- <td>Shard a CTS run into given number of independent chunks, to run on multiple devices in parallel</td>
- </tr>
- <tr>
- <td><code>--serial/-s &lt;deviceID&gt;</code></td>
- <td>Run CTS on the specific device</td>
- </tr>
- <tr>
- <td><code>-t &lt;class_name&gt;#&lt;test_method_name&gt;</code></td>
- <td>Run a specific test method</td>
- </tr>
- <tr>
- <td><code>--force-abi 32|64</code></td>
- <td>On 64-bit devices, run the test against only the 32-bit or 64-bit ABI</td>
- </tr>
- <tr>
- <th>List</th>
- <th>Description</th>
- </tr>
- <tr>
- <td><code>list packages</code></td>
- <td>List all available test packages in the repository</td>
- </tr>
- <tr>
- <td><code>list plans</code></td>
- <td>List all available test plans in the repository</td>
- </tr>
- <tr>
- <td><code>list invocations</code></td>
- <td>List 'run' commands currently being executed on devices</td>
- </tr>
- <tr>
- <td><code>list commands</code></td>
- <td>List all 'run' commands currently in the queue waiting to be assigned to devices</td>
- </tr>
- <tr>
- <td><code>list results</code></td>
- <td>List CTS results currently stored in repository</td>
- </tr>
- <tr>
- <td><code>list devices</code></td>
- <td>List currently connected devices and their state
- <p> </p>
- <p>'Available' devices are functioning, idle devices, available for running tests</p>
- <p> </p>
- <p>'Unavailable' devices are devices visible via adb, but are not responding to adb commands and won't be allocated for tests</p>
- <p> </p>
- <p>'Allocated' devices are devices currently running tests</td>
- </tr>
- <tr>
- <th>Add</th>
- <th>Description</th>
- </tr>
- <tr>
- <td><code>add derivedplan --plan &lt;plan_name&gt;<br>
- --result/-r<br>
- [pass | fail | timeout | notExecuted]<br>
- [--session/-s &lt;session_id&gt;]</code></td>
- <td>Create a plan derived from given result session; use this option to rerun reports and validate test issues</td>
- </tr>
- </tbody>
-</table>
-
<h2 id=using-cts-v2>Using the CTS v2 console</h2>
<p>For Android 7.0 or later, you'll use CTS v2.</p>
@@ -261,11 +132,32 @@ various uses.</p>
</tr>
<tr>
<td><code>run cts</code></td>
- <td><p>Run the default CTS plan (that is, the full CTS invocation).</p>
- <p>The CTS console can accept other commands while tests are in progress.</p>
+ <td><p>Run the default CTS plan (that is, the full CTS invocation). Use this
+ comprehensive option (including preconditions) for device validation.
+ See <a
+ href="https://android.googlesource.com/platform/test/suite_harness/+/master/tools/cts-tradefed/res/config/cts.xml">/test/suite_harness/tools/cts-tradefed/res/config/cts.xml</a>
+ for inclusions.</p>
+ <p>The CTS console can accept other commands while tests are in progress.</p>
+ <p>If no devices are connected, the CTS desktop machine (or host) will wait
+ for a device to be connected before starting tests. If more than one
+ device is connected, the CTS host will choose a device
+ automatically.</p>
+ </tr>
+ <td><code>run cts-dev</code></td>
+ <td><p>Run the default CTS plan (that is, the full CTS invocation) but
+ skip preconditions to save run time for iterative development of a new
+ test. This bypasses verification and setup of the device's
+ configuration, such as pushing media files or checking for Wi-Fi
+ connection, as is done when the <code>--skip-preconditions</code> option is used. This
+ command also skips device-information collection, and all system status checkers. It also
+ runs the tests on only a single ABI. For device validation, avoid this optimization and
+ include all preconditions and checks.
+ See <a href="https://android.googlesource.com/platform/test/suite_harness/+/master/tools/cts-tradefed/res/config/cts-dev.xml">/test/suite_harness/tools/cts-tradefed/res/config/cts-dev.xml</a> for exclusions.</p>
+ <p>The CTS console can accept other commands while tests are in progress.</p>
<p>If no devices are connected, the CTS desktop machine (or host) will wait
- for a device to be connected before starting tests.</p>
- <p>If more than one device is connected, the CTS host will choose a device automatically.</p></td>
+ for a device to be connected before starting tests. If more than one
+ device is connected, the CTS host will choose a device
+ automatically.</p></td>
</tr>
<tr>
<td><code>run retry</code></td>
@@ -347,7 +239,10 @@ various uses.</p>
</tr>
<tr>
<td><code>--skip-preconditions</code></td>
- <td>Bypasses verification and setup of the device's configuration, such as pushing media files or checking for Wi-Fi connection.</td>
+ <td>Skip preconditions to save run time for iterative development of a
+ new test. This bypasses verification and setup of the device's
+ configuration, such as pushing media files or checking for Wi-Fi
+ connection.</td>
</tr>
<tr>
<th>List</th>
@@ -416,5 +311,143 @@ various uses.</p>
</tbody>
</table>
+<h2 id=using-cts-v1>Using the CTS v1 console</h2>
+
+<p>For Android 6.0 or earlier, you'll use CTS v1.</p>
+<h3 id=selecting_cts_plans>Selecting plans</h3>
+<p>The following test plans are available:</p>
+<ul>
+ <li><em>CTS</em>—all tests required for compatibility. </li>
+ <li><em>Signature</em>—the signature verification of all public APIs </li>
+ <li><em>Android</em>—tests for the Android APIs </li>
+ <li><em>Java</em>—tests for the Java core library </li>
+ <li><em>VM</em>—tests for ART or Dalvik </li>
+ <li><em>Performance</em>—performance tests for your implementation </li>
+</ul>
+<p>These can be executed with the <code>run cts</code> command.</p>
+
+<h3 id=cts_reference>CTS v1 console command reference</h3>
+
+<p class="table-caption" id="console-commands">
+ <strong>Table 1.</strong> This table summarizes the CTS v1 console commands for
+various uses.</p>
+<table>
+ <tbody>
+ <tr>
+ <th>Host</th>
+ <th>Description</th>
+ </tr>
+ <tr>
+ <td><code>help</code></td>
+ <td>Display a summary of the most commonly used commands</td>
+ </tr>
+ <tr>
+ <td><code>help all</code></td>
+ <td>Display the complete list of available commands</td>
+ </tr>
+ <tr>
+ <td><code>exit</code></td>
+ <td>Gracefully exit the CTS console. Console will close when all currently running tests are finished</td>
+ </tr>
+ <tr>
+ <th>Run</th>
+ <th>Description</th>
+ </tr>
+ <tr>
+ <td><code>run cts</code></td>
+ <td>Run the specified tests and displays progress information. One of
+<code>--plan</code>, <code>--package</code>, <code>--class</code> or
+<code>--continue-session</code> needs to be specified
+ <p>The CTS console can accept other commands while tests are in progress </p>
+ <p>If no devices are connected, the CTS desktop machine (or host) will wait for a device to be connected before starting tests </p>
+ <p>If more than one device is connected, the CTS host will choose a device automatically</p></td>
+ </tr>
+ <tr>
+ <td><code>--plan &lt;test_plan_name&gt;</code></td>
+ <td>Run the specified test plan</td>
+ </tr>
+ <tr>
+ <td><code>--package/-p &lt;test_package_name&gt;  [--package/-p &lt;test_package2&gt;...]</code></td>
+ <td>Run the specified test packages</td>
+ </tr>
+ <tr>
+ <td><code>--class/-c &lt;class_name&gt; [--method/-m &lt;test_method_name&gt;</code></td>
+ <td>Run the specified test class and/or method</td>
+ </tr>
+ <tr>
+ <td><code>--continue-session</code></td>
+ <td>Run all not executed tests from previous CTS session; the sessions testResult.xml will be updated with the new results</td>
+ </tr>
+ <tr>
+ <td><code>--shards &lt;number_of_shards&gt;</code></td>
+ <td>Shard a CTS run into given number of independent chunks, to run on multiple devices in parallel</td>
+ </tr>
+ <tr>
+ <td><code>--serial/-s &lt;deviceID&gt;</code></td>
+ <td>Run CTS on the specific device</td>
+ </tr>
+ <tr>
+ <td><code>-t &lt;class_name&gt;#&lt;test_method_name&gt;</code></td>
+ <td>Run a specific test method</td>
+ </tr>
+ <tr>
+ <td><code>--force-abi 32|64</code></td>
+ <td>On 64-bit devices, run the test against only the 32-bit or 64-bit ABI</td>
+ </tr>
+ <tr>
+ <td><code>--skip-preconditions</code></td>
+ <td>Skip preconditions to save run time for iterative development of a
+ new test. This bypasses verification and setup of the device's
+ configuration, such as pushing media files or checking for Wi-Fi
+ connection.</td>
+ </tr>
+ <tr>
+ <th>List</th>
+ <th>Description</th>
+ </tr>
+ <tr>
+ <td><code>list packages</code></td>
+ <td>List all available test packages in the repository</td>
+ </tr>
+ <tr>
+ <td><code>list plans</code></td>
+ <td>List all available test plans in the repository</td>
+ </tr>
+ <tr>
+ <td><code>list invocations</code></td>
+ <td>List 'run' commands currently being executed on devices</td>
+ </tr>
+ <tr>
+ <td><code>list commands</code></td>
+ <td>List all 'run' commands currently in the queue waiting to be assigned to devices</td>
+ </tr>
+ <tr>
+ <td><code>list results</code></td>
+ <td>List CTS results currently stored in repository</td>
+ </tr>
+ <tr>
+ <td><code>list devices</code></td>
+ <td>List currently connected devices and their state
+ <p> </p>
+ <p>'Available' devices are functioning, idle devices, available for running tests</p>
+ <p> </p>
+ <p>'Unavailable' devices are devices visible via adb, but are not responding to adb commands and won't be allocated for tests</p>
+ <p> </p>
+ <p>'Allocated' devices are devices currently running tests</td>
+ </tr>
+ <tr>
+ <th>Add</th>
+ <th>Description</th>
+ </tr>
+ <tr>
+ <td><code>add derivedplan --plan &lt;plan_name&gt;<br>
+ --result/-r<br>
+ [pass | fail | timeout | notExecuted]<br>
+ [--session/-s &lt;session_id&gt;]</code></td>
+ <td>Create a plan derived from given result session; use this option to rerun reports and validate test issues</td>
+ </tr>
+ </tbody>
+</table>
+
</body>
</html>
diff --git a/en/compatibility/cts/setup.html b/en/compatibility/cts/setup.html
index 7961778a..f7600b4d 100644
--- a/en/compatibility/cts/setup.html
+++ b/en/compatibility/cts/setup.html
@@ -26,11 +26,11 @@
<h2 id=physical_environment>Physical environment</h2>
<h3 id=ble_beacons>Bluetooth LE beacons</h3>
-<p>If the DUT supports the Bluetooth LE feature, then at least three
-Bluetooth LE beacons should be placed within five meters of the DUT for Bluetooth
-LE scan testing. Those beacons can be any kind, do not need to be
-configured or emit anything specific, and can include iBeacon,
-Eddystone, or even devices simulating BLE beacons.</p>
+<p>If the device-under-test (DUT) supports the Bluetooth LE feature, then at
+least three Bluetooth LE beacons should be placed within five meters of the DUT
+for Bluetooth LE scan testing. Those beacons can be any kind, do not need to be
+configured or emit anything specific, and can include iBeacon, Eddystone, or
+even devices simulating BLE beacons.</p>
<h3 id=camera>Cameras</h3>
<p>When running camera CTS, you are recommended to use normal lighting
@@ -39,7 +39,7 @@ not too close to the lens (the distance depends on the device's minimum focus
distance).</p>
<p>If the DUT supports external cameras, such as USB
-webcams, then an external camera must be plugged in when running CTS.
+webcams, an external camera must be plugged in when running CTS.
Otherwise, the CTS tests will fail.</p>
<h3 id="gnss">GPS/GNSS</h3>
@@ -73,15 +73,16 @@ href="http://en.wikipedia.org/wiki/List_of_IPv6_tunnel_brokers">list of IPv6
tunnel brokers</a>.</p>
<h3 id="rtt">Wi-Fi RTT (Round Trip Time)</h3>
-<p>Android 9 adds an API for a <a ref="/devices/tech/connect/wifi-rtt">Wi-Fi RTT</a> capability, which
-allows devices to measure their distance to access points with an accuracy of 1 to 2 meters,
-thus increasing indoor location accuracy significantly. Here are two recommended devices
-supporting Wi-Fi RTT: <a href="https://store.google.com/product/google_wifi">Google Wifi</a>
-and <a href="https://fit-iot.com/web/products/fitlet2/">Compulab's Filet2 Access Point</a> (set to 40MHz bandwidth at 5GHz).</p>
+<p>Android 9 adds an API for a <a ref="/devices/tech/connect/wifi-rtt">Wi-Fi RTT</a> capability,
+which allows devices to measure their distance to access points with an accuracy of 1 to 2 meters,
+thus increasing indoor location accuracy significantly. Here are two recommended devices
+supporting Wi-Fi RTT: <a href="https://store.google.com/product/google_wifi">Google Wifi</a>
+and <a href="https://fit-iot.com/web/products/fitlet2/">Compulab's Filet2 Access Point</a>
+(set to 40MHz bandwidth at 5GHz).</p>
-<p>The access points should be powered up, but not required to be connected to any network.
-Access points do not need to be next to the testing device; however, they are recommended to be within a distance of 40 ft from the DUT.
-One access point is typically sufficient.</p>
+<p>The access points should be powered up, but not required to be connected to any network.
+Access points do not need to be next to the testing device; however, they are recommended to be
+within a distance of 40 ft from the DUT. One access point is typically sufficient.</p>
<h2 id=desktop_setup>Desktop machine setup</h2>
@@ -219,15 +220,25 @@ source</a>) which do not contain any code except for the manifest: </p>
preload the apps into the appropriate directories on the system image without
re-signing them.</p>
- <h3 id=sample-applet>Sample Applet</h3>
- <p>Android 9 introduced Open Mobile API Test cases, which are used to check if the
-underlying implementation of Secure Elements are implemented as per standard. These test
-cases would require the installation of a special applet which can be used by the
-CTS application to communicate with. One can use the <a href="https://android-review.googlesource.com/c/platform/cts/+/700517">sample applet</a> as provided.</p>
- <p>This is applicable to devices, which have eSE(embedded Secure Element), SIM or SDs. See <a href="/compatibility/cts/secure-element">CTS Test for Secure Element</a> for more detailed information on Open Mobile
-API Test cases and Access Control Test cases.</p>
-
-<h3 id=storage_requirements>Storage requirements</h3>
+ <h3 id="sample-applet">Sample Applet</h3>
+ <p>
+ Android 9 introduced Open Mobile APIs. For devices that plan to report
+ more than one secure element, CTS adds test cases to validate the
+ behavior of the Open Mobile APIs. These test cases require the one-time
+ installation of a sample applet into the embedded Secure Element (eSE)
+ of the Device Under Test (DUT) or into the SIM card used by the DUT. The
+ <a
+ href="https://android.googlesource.com/platform/cts/+/master/tests/tests/secure_element/sample_applet/Google-eSE-test.cap" class="external">eSE
+ sample applet</a> and the <a
+ href="https://android.googlesource.com/platform/cts/+/master/tests/tests/secure_element/sample_applet/uicc/google-cardlet.cap" class="external">SIM
+ sample applet</a> can be found in the AOSP repository.
+ </p>
+
+ <p>
+ See <a href="/compatibility/cts/secure-element">CTS Test for Secure Element</a> for more
+ detailed information on Open Mobile API Test cases and Access Control Test cases.</p>
+
+<h3 id="storage_requirements">Storage requirements</h3>
<p>The CTS media stress tests require video clips to be on external storage
(<code>/sdcard</code>). Most of the clips are from <a
href="https://peach.blender.org/">Big Buck Bunny</a> which is copyrighted by
diff --git a/en/compatibility/tests/development/blueprints.md b/en/compatibility/tests/development/blueprints.md
index 96054162..9b8a4cfb 100644
--- a/en/compatibility/tests/development/blueprints.md
+++ b/en/compatibility/tests/development/blueprints.md
@@ -19,7 +19,7 @@ Book: /_book.yaml
limitations under the License.
-->
-# Simple Test Configuration
+# Simple Build Configuration
Each new test module must have a configuration file to direct the build system
with module metadata, compile-time dependencies and packaging instructions.
@@ -31,8 +31,9 @@ branch.
Soong uses Blueprint or `.bp` files, which are JSON-like simple declarative
descriptions of modules to build. This format replaces the
-Make-based system used in previous releases. To accommodate custom testing or
-use the Android [Compatibility Test Suite](compatibility/cts) (CTS), follow the
+[Make-based system](https://developer.android.com/ndk/guides/android_mk){:
+.external} used in previous releases. To accommodate custom testing or use the
+Android [Compatibility Test Suite](compatibility/cts) (CTS), follow the
[Complex Test Configuration](/compatibility/tests/development/test-config)
instead.
diff --git a/en/compatibility/tests/development/index.md b/en/compatibility/tests/development/index.md
index 6fc66d6c..b08005b7 100644
--- a/en/compatibility/tests/development/index.md
+++ b/en/compatibility/tests/development/index.md
@@ -25,10 +25,10 @@ To integrate tests into a platform continuous testing service, they should meet
the guidelines on this page and follow this recommended flow.
1. Use the [Soong build system](https://android.googlesource.com/platform/build/soong/)
- for [Simple Test Configuration](blueprints).
-1. Employ [Test Mapping](test-mapping) to easily create pre- and post-submit
- test rules directly in the Android source tree.
-1. Run tests locally using [Atest](atest).
+ for [Simple Test Configuration](/compatibility/tests/development/blueprints).
+1. Employ [Test Mapping](/compatibility/tests/development/test-mapping) to
+ easily create pre- and post-submit test rules directly in the Android source tree.
+1. Run tests locally using [Atest](/compatibility/tests/development/atest).
## Test types
diff --git a/en/compatibility/tests/development/test-mapping.md b/en/compatibility/tests/development/test-mapping.md
index 5f843199..1c113219 100644
--- a/en/compatibility/tests/development/test-mapping.md
+++ b/en/compatibility/tests/development/test-mapping.md
@@ -88,7 +88,12 @@ Here is a sample TEST_MAPPING file:
{
"include-annotation": "android.platform.test.annotations.RequiresDevice"
}
- ]
+ ],
+ "file_patterns": ["(/|^)Window[^/]*\\.java", "(/|^)Activity[^/]*\\.java"]
+ },
+ {
+ "name" : "net_test_avrcp",
+ "host" : true
}
],
"postsubmit": [
@@ -120,11 +125,24 @@ use class `name` or test method `name`. To narrow down the tests to run, you can
use options such as `include-filter` here. See
([include-filter sample usage](https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/dex/TEST_MAPPING#7)).
-The `imports` attribute allows you to include tests in other TEST_MAPPING files
+The **host** setting of a test indicates whether the test is a deviceless test
+running on host or not. The default value is **false**, meaning the test
+requires a device to run. The supported test types are
+[HostGTest](/compatibility/tests/development/native) for native tests and
+[HostTest](/compatibility/tests/development/jar) for JUnit tests.
+
+The **file_patterns** attribute allows you to set a list of regex strings for
+matching the relative path of any source code file (relative to the directory
+containing the TEST_MAPPING file). In above example, test `CtsWindowManagerDeviceTestCases`
+will run in presubmit only when any java file starts with Window or Activity,
+which exists in the same directory of the TEST_MAPPING file or any of its sub
+directories, is changed. Note that `\` needs to escaped as it's in a JSON file.
+
+The **imports** attribute allows you to include tests in other TEST_MAPPING files
without copying the content. Note that the TEST_MAPPING files in the parent
directories of the imported path will also be included.
-The `options` attribute contains additional TradeFed command line options. In
+The **options** attribute contains additional TradeFed command line options. In
the above example, only tests with annotation `Presubmit` will run in presubmit;
all tests will run in postsubmit.
@@ -240,6 +258,17 @@ and its parent directories:
<code class="devsite-terminal">atest [--test-mapping] [src_path]:postsubmit</code>
</pre>
+### Running only tests that require no device
+
+You can use option **--host** for Atest to only run tests configured against the
+host that require no device. Without this option, Atest will run both tests, the
+ones requiring device and the ones running on host and require no device. The
+tests will be run in two seperate suites.
+
+<pre>
+<code class="devsite-terminal">atest [--test-mapping] --host</code>
+</pre>
+
### Identifying test groups
You can specify test groups in the Atest command. Note that presubmit tests are
diff --git a/en/compatibility/vts/codelab-video.html b/en/compatibility/vts/codelab-video.html
index 0144a45e..059d7667 100644
--- a/en/compatibility/vts/codelab-video.html
+++ b/en/compatibility/vts/codelab-video.html
@@ -24,8 +24,8 @@
<p>The Vendor Test Suite (VTS) codelabs and video tutorials provide details
- on running and developing VTS and CTS-on-GSI on Android {{
- androidPVersionNumber }} and Android 8.1.</p>
+ on running and developing VTS and CTS-on-GSI on Android {{ androidPVersionNumber }} and Android
+ 8.1.</p>
<h2> Videos and Codelabs for Android 9</h2>
@@ -46,7 +46,9 @@
<td><strong>Overview</strong>
</td>
- <td>Android Vendor Test Suite (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=m6ALQGf3Yt4&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=2&t=0s">
+ Android Vendor Test Suite</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#0">Introduction</a>
@@ -60,7 +62,9 @@
<td><strong>How to Run VTS and CTS-on-GSI</strong>
</td>
- <td>How to Run VTS and CTS-on-GSI (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=fbYx_g6bbCA&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=8">
+ How to Run VTS and CTS-on-GSI</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts-8/#1">Build
@@ -73,7 +77,9 @@
<td><strong>How to Flash GSI</strong>
</td>
- <td>How to Flash General System Image (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=rRgFCEZyA7Q&index=7&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H">
+ How to Flash General System Image</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#3">Prepare
@@ -88,7 +94,9 @@
<td><strong>Test Framework</strong>
</td>
- <td>Test Framework Changes (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=SUG82_-V88o&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=9">
+ Test Framework Changes</a></td>
<td><a href=
"https://source.android.com/compatibility/vts/shell-commands">Test
@@ -101,7 +109,9 @@
<td><strong>Test Time Optimization</strong>
</td>
- <td>Test Time Optimization (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=i5TKp7Wygn0&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=10">
+ Test Time Optimization</a></td>
<td>--</td>
</tr>
@@ -111,7 +121,9 @@
<td><strong>Multi-Device Testing</strong>
</td>
- <td>Multi-Device Tests (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=NIx5IDaNovg&index=6&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H">
+ Multi-Device Tests</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#12">Write a
@@ -160,7 +172,9 @@
<td><strong>Conditional Tests</strong>
</td>
- <td>Conditional Tests (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=CyxNqWmcCD4&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=2">
+ Conditional Tests</a></td>
<td>--</td>
</tr>
@@ -170,7 +184,9 @@
<td><strong>Service Name Aware Testing</strong>
</td>
- <td>Service name aware HAL (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=kbX98ur-hlE&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=5">
+ Service Name Aware HAL Testing</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#5">Choose a
@@ -183,7 +199,9 @@
<td><strong>HAL Extension Tests</strong>
</td>
- <td>HAL Extension Test (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=B55Rkcqpncc&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=3">
+ HAL Extension Test</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#6">Write a
@@ -196,7 +214,9 @@
<td><strong>Framework Backward Compatibility</strong>
</td>
- <td>Android Framework Backward Compatibility (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=JXqVlkJYbTE&index=4&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H">
+ Android Framework Backward Compatibility</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#7">Write a
@@ -241,7 +261,7 @@
<td><a href=
"https://www.youtube.com/watch?v=F41dHKYPoic&amp;list=PLWz5rJ2EKKc9JOMtoWWMJHFHgvXDoThva&amp;index=6">
- Performance Testing</a>
+ VTS Support for Performance Testing</a>
</td>
<td>
@@ -311,7 +331,9 @@
<td><strong>End-to-End Automation</strong>
</td>
- <td>Automatic Measurement Infrastructure (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=3Ay8SpoAcPw&index=11&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H">
+ Automatic Measurement Infrastructure</a></td>
<td>--</td>
</tr>
@@ -339,7 +361,9 @@
<td><strong>Partner Engineering</strong>
</td>
- <td>Treble Readiness (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=vB4WOe3CmO4&index=12&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H">
+ Treble Readiness</a></td>
<td>--</td>
</tr>
@@ -349,7 +373,9 @@
<td><strong>Conclusion</strong>
</td>
- <td>Call for Contributions (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=YzM2lGGJs70&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=13">
+ Call for Contributions</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#14">Report
@@ -411,7 +437,8 @@
<td><strong>How to Flash GSI</strong>
</td>
- <td>How to Flash General System Image (Coming Soon)</td>
+ <td><a href=
+ "https://www.youtube.com/watch?v=rRgFCEZyA7Q&list=PLWz5rJ2EKKc_b3YitO08nBPJamBBmCL_H&index=7">How to Flash General System Image</a></td>
<td><a href=
"https://codelabs.developers.google.com/codelabs/android-vts/#3">Prepare
diff --git a/en/compatibility/vts/index.html b/en/compatibility/vts/index.html
index 982dbed9..70235877 100644
--- a/en/compatibility/vts/index.html
+++ b/en/compatibility/vts/index.html
@@ -32,8 +32,8 @@
<ul>
<li><a href="/compatibility/vts/codelab-video">Codelab and Video Tutorials</a>.
- Describes the videos and codelabs on running and developing VTS and CTS-on-GSI on Android {{
- androidPVersionNumber }} and Android 8.1.</li>
+ Describes the videos and codelabs on running and developing VTS and CTS-on-GSI on Android
+ {{ androidPVersionNumber }} and Android 8.1.</li>
<li><a href="/compatibility/vts/systems">Systems Testing with VTS</a>.
Describes how to use VTS to test an Android native system implementation, set
up a testing environment, then test a patch using a VTS plan.</li>