1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
|
page.title=Using CTS Verifier
@jd:body
<!--
Copyright 2015 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="qv-wrapper">
<div id="qv">
<h2>In this document</h2>
<ol id="auto-toc">
</ol>
</div>
</div>
<p>The Android Compatibility Test Suite Verifier (CTS Verifier) is a supplement to
the Compatibility Test Suite (CTS). While CTS checks those APIs and functions
that can be automated, CTS Verifier provides tests for those APIs and functions
that cannot be tested on a stationary device without manual input, like audio
quality, touchscreen, accelerometer, camera, etc.</p>
<h2 id=test_preparation>Test preparation</h2>
<p>The device must have verified Android API compatibility by successfully passing
the Compatibility Test Suite.</p>
<h3 id=hardware_requirements>Hardware requirements</h3>
<ul>
<li> A Linux computer with USB 2.0 compatible port
<li> A second Android device with a known compatible Bluetooth, Wi-Fi direct, and
NFC Host Card Emulation (HCE) implementation
</ul>
<h3 id=setup>Setup</h3>
<ul>
<li>Install the <a href="http://developer.android.com/sdk/index.html">Android
SDK</a> on the Linux computer <li>Download the appropriate <a
href="downloads.html">CTS Verifier.apk</a> for the
version of Android under test.
<li>Install CTS Verifier.apk to the <em>Device Under Test</em> (DUT). <br>
<code>adb install -r -g CtsVerifier.apk</code>
<li>Ensure that the device has its system data and time set correctly.
</ul>
<h2 id=cts_test_procedure>CTS Verifier test procedure</h2>
<ol>
<li>After the CTS Verifier.apk has been installed, launch the CTS Verifier
application:
<img src="images/cts-verifier-icon.png" alt="CTS Verifier icon in launcher" id="figure1" />
<p class="img-caption">
<strong>Figure 1.</strong> CTS Verifier icon
</p>
<li>Once opened, the CTS Verifier displays a list of all test sets available for
manual verification:
<img src="images/cts-verifier-menu.png" alt="CTS Verifier menu of tests" id="figure2" />
<p class="img-caption">
<strong>Figure 2.</strong> CTS Verifier menu of tests
</p>
<li>Each test contains a set of common elements (in some tests, Pass/Fail is
determined automatically):
<ul>
<li><em>Info</em>—a set of instructions to run the test. This will appear as a popup the first
time each test is opened or whenever the <strong>Info</strong> button (？) is pressed.
<li><em>Pass</em>—If the DUT meets the test requirements per the instructions from Info, press
the <strong>Pass</strong> button (✓).
<li><em>Fail</em>—If the DUT does not meet the test requirements per the instructions from Info,
press the <strong>Fail</strong> button (！).
</ul>
<img src="images/video-verifier.png" alt="Streaming video quality verifier" id="figure3" />
<p class="img-caption">
<strong>Figure 3.</strong> Video quality verifier
</p>
</ol>
<h2 id=specific_test_requirements>Specific test requirements</h2>
<h3 id=usb_accessory>USB Accessory</h3>
<p>In order to run the USB Accessory test, you need a Linux computer to run the
USB desktop machine (host) program.</p>
<ol>
<li>Connect the DUT to a computer.
<li>Execute the cts-usb-accessory program on the computer found in the CTS Verifier
package.
<li>A popup message will appear on the DUT. Select <strong>OK</strong> and go into the USB Accessory Test in the CTS Verifier application.
<br>
<img src="images/screen-lock-test.png" alt="CTS Verifier screen lock test" id="figure4" />
<p class="img-caption">
<strong>Figure 4.</strong> Screen lock test
</p>
<li>Output similar to below will appear on the computer’s console.
</ol>
<pre>
out/host/linux-x86/cts-verifier/android-cts-verifier$ <strong>./cts-usb-accessory</strong>
CTS USB Accessory Tester
Found possible Android device (413c:2106) - attempting to switch to accessory
mode...
Failed to read protocol version
Found Android device in accessory mode (18d1:2d01)...
[RECV] Message from Android device #0
[SENT] Message from Android accessory #0
[RECV] Message from Android device #1
[SENT] Message from Android accessory #1
[RECV] Message from Android device #2
[SENT] Message from Android accessory #2
[RECV] Message from Android device #3
[SENT] Message from Android accessory #3
[RECV] Message from Android device #4
[SENT] Message from Android accessory #4
[RECV] Message from Android device #5
[SENT] Message from Android accessory #5
[RECV] Message from Android device #6
[SENT] Message from Android accessory #6
[RECV] Message from Android device #7
[SENT] Message from Android accessory #7
[RECV] Message from Android device #8
[SENT] Message from Android accessory #8
[RECV] Message from Android device #9
[SENT] Message from Android accessory #9
[RECV] Message from Android device #10
[SENT] Message from Android accessory #10
</pre>
<h3 id=camera_field_of_view_calibration>Camera field of view calibration</h3>
<p>This field of view calibration procedure is designed to be a quick way to
determine the device field of view with moderate accuracy.
<p><strong>Setup</strong> - Print the <a
href="{@docRoot}compatibility/calibration-pattern.pdf">calibration-pattern.pdf</a>
target file and mount it on a rigid backing. Print on 11” x 17” or A3. Orient
the camera device and the printed target as shown in the diagram below:</p>
<img src="images/camera-printed-target.png" alt="Camera printed target" id="figure5" />
<p class="img-caption">
<strong>Figure 5.</strong> Camera printed target
</p>
<p><strong>Setting the target width</strong> - Measure the distance between the
solid lines on the target pattern in centimeters to account for printing
inaccuracies (~38 cm).</p>
<ol>
<li>Start the calibration application.
<li>Press the setup button and select “Marker distance” to enter the distance.
<li>Measure and enter the distance to the target pattern (~100 cm).
<li>Press the back button to return to the calibration preview.
</ol>
<p><strong>Calibration process</strong> - Verify that the device and target are
placed as shown in the figure and the correct distances have been entered into
the setup dialog.The preview will display the image with a vertical line
overlaid onto it. This line should align with the center line of the target
pattern. The transparent grid can be used with the other vertical lines to
ensure that the optical axis is orthogonal to the target.</p>
<ol>
<li>Select an image resolution to test from the selector at the bottom left.
<li>Tap the screen to take a photo and enter the calibration mode (described
below).
<li>Hit the back button and repeat for all supported image resolutions.
</ol>
<p><strong>Calibration test (per resolution)</strong> In the calibration mode, the photo will be displayed with two vertical lines
overlaid onto the image.These lines should align with the vertical lines on the target pattern within a
few pixels. If they do not, then the reported field of view for that mode is
inaccurate (assuming the setup is correct).Adjust the slider at the bottom of the screen until the overlay aligns with the
target pattern as closely as possible. The displayed field of view will be a
close approximation to the correct value when the overlay and the target
pattern image are aligned. The reported field of view should be within +/-1
degree of the calibration value.</p>
<h2 id=exporting_test_reports>Exporting test reports</h2>
<ol>
<li>
After all tests are completed, tap the <strong>Save (disk)</strong> icon.
<br>
<img src="images/verifier-save-icon.png" alt="CTS Verifier Save icon" id="figure6" />
<p class="img-caption">
<strong>Figure 6.</strong> CTS Verifier Save icon. <em>Note:</em> In
Android 7.0 and later, the preview feature is removed:
<img src="images/verifier-preview-icon.png" width="24" height="24">
</p>
</li>
<li>
A path to the saved report will be displayed in pop-up (e.g.
<code>/sdcard/verifierReports/ctsVerifierReport-date-time.zip</code>).
Record the path.
<br>
<img src="images/path-saved-report.png" alt="CTS Verifier path to saved report " id="figure7" />
<p class="img-caption">
<strong>Figure 7.</strong> CTS Verifier path to saved report
</p>
</li>
<li>
Connect the device via USB to a computer with the SDK installed.
</li>
<li>
From the computer’s SDK installation, run
<code>adb pull CTSVerifierReportPath</code> to download the report from the device.
<ul>
<li>
To download all reports run: <code>adb pull /sdcard/verifierReports</code>
<br/>
For Android 6.0 and earlier, run: <code>adb pull /mnt/sdcard/ctsVerifierReports/</code>
</li>
<li>
The name of the reports are time-stamped based on the DUT’s system time.
</li>
<li>
To clear results after they have been selected, select
<strong>Menu > Clear</strong>. This will clear the Pass/Fail results.
</li>
</ul>
</li>
</ol>
|
