diff options
Diffstat (limited to 'en/compatibility/vts/ui.html')
-rw-r--r-- | en/compatibility/vts/ui.html | 160 |
1 files changed, 160 insertions, 0 deletions
diff --git a/en/compatibility/vts/ui.html b/en/compatibility/vts/ui.html new file mode 100644 index 00000000..a3894f59 --- /dev/null +++ b/en/compatibility/vts/ui.html @@ -0,0 +1,160 @@ +<html devsite> + <head> + <title>VTS Dashboard UI</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p> +The VTS Dashboard provides a cohesive user interface that uses material design +to effectively display information about test results, profiling, and coverage. +Dashboard styling uses open-source Javascript libraries including Materialize +CSS and jQueryUI to process data delivered by Java servlets in Google App +Engine. +</p> + +<h2>Dashboard home</h2> +<p> +The Dashboard home page displays a list of test suites a user has added to +favorites. +</p> +<img src="images/treble_vts_ui_main.png" title="VTS Dashboard landing page"> +<figcaption><strong>Figure 1.</strong> VTS Dashboard, home page.</figcaption> + +<p> +From this list, users can: +</p> +<ul> +<li>Select a test suite to view results for that suite. +<li>Click <strong>SHOW ALL</strong> to view all VTS test names. +<li>Select the <strong>Edit</strong> icon to modify the Favorites list.<br> +<img src="images/treble_vts_ui_favorites.png" title="VTS Dashboard favorites"> +<figcaption><strong>Figure 2.</strong> VTS Dashboard, editing Favorites +page.</figcaption></li> +</ul> + +<h2 id=test-results>Test results</h2> +<p> +Test Results displays the latest information about the selected test suite, +including a list of profiling points, a table of test case results in +chronological order, and a pie chart displaying the result breakdown of the +latest run (users can load older data by paging right). +</p> + +<img src="images/treble_vts_ui_results.png" title="VTS Dashboard results"> +<figcaption><strong>Figure 3.</strong> VTS Dashboard, test results.</figcaption> + +<p> +Users can filter data using queries or by modifying the test type (pre-submit, +post-submit, or both). Search queries support general tokens and field-specific +qualifiers; supported search fields are: device build ID, branch, target name, +device name, and test build ID. These are specified in the format: +<var>FIELD-ID</var>="<var>SEARCH QUERY</var>". Quotes are used to treat multiple +words as a single token to match with the data in the columns. +</p> + +<h2 id=profiling>Data profiling</h2> +<p> +Users can select a profiling point to reach an interactive view of the +quantitative data for that point in a <strong>line graph</strong> or +<strong>histogram</strong> (examples below). By default, the view displays the +latest information; users can use the date picker to load specific time windows. +</p> +<img src="images/treble_vts_ui_performance.png" title="VTS Dashboard performance"> +<figcaption><strong>Figure 4.</strong> VTS Dashboard, line graph performance. +</figcaption> +<p> +Line graphs display data from a collection of unordered performance values, +which can be useful when a test of performance produces a vector of performance +values that vary as a function of another variable (e.g., throughput versus +message size). +</p> +<img src="images/treble_vts_ui_histogram.png" title="VTS Dashboard histogram"> +<figcaption><strong>Figure 5.</strong> VTS Dashboard, histogram performance.</figcaption> + +<h2 id=coverage>Test coverage</h2> +<p> +Users can view coverage information from the coverage percent link in test +results. +</p> +<img src="images/treble_vts_ui_coverage.png" title="VTS Dashboard coverage"> +<figcaption> +<strong>Figure 6.</strong> VTS Dashboard, coverage percentages.</figcaption> + +<p> +For each test case and source file, users can view an expandable element +containing color-coded source code according to the coverage provided by the +selected test: +</p> +<img src="images/treble_vts_ui_coverage_source.png" title="VTS Dashboard coverage_source"> +<figcaption> +<strong>Figure 7.</strong> VTS Dashboard, coverage source code.</figcaption> + +<ul> +<li>Uncovered lines are highlighted <font style="color:red">red</font>.</li> +<li>Covered lines are highlighted <font style="color:green">green</font>.</li> +<li>Non-executable lines are <strong>uncolored</strong>.</li> +</ul> + +<p> +Coverage information is grouped depending into sections depending on how it was +provided at run-time. Tests may upload coverage: +</p> +<ul> +<li><strong>Per function</strong>. Section headers have the format "Coverage: +<var>FUNCTION-NAME</var>".</li> +<li><strong>In Total</strong> (provided at the end of the test run). Only one +header is present: "Coverage: All".</li> +</ul> + +<p> +The Dashboard fetches source code client-side from a server, which uses the +open-source +<a href="https://gerrit-review.googlesource.com/Documentation/rest-api.html" +class="external">Gerrit REST API</a>. +</p> + +<h2 id=monitor>Monitoring & testing</h2> +<p> +The VTS Dashboard provides the following monitors and unit tests. +</p> +<ul> +<li><strong>Test email alerts</strong>. Alerts are configured in a Cron job that +executes at a fixed interval of two (2) minutes. The job reads the VTS status +table to determine if new data has been uploaded to each table, done by checking +the test's raw data upload timestamp is newer than the last status update +timestamp. If the upload timestamp is newer, the job queries for new data +between now and the last raw data upload. New test case failures, continued test +case failures, transient test case failures, test case fixes, an inactive tests +are determined; this information is then sent in email format to the subscribers +of each test.</li> +<li><strong>Web service health</strong>. Google Stackdriver integrates with +Google App Engine to provide easy monitoring of the VTS Dashboard. Simple uptime +checks verify pages can be accessed while other tests can be created to verify +latency on each page, servlet, or database. These checks ensure the Dashboard is +always accessible (else an administrator will be notified).</li> +<li><strong>Analytics</strong>. You can integrate a VTS Dashboard page with +Google Cloud Analytics by specifying a valid Analytics ID in the page +configuration (the pom.xml file). Integration provides a more robust analysis of +page usage, user interaction, locality, session statistics, etc.</li> +</ul> + + </body> +</html> |