aboutsummaryrefslogtreecommitdiff
path: root/en/compatibility/vts/ui.html
diff options
context:
space:
mode:
Diffstat (limited to 'en/compatibility/vts/ui.html')
-rw-r--r--en/compatibility/vts/ui.html160
1 files changed, 160 insertions, 0 deletions
diff --git a/en/compatibility/vts/ui.html b/en/compatibility/vts/ui.html
new file mode 100644
index 00000000..a3894f59
--- /dev/null
+++ b/en/compatibility/vts/ui.html
@@ -0,0 +1,160 @@
+<html devsite>
+ <head>
+ <title>VTS Dashboard UI</title>
+ <meta name="project_path" value="/_project.yaml" />
+ <meta name="book_path" value="/_book.yaml" />
+ </head>
+ <body>
+ <!--
+ Copyright 2017 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+
+<p>
+The VTS Dashboard provides a cohesive user interface that uses material design
+to effectively display information about test results, profiling, and coverage.
+Dashboard styling uses open-source Javascript libraries including Materialize
+CSS and jQueryUI to process data delivered by Java servlets in Google App
+Engine.
+</p>
+
+<h2>Dashboard home</h2>
+<p>
+The Dashboard home page displays a list of test suites a user has added to
+favorites.
+</p>
+<img src="images/treble_vts_ui_main.png" title="VTS Dashboard landing page">
+<figcaption><strong>Figure 1.</strong> VTS Dashboard, home page.</figcaption>
+
+<p>
+From this list, users can:
+</p>
+<ul>
+<li>Select a test suite to view results for that suite.
+<li>Click <strong>SHOW ALL</strong> to view all VTS test names.
+<li>Select the <strong>Edit</strong> icon to modify the Favorites list.<br>
+<img src="images/treble_vts_ui_favorites.png" title="VTS Dashboard favorites">
+<figcaption><strong>Figure 2.</strong> VTS Dashboard, editing Favorites
+page.</figcaption></li>
+</ul>
+
+<h2 id=test-results>Test results</h2>
+<p>
+Test Results displays the latest information about the selected test suite,
+including a list of profiling points, a table of test case results in
+chronological order, and a pie chart displaying the result breakdown of the
+latest run (users can load older data by paging right).
+</p>
+
+<img src="images/treble_vts_ui_results.png" title="VTS Dashboard results">
+<figcaption><strong>Figure 3.</strong> VTS Dashboard, test results.</figcaption>
+
+<p>
+Users can filter data using queries or by modifying the test type (pre-submit,
+post-submit, or both). Search queries support general tokens and field-specific
+qualifiers; supported search fields are: device build ID, branch, target name,
+device name, and test build ID. These are specified in the format:
+<var>FIELD-ID</var>="<var>SEARCH QUERY</var>". Quotes are used to treat multiple
+words as a single token to match with the data in the columns.
+</p>
+
+<h2 id=profiling>Data profiling</h2>
+<p>
+Users can select a profiling point to reach an interactive view of the
+quantitative data for that point in a <strong>line graph</strong> or
+<strong>histogram</strong> (examples below). By default, the view displays the
+latest information; users can use the date picker to load specific time windows.
+</p>
+<img src="images/treble_vts_ui_performance.png" title="VTS Dashboard performance">
+<figcaption><strong>Figure 4.</strong> VTS Dashboard, line graph performance.
+</figcaption>
+<p>
+Line graphs display data from a collection of unordered performance values,
+which can be useful when a test of performance produces a vector of performance
+values that vary as a function of another variable (e.g., throughput versus
+message size).
+</p>
+<img src="images/treble_vts_ui_histogram.png" title="VTS Dashboard histogram">
+<figcaption><strong>Figure 5.</strong> VTS Dashboard, histogram performance.</figcaption>
+
+<h2 id=coverage>Test coverage</h2>
+<p>
+Users can view coverage information from the coverage percent link in test
+results.
+</p>
+<img src="images/treble_vts_ui_coverage.png" title="VTS Dashboard coverage">
+<figcaption>
+<strong>Figure 6.</strong> VTS Dashboard, coverage percentages.</figcaption>
+
+<p>
+For each test case and source file, users can view an expandable element
+containing color-coded source code according to the coverage provided by the
+selected test:
+</p>
+<img src="images/treble_vts_ui_coverage_source.png" title="VTS Dashboard coverage_source">
+<figcaption>
+<strong>Figure 7.</strong> VTS Dashboard, coverage source code.</figcaption>
+
+<ul>
+<li>Uncovered lines are highlighted <font style="color:red">red</font>.</li>
+<li>Covered lines are highlighted <font style="color:green">green</font>.</li>
+<li>Non-executable lines are <strong>uncolored</strong>.</li>
+</ul>
+
+<p>
+Coverage information is grouped depending into sections depending on how it was
+provided at run-time. Tests may upload coverage:
+</p>
+<ul>
+<li><strong>Per function</strong>. Section headers have the format "Coverage:
+<var>FUNCTION-NAME</var>".</li>
+<li><strong>In Total</strong> (provided at the end of the test run). Only one
+header is present: "Coverage: All".</li>
+</ul>
+
+<p>
+The Dashboard fetches source code client-side from a server, which uses the
+open-source
+<a href="https://gerrit-review.googlesource.com/Documentation/rest-api.html"
+class="external">Gerrit REST API</a>.
+</p>
+
+<h2 id=monitor>Monitoring &amp; testing</h2>
+<p>
+The VTS Dashboard provides the following monitors and unit tests.
+</p>
+<ul>
+<li><strong>Test email alerts</strong>. Alerts are configured in a Cron job that
+executes at a fixed interval of two (2) minutes. The job reads the VTS status
+table to determine if new data has been uploaded to each table, done by checking
+the test's raw data upload timestamp is newer than the last status update
+timestamp. If the upload timestamp is newer, the job queries for new data
+between now and the last raw data upload. New test case failures, continued test
+case failures, transient test case failures, test case fixes, an inactive tests
+are determined; this information is then sent in email format to the subscribers
+of each test.</li>
+<li><strong>Web service health</strong>. Google Stackdriver integrates with
+Google App Engine to provide easy monitoring of the VTS Dashboard. Simple uptime
+checks verify pages can be accessed while other tests can be created to verify
+latency on each page, servlet, or database. These checks ensure the Dashboard is
+always accessible (else an administrator will be notified).</li>
+<li><strong>Analytics</strong>. You can integrate a VTS Dashboard page with
+Google Cloud Analytics by specifying a valid Analytics ID in the page
+configuration (the pom.xml file). Integration provides a more robust analysis of
+page usage, user interaction, locality, session statistics, etc.</li>
+</ul>
+
+ </body>
+</html>