diff options
| author | Android Partner Docs <noreply@android.com> | 2017-09-18 22:26:40 +0000 |
|---|---|---|
| committer | android-build-merger <android-build-merger@google.com> | 2017-09-18 22:26:40 +0000 |
| commit | 6eedd0de9fbc4d4b6588f9462105ae0ef808594a (patch) | |
| tree | 2a4be68d8e74744710c58e454e448205defc3d76 /en/devices/tech | |
| parent | 1334d8b03b49a04bbf7a8383380c248faa4a492b (diff) | |
| parent | 6ed0385715ee494df4d2308c30b98d2cb3010d34 (diff) | |
| download | source.android.com-6eedd0de9fbc4d4b6588f9462105ae0ef808594a.tar.gz | |
Merge "Docs: Changes to source.android.com" am: 5ef6ffc303 am: 9f9cf10451
am: 6ed0385715
Change-Id: I516e4b2ae5bbfd19e9a98be5ae5360aca8c85c72
Diffstat (limited to 'en/devices/tech')
| -rw-r--r-- | en/devices/tech/config/filesystem.html | 2 | ||||
| -rw-r--r-- | en/devices/tech/dalvik/index.html | 13 | ||||
| -rw-r--r-- | en/devices/tech/ota/ab_implement.html | 333 | ||||
| -rw-r--r-- | en/devices/tech/ota/ab_updates.html | 1346 | ||||
| -rw-r--r-- | en/devices/tech/ota/images/ab-updates-state-machine.png | bin | 37851 -> 28998 bytes | |||
| -rw-r--r-- | en/devices/tech/ota/reduce_size.html | 249 |
6 files changed, 920 insertions, 1023 deletions
diff --git a/en/devices/tech/config/filesystem.html b/en/devices/tech/config/filesystem.html index c1bf33d5..10b7f0b4 100644 --- a/en/devices/tech/config/filesystem.html +++ b/en/devices/tech/config/filesystem.html @@ -132,7 +132,7 @@ way for starting privileged services).</p> <td>The filesystem path to configure. A path ending in / is considered a dir, else it's a file. <br><br>It is an error to specify multiple sections with the same - <code>[path]</code> in different files. In Python versions <= 3.2, the same + <code>[path]</code> in different files. In Python versions <= 3.2, the same file may contain sections that override the previous section; in Python 3.2, it's set to strict mode.</td> </tr> diff --git a/en/devices/tech/dalvik/index.html b/en/devices/tech/dalvik/index.html index 88cd47ba..2c84a8e5 100644 --- a/en/devices/tech/dalvik/index.html +++ b/en/devices/tech/dalvik/index.html @@ -144,12 +144,13 @@ by including both Java and native stack information. </p> <h2 id="Reporting_Problems">Reporting Problems</h2> <p>If you run into any issues that aren’t due to app JNI issues, please report -them via the Android Open Source Project Issue Tracker at <a -href="http://b.android.com">http://b.android.com</a>. -Please include an <code>"adb bugreport"</code> and link to the app in Google -Play store if available. Otherwise, if possible, attach an APK that reproduces -the issue. Please note that issues (including attachments) are publicly -visible.</p> +them through the <a href="/source/report-bugs#platform">Android Open Source +Project Issue Tracker</a>. Include an <code>adb bugreport</code> and link to +the app in Google Play store if available. Otherwise, if possible, attach an +APK that reproduces the issue.</p> + +<aside class="caution"><strong>Reminder</strong>: Issues (including attachments) +are publicly visible.</aside> </body> </html> diff --git a/en/devices/tech/ota/ab_implement.html b/en/devices/tech/ota/ab_implement.html new file mode 100644 index 00000000..4470c978 --- /dev/null +++ b/en/devices/tech/ota/ab_implement.html @@ -0,0 +1,333 @@ +<html devsite> + <head> + <title>Implementing A/B Updates</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>OEMs and SoC vendors who want to implement A/B system updates must ensure +their bootloader implements the boot_control HAL and passes the +<a href="#kernel">correct parameters</a> to the kernel.</p> + + +<h2 id=bootcontrol>Implementing the boot control HAL</h2> +<p>A/B-capable bootloaders must implement the <code>boot_control</code> HAL at +<code><a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h" class="external">hardware/libhardware/include/hardware/boot_control.h</a></code>. You can test implementations using the +<code><a href="https://android.googlesource.com/platform/system/extras/+/master/bootctl/" class="external">system/extras/bootctl</a></code> utility and +<code><a href="https://android.googlesource.com/platform/system/extras/+/refs/heads/master/tests/bootloader/" class="external">system/extras/tests/bootloader/</a></code>. +</p> + +<p>You must also implement the state machine shown below:</p> +<img src="images/ab-updates-state-machine.png"> +<figcaption><strong>Figure 1.</strong> Bootloader state machine</figcaption> + +<h2 id=kernel>Setting up the kernel</h2> +<p>To implement A/B system updates:</p> +<ol> +<li>Cherrypick the following kernel patch series (if needed): + <ul> + <li>If booting without ramdisk and using "boot as recovery", cherrypick + <a href="https://android-review.googlesource.com/#/c/158491/" class="external">android-review.googlesource.com/#/c/158491/</a>.</li> + <li>To set up dm-verity without ramdisk, cherrypick + <a href="https://android-review.googlesource.com/#/q/status:merged+project:kernel/common+branch:android-3.18+topic:A_B_Changes_3.18" class="external">android-review.googlesource.com/#/q/status:merged+project:kernel/common+branch:android-3.18+topic:A_B_Changes_3.18</a>.</li> + </ul> +</li> +<li>Ensure kernel command line arguments contain the following extra arguments: +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal">skip_initramfs rootwait ro init=/init root="/dev/dm-0 dm=system none ro,0 1 android-verity <public-key-id> <path-to-system-partition>"</code></pre> +... where the <code><public-key-id></code> value is the ID of the public +key used to verify the verity table signature (for details, see +<a href="/security/verifiedboot/dm-verity.html">dm-verity</a>).</li> +<li>Add the .X509 certificate containing the public key to the system keyring: + <ol> + <li>Copy the .X509 certificate formatted in the <code>.der</code> format to the + root of the <code>kernel</code> directory. If the .X509 certificate is + formatted as a <code>.pem</code> file, use the following <code>openssl</code> + command to convert from <code>.pem</code> to <code>.der</code> format: + <pre class="devsite-terminal devsite-click-to-copy"> +openssl x509 -in <x509-pem-certificate> -outform der -out <x509-der-certificate></pre> + </li> + <li>Build the <code>zImage</code> to include the certificate as part of the + system keyring. To verify,check the <code>procfs</code> entry (requires + <code>KEYS_CONFIG_DEBUG_PROC_KEYS</code> to be enabled): +<pre class="devsite-click-to-copy"> +angler:/# cat /proc/keys + +1c8a217e I------ 1 perm 1f010000 0 0 asymmetri +Android: 7e4333f9bba00adfe0ede979e28ed1920492b40f: X509.RSA 0492b40f [] +2d454e3e I------ 1 perm 1f030000 0 0 keyring +.system_keyring: 1/4</pre> + Successful inclusion of the .X509 certificate indicates the presence of the + public key in the system keyring (highlight denotes the public key ID).</li> + <li>Replace the space with <code>#</code> and pass it as + <code><public-key-id></code> in the kernel command line. For example, + pass <code>Android:#7e4333f9bba00adfe0ede979e28ed1920492b40f</code> in place of + <code><public-key-id></code>.</li> + </ol> +</li> +</ol> + +<h2 id="build-variables">Setting build variables</h2> + +<p>A/B-capable bootloaders must meet the following build variable criteria:</p> + +<table> +<tr> +<th>Must define for A/B target</th> +<td> +<ul> +<li><code>AB_OTA_UPDATER := true</code></li> +<li><code>AB_OTA_PARTITIONS := \</code><br/> +<code> boot \</code><br/> +<code> system \</code><br/> +<code> vendor</code><br/> +and other partitions updated through <code>update_engine</code> (radio, +bootloader, etc.)</li> +<li><code>BOARD_BUILD_SYSTEM_ROOT_IMAGE := true</code></li> +<li><code>TARGET_NO_RECOVERY := true</code></li> +<li><code>BOARD_USES_RECOVERY_AS_BOOT := true</code></li> +<li><code>PRODUCT_PACKAGES += \</code><br/> +<code> update_engine \</code><br/> +<code> update_verifier</code></li> +</ul> + +For an example, refer to +<code><a href="https://android.googlesource.com/device/google/marlin/+/android-7.1.0_r1/device-common.mk" class="external">/device/google/marlin/+/android-7.1.0_r1/device-common.mk</a></code>. +You can optionally conduct the post-install (but pre-reboot) dex2oat step +described in <a href="#compilation">Compiling</a>. +</td> +</tr> +<th>Cannot define for A/B target</th> +<td> +<ul> +<li><code>BOARD_RECOVERYIMAGE_PARTITION_SIZE</code></li> +<li><code>BOARD_CACHEIMAGE_PARTITION_SIZE</code></li> +<li><code>BOARD_CACHEIMAGE_FILE_SYSTEM_TYPE</code></li> +</ul> +</td> +</tr> +<tr> +<th>Optional for debug builds</th> +<td><code>PRODUCT_PACKAGES_DEBUG += update_engine_client</code></td> +</tr> +<tr> +</table> + +<h2 id="partitions">Setting partitions (slots)</h2> +<p>A/B devices do not need a recovery partition or cache partition because +Android no longer uses these partitions. The data partition is now used for the +downloaded OTA package, and the recovery image code is on the boot partition. +All partitions that are A/B-ed should be named as follows (slots are always +named <code>a</code>, <code>b</code>, etc.): <code>boot_a</code>, +<code>boot_b</code>, <code>system_a</code>, <code>system_b</code>, +<code>vendor_a</code>, <code>vendor_b</code>.</p> + +<h3 id=cache>Cache</h3> + +<p>For non-A/B updates, the cache partition was used to store downloaded OTA +packages and to stash blocks temporarily while applying updates. There was +never a good way to size the cache partition: how large it needed to be +depended on what updates you wanted to apply. The worst case would be a cache +partition as large as the system image. With A/B updates there's no need to +stash blocks (because you're always writing to a partition that isn't currently +used) and with streaming A/B there's no need to download the whole OTA package +before applying it.</p> + +<h3 id=recovery>Recovery</h3> + +<p>The recovery RAM disk is now contained in the <code>boot.img</code> file. +When going into recovery, the bootloader <strong>cannot</strong> put the +<code>skip_initramfs</code> option on the kernel command line.</p> + +<p>For non-A/B updates, the recovery partition contains the code used to apply +updates. A/B updates are applied by <code>update_engine</code> running in the +regular booted system image. There is still a recovery mode used to implement +factory data reset and sideloading of update packages (which is where the name +"recovery" came from). The code and data for recovery mode is stored in the +regular boot partition in a ramdisk; to boot into the system image, the +bootloader tells the kernel to skip the ramdisk (otherwise the device boots into +recovery mode. Recovery mode is small (and much of it was already on the boot +partition), so the boot partition doesn't increase in size.</p> + +<h3 id="fstab">Fstab</h3> + +<p>The <code>slotselect</code> argument <strong>must</strong> be on the line for +the A/B-ed partitions. For example:</p> + +<pre class="devsite-click-to-copy"> +<path-to-block-device>/vendor /vendor ext4 ro +wait,verify=<path-to-block-device>/metadata,slotselect +</pre> + +<p>No partition should be named <code>vendor</code>. Instead, partition +<code>vendor_a</code> or <code>vendor_b</code> will be selected and mounted on +the <code>/vendor</code> mount point.</p> + +<h3 id="kernel-slot-arguments">Kernel slot arguments</h3> + +<p>The current slot suffix should be passed either through a specific device +tree (DT) node (<code>/firmware/android/slot_suffix</code>) or through the +<code>androidboot.slot_suffix</code> command line argument.</p> + +<p>By default, fastboot flashes only slot <code>a</code> on an A/B device and +sets the current slot to <code>a</code>. If the update package also contains +images for slot <code>b</code>, fastboot flashes those images as well. Available +options include:</p> + +<ul> +<li><code>--slot</code>. Prompt fastboot to use slot <code>b</code> instead of +slot <code>a</code>.</li> +<li><code>--set-active</code>. Set the slot as active.</li> +<li><code>fastboot --help</code>. Get details on commands.</li> +</ul> + +<p>If the bootloader implements fastboot, it should support the command +<code>set_active <slot></code> that sets the current active slot +to the given slot (this must also clear the unbootable flag for that slot and +reset the retry count to default values). The bootloader should also support the +following variables:</p> + +<ul> +<li><code>has-slot:<partition-base-name-without-suffix></code>. Returns +“yes” if the given partition supports slots, “no” otherwise.</li> +<li><code>current-slot</code>. Returns the slot suffix that will be booted from +next.</li> +<li><code>slot-count</code>. Returns an integer representing the number of +available slots. Currently, two slots are supported so this value is +<code>2</code>.</li> +<li><code>slot-successful:<slot-suffix></code>. Returns "yes" if the given +slot has been marked as successfully booting, "no" otherwise.</li> +<li><code>slot-unbootable:<slot-suffix></code>. Returns “yes” if the given +slot is marked as unbootable, "no" otherwise.</li> +<li><code>slot-retry-count<slot suffix></code>. Number of retries remaining to +attempt to boot the given slot.</li> +</ul> + +<p>To view all variables, run +<code class="devsite-terminal devsite-click-to-copy">fastboot getvar all</code>. +</p> + +<h2 id="ota-package-generation">Generating OTA packages</h2> + +<p>The <a href="/devices/tech/ota/tools.html">OTA package tools</a> follow the +same commands as the commands for non-A/B devices. The +<code>target_files.zip</code> file must be generated by defining the build +variables for the A/B target. The OTA package tools automatically identify and +generate packages in the format for the A/B updater.</p> + +<p>Examples:</p> +<ul> +<li>To generate a full OTA: +<pre class="devsite-terminal devsite-click-to-copy"> +./build/tools/releasetools/ota_from_target_files \ + dist_output/tardis-target_files.zip ota_update.zip +</pre> +</li> +<li>To generate an incremental OTA: +<pre class="devsite-terminal devsite-click-to-copy"> +./build/tools/releasetools/ota_from_target_files \ + -i PREVIOUS-tardis-target_files.zip \ + dist_output/tardis-target_files.zip incremental_ota_update.zip +</pre> +</li> +</ul> + +<h2 id="configuration">Configuring partitions</h2> + +<p>The <code>update_engine</code> can update any pair of A/B partitions defined +in the same disk. A pair of partitions has a common prefix (such as +<code>system</code> or <code>boot</code>) and per-slot suffix (such as +<code>_a</code>). The list of partitions for which the payload generator defines +an update is configured by the <code>AB_OTA_PARTITIONS</code> make variable.</p> + +<p>For example, if a pair of partitions <code>bootloader_a</code> and +<code>booloader_b</code> are included (<code>_a</code> and <code>_b</code> are +the slot suffixes), you can update these partitions by specifying the following +on the product or board configuration:</p> + +<pre class="devsite-click-to-copy"> +AB_OTA_PARTITIONS := \ + boot \ + system \ + bootloader +</pre> + +<p>All partitions updated by <code>update_engine</code> must not be modified by +the rest of the system. During incremental or <em>delta</em> updates, the binary +data from the current slot is used to generate the data in the new slot. Any +modification may cause the new slot data to fail verification during the update +process, and therefore fail the update.</p> + +<h2 id="post-install">Configuring post-installation</h2> + +<p>You can configure the post-install step differently for each updated +partition using a set of key-value pairs. To run a program located at +<code>/system/usr/bin/postinst</code> in a new image, specify the path relative +to the root of the filesystem in the system partition.</p> + +<p>For example, <code>usr/bin/postinst</code> is +<code>system/usr/bin/postinst</code> (if not using a RAM disk). Additionally, +specify the filesystem type to pass to the <code>mount(2)</code> system call. +Add the following to the product or device <code>.mk</code> files (if +applicable):</p> + +<pre class="devsite-click-to-copy"> +AB_OTA_POSTINSTALL_CONFIG += \ + RUN_POSTINSTALL_system=true \ + POSTINSTALL_PATH_system=usr/bin/postinst \ + FILESYSTEM_TYPE_system=ext4 +</pre> + +<h2 id="compilation">Compiling</h2> +<p>For security reasons, <code>system_server</code> cannot use +<a href="/devices/tech/dalvik/jit-compiler">just-in-time (JIT)</a> compilation. +This means you must compile ahead of time odex files for +<code>system_server</code> and its dependencies at a minimum; anything else is +optional.</p> + +<p>To compile apps in the background, you must add the following to the +product's device configuration (in the product's device.mk):</p> + +<ol> +<li>Include the native components in the build to ensure compilation script and +binaries are compiled and included in the system image. +<pre class="devsite-click-to-copy"> + # A/B OTA dexopt package + PRODUCT_PACKAGES += otapreopt_script +</pre></li> +<li>Connect the compilation script to <code>update_engine</code> such that runs +as a post-install step. +<pre class="devsite-click-to-copy"> + # A/B OTA dexopt update_engine hookup + AB_OTA_POSTINSTALL_CONFIG += \ + RUN_POSTINSTALL_system=true \ + POSTINSTALL_PATH_system=system/bin/otapreopt_script \ + FILESYSTEM_TYPE_system=ext4 \ + POSTINSTALL_OPTIONAL_system=true +</pre> +</li> +</ol> + +<p>For help installing the preopted files in the unused second system partition, +refer to <a href="/devices/tech/dalvik/configure.html#other_odex">First boot +installation of DEX_PREOPT files</a>.</p> + + </body> +</html> diff --git a/en/devices/tech/ota/ab_updates.html b/en/devices/tech/ota/ab_updates.html index 871d8f63..5d51df1f 100644 --- a/en/devices/tech/ota/ab_updates.html +++ b/en/devices/tech/ota/ab_updates.html @@ -21,138 +21,134 @@ limitations under the License. --> +<p>A/B system updates, also known as seamless updates, ensure a workable booting +system remains on the disk during an +<a href="/devices/tech/ota/index.html">over-the-air (OTA) update</a>. This +approach reduces the likelihood of an inactive device after an update, which +means fewer device replacements and device reflashes at repair and warranty +centers. Other commercial-grade operating systems such as +<a href="https://www.chromium.org/chromium-os">ChromeOS</a> also use A/B updates +successfully.</p> - -<p> - A/B system updates, also known as seamless updates, ensure a workable booting - system remains on the disk during an <a href="/devices/tech/ota/index.html" - >over-the-air (OTA) update</a>. This reduces the likelihood of an inactive - device afterward, which means fewer device replacements and device reflashes at - repair and warranty centers. This approach is already explored successfully by - other commercial-grade operating systems, such as <a - href="https://www.chromium.org/chromium-os">ChromeOS</a>, and Android 8.0 - comes with the necessary platform changes to conduct streaming updates.</p> - -<p class="note"><strong>Note:</strong> Android 7.1, in which A/B updates were -introduced, requires the following patches to be cherrypicked before streaming -updates can be enabled. This is true whether using <a -href="https://www.android.com/gms/">Google Mobile Services (GMS)</a> or any -other update client.</p> +<p>A/B system updates provide the following benefits:</p> <ul> - <li><a href="https://android-review.googlesource.com/333624">Allow to cancel a proxy resolution request</a></li> - <li><a href="https://android-review.googlesource.com/333625">Fix terminating a transfer while resolving proxies</a></li> - <li><a href="https://android-review.googlesource.com/333626">Add unittest for TerminateTrasnfer between ranges</a></li> - <li><a href="https://android-review.googlesource.com/333627">Cleanup the RetryTimeoutCallback()</a></li> +<li>OTA updates can occur while the system is running, without interrupting the +user (including app optimizations that occur after a reboot). This means users +can continue to use their devices during an OTA—the only downtime during +an update is when the device reboots into the updated disk partition.</li> +<li>If an OTA fails, the device boots into the pre-OTA disk partition and +remains usable. The download of the OTA can be attempted again.</li> +<li>Any errors (such as I/O errors) affect only the <strong>unused</strong> +partition set and can be retried. Such errors also become less likely because +the I/O load is deliberately low to avoid degrading the user experience.</li> +<li>The cache partition is no longer used to store OTA update packages, so there +is no need for sizing the cache partition.</li> +<li><a href="/security/verifiedboot/dm-verity.html">dm-verity</a> guarantees a +device will boot an uncorrupted image. If a device doesn't boot due to a bad OTA +or dm-verity issue, the device can reboot into an old image. (Android +<a href="/security/verifiedboot/">Verified Boot</a> does not require A/B +updates.)</li> </ul> -<p> - Users don't always have enough space on <code>/data</code> to download the - update package, and neither OEMs nor users want to waste space on a - <code>/cache</code> partition; so some users go without updates because - they have nowhere to store the update package. A/B updates have the option of - streaming the update to address this issue: streaming writes blocks - straight to the B partition as they are downloaded, without having to store - them on <code>/data</code>. Therefore, streaming A/B updates need almost no - temporary storage and need just enough for roughly 100 KiB of metadata.</p> - -<p>Customers can continue to use their devices during an OTA. The only downtime - during an update is when the device reboots into the updated disk partition. If - the OTA fails, the device is still useable since it will boot into the pre-OTA - disk partition. The download of the OTA can be attempted again. A/B system - updates implemented through OTA are recommended for new devices only. </p> - -<p> - A/B system updates affect: -</p> +<h2 id=overview>About A/B system updates</h2> + +<p>A/B system updates affect the following:</p> <ul> - <li>Interactions with the bootloader</li> - <li>Partition selection</li> - <li>The build process</li> - <li>OTA update package generation</li> +<li>Partition selection (slots), the <code>update_engine</code> daemon, and +bootloader interactions (described below)</li> +<li>Build process and OTA update package generation (described in +<a href="/devices/tech/ota/ab_implement.html">Implementing A/B Updates</a>)</li> </ul> -<p> - The existing <a href="/security/verifiedboot/dm-verity.html">dm-verity</a> - feature guarantees the device will boot an uncorrupted image. If - a device doesn't boot, because of a bad OTA or dm-verity issue, the device can - reboot into an old image. -</p> - -<p class="note"><strong>Note:</strong> Android <a - href="/security/verifiedboot/">Verified Boot</a> does not require A/B - updates.</p> - -<h2 id="overview">Overview</h2> - -<p> - The A/B system is robust because any errors (such as I/O errors) affect only - the <strong>unused</strong> partition set and can be retried. Such errors also - become less likely because the I/O load is deliberately low to avoid degrading - the user experience. -</p> - -<p> - OTA updates can occur while the system is running, without interrupting the - user. This includes the app optimizations that occur after a reboot. - Additionally, the cache partition is no longer used to store OTA update - packages; there is no need for sizing the cache partition. -</p> - -<p> - A/B system updates use a background daemon called <code>update_engine</code> - and two sets of partitions. The two sets of partitions are referred to as - <em>slots</em>, normally as slot A and slot B. The system runs from one slot, - the <em>current</em> slot, while the partitions in the <em>unused</em> slot are - not accessed by the running system (for normal operation). -</p> +<aside class="note"><strong>Note:</strong> A/B system updates implemented through +OTA are recommended for new devices only.</aside> + +<h3 id=slots>Partition selection (slots)</h3> + +<p>A/B system updates use two sets of partitions referred to as <em>slots</em> +(normally slot A and slot B). The system runs from the <em>current</em> slot +while the partitions in the <em>unused</em> slot are not accessed by the running +system during normal operation. This approach makes updates fault resistant by +keeping the unused slot as a fallback: If an error occurs during or immediately +after an update, the system can rollback to the old slot and continue to have a +working system. To achieve this goal, no partition used by the <em>current</em> +slot should be updated as part of the OTA update (including partitions for which +there is only one copy).</p> + +<p>Each slot has a <em>bootable</em> attribute that states whether the slot +contains a correct system from which the device can boot. The current slot is +bootable when the system is running, but the other slot may have an old (still +correct) version of the system, a newer version, or invalid data. Regardless of +what the <em>current</em> slot is, there is one slot that is the <em>active</em> +slot (the one the bootloader will boot form on the next boot) or the +<em>preferred</em> slot.</p> + +Each slot also has a <em>successful</em> attribute set by the user space, which +is relevant only if the slot is also bootable. A successful slot should be able +to boot, run, and update itself. A bootable slot that was not marked as +successful (after several attempts were made to boot from it) should be marked +as unbootable by the bootloader, including changing the active slot to another +bootable slot (normally to the slot running immediately before the attempt to +boot into the new, active one). The specific details of the interface are +defined in +<code><a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h" class="external-link">boot_control.h</a></code>. +</p> + +<h3 id="update-engine">Update engine daemon</h3> + +<p>A/B system updates use a background daemon called <code>update_engine</code> +to prepare the system to boot into a new, updated version. This daemon can +perform the following actions:</p> -<p> - The goal of this feature is to make updates fault resistant by keeping the - unused slot as a fallback. If there is an error during an update or immediately - after an update, the system can rollback to the old slot and continue to have a - working system. To achieve this goal, none of the partitions used by the - <em>current</em> slot should be updated as part of the OTA update (including - partitions for which there is only one copy). -</p> +<ul> +<li>Read from the current slot A/B partitions and write any data to the unused +slot A/B partitions as instructed by the OTA package.</li> +<li>Call the <code>boot_control</code> interface in a pre-defined workflow.</li> +<li>Run a <em>post-install</em> program from the <em>new</em> partition after +writing all the unused slot partitions, as instructed by the OTA package. (For +details, see <a href="#post-installation">Post-installation</a>).</li> +</ul> -<p> - Each slot has a <em>bootable</em> attribute, which states whether the slot - contains a correct system from which the device can boot. The current slot is - clearly bootable when the system is running, but the other slot may have an old - (still correct) version of the system, a newer version, or invalid data. - Regardless of what the <em>current</em> slot is, there is one slot which is the - <em>active</em> or preferred slot. The active slot is the one the bootloader - will boot from on the next boot. Finally, each slot has a <em>successful</em> - attribute set by the user space, which is only relevant if the slot is also - bootable. -</p> +<p>As the <code>update_engine</code> daemon is not involved in the boot process +itself, it is limited in what it can do during an update by the +<a href="/security/selinux/">SELinux</a> policies and features in the +<em>current</em> slot (such policies and features can't be updated until the +system boots into a new version). To maintain a robust system, the update +process <strong>should not</strong> modify the partition table, the contents of +partitions in the current slot, or the contents of non-A/B partitions that can't +be wiped with a factory reset.</p> + +<p>The <code>update_engine</code> source is located in +<code><a href="https://android.googlesource.com/platform/system/update_engine/" class="external">system/update_engine</a></code>. +The A/B OTA dexopt files are split between <code>installd</code> and a package +manager:</p> +<ul> +<li><code><a href="https://android.googlesource.com/platform/frameworks/native/+/master/cmds/installd/" class="external-link">frameworks/native/cmds/installd/</a></code>ota* +includes the postinstall script, the binary for chroot, the installd clone that +calls dex2oat, the post-OTA move-artifacts script, and the rc file for the move +script.</li> +<li><code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptService.java" class="external-link">frameworks/base/services/core/java/com/android/server/pm/OtaDexoptService.java</a></code> +(plus <code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptShellCommand.java" class="external-link">OtaDexoptShellCommand</a></code>) +is the package manager that prepares dex2oat commands for applications.</li> +</ul> -<p> - A successful slot should be able to boot, run, and update itself. A bootable - slot that was not marked as successful (after several attempts were made to - boot from it) should be marked as unbootable by the bootloader, including - changing the active slot to another bootable slot (normally to the slot running - right before the attempt to boot into the new, active one). The specific - details of the interface are defined in - <code><a class="external-link" target="_blank" - href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h" >boot_control.h</a></code>. +<p>For a working example, refer to +<code><a href="https://android.googlesource.com/device/google/marlin/+/nougat-dr1-release/device-common.mk" class="external-link">/device/google/marlin/device-common.mk</a></code>. </p> -<h3 id="bootloader-state-examples">Bootloader state examples</h3> +<h3 id="bootloader-interactions">Bootloader interactions</h3> -<p> - The <code>boot_control</code> HAL is used by <code>update_engine</code> (and - possibly other daemons) to instruct the bootloader what to boot from. These - are common example scenarios and their associated states: -</p> +<p>The <code>boot_control</code> HAL is used by <code>update_engine</code> (and +possibly other daemons) to instruct the bootloader what to boot from. Common +example scenarios and their associated states include the following:</p> <ul> <li> <strong>Normal case</strong>: The system is running from its current slot, - either slot A or B. No updates have been applied so far. The system's current - slot is bootable, successful, and the active slot. + either slot A or B. No updates have been applied so far. The system's + current slot is bootable, successful, and the active slot. </li> <li> <strong>Update in progress</strong>: The system is running from slot B, so @@ -163,706 +159,229 @@ other update client.</p> <li> <strong>Update applied, reboot pending</strong>: The system is running from slot B, slot B is bootable and successful, but slot A was marked as active - (and therefore is marked as bootable). Slot A is not yet marked as successful - and some number of attempts to boot from slot A should be made by the - bootloader. + (and therefore is marked as bootable). Slot A is not yet marked as + successful and some number of attempts to boot from slot A should be made by + the bootloader. </li> <li> <strong>System rebooted into new update</strong>: The system is running from - slot A for the first time, slot B is still bootable and successful while slot - A is only bootable, and still active but not successful. A user space daemon - should mark slot A as successful after some checks are made. + slot A for the first time, slot B is still bootable and successful while + slot A is only bootable, and still active but not successful. A user space + daemon should mark slot A as successful after some checks are made. </li> </ul> -<h3 id="update-engine-features">Update Engine features</h3> - -<p> - The <code>update_engine</code> daemon runs in the background and prepares the - system to boot into a new, updated version. The <code>update_engine</code> - daemon is not involved in the boot process itself and is limited in what it can - do during an update. The <code>update_engine</code> daemon can do the - following: -</p> - +<h2 id="streaming-updates">Streaming updates</h2> +<p>User devices don't always have enough space on <code>/data</code> to download +the update package. As neither OEMs nor users want to waste space on a +<code>/cache</code> partition, some users go without updates because the device +has nowhere to store the update package. To address this issue, Android 8.0 adds +support for streaming A/B updates that write blocks directly to the B partition +as they are downloaded, without having to store the blocks on +<code>/data</code>. Streaming A/B updates need almost no temporary storage and +require just enough storage for roughly 100 KiB of metadata.</p> + +<p>To enable streaming updates in Android 7.1, cherrypick the following +patches:</p> <ul> - <li> - Read from the current slot A/B partitions and write any data to the unused - slot A/B partitions as instructed by the OTA package - </li> - <li> - Call the <code>boot_control</code> interface in a pre-defined workflow - </li> - <li> - Run a <em>post-install</em> program from the <em>new</em> partition after - writing all the unused slot partitions, as instructed by the OTA package - </li> +<li> +<a href="https://android-review.googlesource.com/333624" class="external">Allow +to cancel a proxy resolution request</a></li> +<li> +<a href="https://android-review.googlesource.com/333625" class="external">Fix +terminating a transfer while resolving proxies</a></li> +<li> +<a href="https://android-review.googlesource.com/333626" class="external">Add +unittest for TerminateTransfer between ranges</a></li> +<li> +<a href="https://android-review.googlesource.com/333627" class="external">Cleanup +the RetryTimeoutCallback()</a></li> </ul> -<p> - The post-install step is described in detail below. Note that the - <code>update_engine</code> daemon is limited by the - <a href="/security/selinux/">SELinux</a> policies and features in the - <em>current</em> slot; those policies and features can't be updated until the - system boots into a new version. To achieve a robustness goal, the update - process should not: -</p> - -<ul> - <li>Modify the partition table</li> - <li>Modify the contents of partitions in the current slot</li> - <li> - Modify the contents of non-A/B partitions that can't be wiped with a - factory reset - </li> - </ul> - - <h3 id="update-engine-source">Update Engine source</h3> - -<p>The source to <code>update_engine</code> is in <code><a - class="external-link" target="_blank" href="https://android.googlesource.com/platform/system/update_engine/">system/update_engine</a></code>. - The A/B OTA dexopt files are split between installd and package manager:</p> -<ul> - <li><code><a class="external-link" target="_blank" href="https://android.googlesource.com/platform/frameworks/native/+/master/cmds/installd/">frameworks/native/cmds/installd/</a></code>ota* ---- the postinstall script, the binary for chroot, the installd clone -that calls dex2oat, the post-OTA move-artifacts script, the rc file for -the move script.</li> - <li><code><a class="external-link" target="_blank" - href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptService.java">frameworks/base/services/core/java/com/android/server/pm/OtaDexoptService.java</a></code> - (plus <code><a class="external-link" target="_blank" href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptShellCommand.java">OtaDexoptShellCommand</a></code>) is the package manager side that will prepare - all the dex2oat commands for the apps</li> -</ul> - -<p>A working example can be found in <code><a class="external-link" target="_blank" href="https://android.googlesource.com/device/google/marlin/+/nougat-dr1-release/device-common.mk">/device/google/marlin/device-common.mk</a></code>.</p> +<p>These patches are required for Android 7.1 whether using +<a href="https://www.android.com/gms/">Google Mobile Services (GMS)</a> or any +other update client.</p> <h2 id="life-of-an-a-b-update">Life of an A/B update</h2> -<p> - The update process starts when an OTA package, referred to in code as a - <em>payload</em>, is available for downloading. Policies in the device may - defer the payload download and application based on battery level, user - activity, whether it is connected to a charger, or other policies. But since - the update runs in the background, the user might not know that an update is - in progress and the process can be interrupted at any point due to policies or - unexpected reboots. -</p> - -<p> - Optionally, metadata in the OTA package itself indicates the update can be - streamed. The same package can be used for non-streaming installation, as - well. The server may use the metadata to tell the client it's streaming - so the client will hand off the OTA to <code>update_engine</code> correctly. - To enable streaming updates, manufacturers with their own server and client - would need to: - <ol> - <li>on the server, identify the update is streaming (or just assume all are)</li> - <li>on the client, make the correct call to <code>update_engine</code> for streaming</li> - </ol> - -<p> - Device manufacturers should use the fact that the package is of the streaming - variant to send a flag to the client to trigger hand off to the framework - side as streaming. -</p> - -<p> - The steps in the update process after a payload is available are as follows: -</p> - -<p> - <strong>Step 1:</strong> The current slot (or "source slot") is marked as - successful (if not already marked) with <code>markBootSuccessful()</code>. -</p> - -<p> - <strong>Step 2:</strong> The unused slot (or "target slot") is marked as - unbootable by calling the function <code>setSlotAsUnbootable()</code>. -</p> - -<p> - The current slot is always marked as successful at the beginning of the update - to prevent the bootloader from falling back to the unused slot, which will soon - have invalid data. If the system has reached the point where it can start - applying an update, the current slot is marked as successful even if other - major components are broken (such as the UI in a crash loop) since it's - possible to push new software to fix these major problems. -</p> - -<p> - The update payload is an opaque blob with the instructions to update to the - new version. The update payload consists of basically two parts: the metadata - and the extra data associated with the instructions. The metadata is - relatively small and contains a list of operations to produce and verify the - new version on the target slot. For example, an operation could decompress a - certain blob and write it to certain blocks in a target partition, or read from - a source partition, apply a binary patch, and write to certain blocks in a - target partition. The extra data associated to the operations, not included in - the metadata, is the bulk of the update payload and would consist of the - compressed blob or binary patch in these examples. -</p> - -<p> - <strong>Step 3:</strong> The payload metadata is downloaded. -</p> - -<p> - <strong>Step 4:</strong> For each operation defined in the metadata, in order, - the associated data (if any) is downloaded to memory, the operation is applied, - and the associated memory is discarded. -</p> - -<p> - These two steps take most of the update time, as they involve writing and - downloading large amounts of data, and are likely to be interrupted for reasons - of policy or reboot. -</p> - -<p> - <strong>Step 5:</strong> The whole partitions are re-read and verified against - the expected hash. -</p> - -<p> - <strong>Step 6:</strong> The post-install step (if any) is run. -</p> - -<p> - In the case of an error during the execution of any step, the update fails and - is re-attempted with possibly a different payload. If all the steps so far have - succeeded, the update succeeds and the last step is executed. -</p> - -<p> - <strong>Step 7:</strong> The <em>unused slot</em> is marked as active by - calling <code>setActiveBootSlot()</code>. -</p> - -<p> - Marking the unused slot as active doesn't mean it will finish booting. The - bootloader—or system itself—can switch the active slot back if it doesn't - read a successful state. -</p> - -<h3 id="post-install-step">Post-install step</h3> - -<p> - The post-install step consists of running a program from the "new update" - version while still running in the old version. If defined in the OTA package, - this step is mandatory and the program must return with exit code - <code>0</code>; otherwise, the update fails. -</p> - -<p> - For every partition where a post-install step is defined, - <code>update_engine</code> mounts the new partition into a specific location - and executes the program specified in the OTA relative to the mounted - partition. For example, if the post-install program is defined as - <code>usr/bin/postinstall</code> in the system partition, this partition from - the unused slot will be mounted in a fixed location (for example, in - <code>/postinstall_mount</code>) and the - <code>/postinstall_mount/usr/bin/postinstall</code> command will be executed. - Note that for this step to work, the following are required: -</p> - -<ul> - <li> - The old kernel needs to be able to mount the new filesystem format. The - filesystem type cannot change unless there's support for it in the old - kernel (which includes details such as the compression algorithm used if - using a compressed filesystem like SquashFS). - </li> - <li> - The old kernel needs to understand the new partition's post-install program - format. If using an ELF binary, it should be compatible with the old - kernel (e.g. a 64-bit new program running on an old 32-bit kernel if the - architecture switched from 32- to 64-bit builds). Also, the libraries - will be loaded from the old system image, not the new one, unless the loader - (<code>ld</code>) is instructed to use other paths or build a static binary. - </li> - <li> - The new post-install program will be limited by the SELinux policies defined - in the old system. - </li> -</ul> - -<p> - An example case is to use a shell script as a post-install program (interpreted - by the old system's shell binary with a <code>#!</code> marker at the top) and - then set up library paths from the new environment for executing a more complex - binary post-install program. -</p> - -<p> - Another example case is to run the post-install step from a dedicated smaller - partition, so the filesystem format in the main system partition can be updated - without incurring backward compatibility issues or stepping-stone updates, - allowing users to update straight to the latest version from a factory image. -</p> - -<p> - Due to the SELinux policies, the post-install step is suitable for - performing tasks required by design on a given device or other best-effort - tasks: update the A/B-capable firmware or bootloader, prepare copies of some - databases for the new version, etc. This step is not suitable for one-off bug - fixes before reboot that require unforeseen permissions. -</p> - -<p> - The selected post-install program runs in the <code>postinstall</code> SELinux - context. All the files in the new mounted partition will be tagged with - <code>postinstall_file</code>, regardless of what their attributes are after - rebooting into that new system. Changes to the SELinux attributes in the new - system won't impact the post-install step. If the post-install program needs - extra permissions, those must be added to the post-install context. -</p> - -<h2 id="implementation">Implementation</h2> - -<p> - OEMs and SoC vendors who wish to implement the feature must add the following - support to their bootloaders: -</p> - -<ul> - <li> - Pass the <a href="#kernel-command-line-arguments">correct parameters</a> - to the kernel - </li> - <li> - Implement the <code>boot_control</code> HAL - (<code><a class="external-link nowrap" target="_blank" - href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h" - >hardware/libhardware/include/hardware/boot_control.h</a></code>) - </li> - <li>Implement the state machine as shown in Figure 1:</li> -</ul> - -<img src="/devices/tech/ota/images/ab-updates-state-machine.png"> - -<p class="img-caption"><strong>Figure 1.</strong> Bootloader state machine</p> - -<p> - The boot control HAL can be tested using the - <code><a class="external-link" target="_blank" - href="https://android.googlesource.com/platform/system/extras/+/master/bootctl/" - >system/extras/bootctl</a></code> utility. -</p> - -<p>Some tests have been implemented for Brillo:</p> - -<ul> - <li> - <code><a class="external-link nowrap" target="_blank" - href="https://android.googlesource.com/platform/system/extras/+/refs/heads/master/tests/bootloader/" - >system/extras/tests/bootloader/</a></code> - </li> - <li> - <a class="external-link nowrap" target="_blank" - href="https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/server/site_tests/brillo_BootLoader/brillo_BootLoader.py" - >chromium.googlesource.com/chromiumos/third_party/autotest/+/master/server/site_tests/brillo_BootLoader/brillo_BootLoader.py</a> - </li> -</ul> - -<h3 id="kernel-patches">Kernel patches</h3> - -<ul> - <li> - <a class="external-link nowrap" target="_blank" - href="https://android-review.googlesource.com/#/c/158491/" - >android-review.googlesource.com/#/c/158491/</a> - </li> - <li> - <a class="external-link nowrap" target="_blank" - href="https://android-review.googlesource.com/#/q/status:merged+project:kernel/common+branch:android-3.18+topic:A_B_Changes_3.18" - >android-review.googlesource.com/#/q/status:merged+project:kernel/common+branch:android-3.18+topic:A_B_Changes_3.18</a> - </li> -</ul> - -<h3 id="kernel-command-line-arguments">Kernel command line arguments</h3> - -<p> - The kernel command line arguments <strong>must</strong> contain the following - extra arguments: -</p> - -<pre class="devsite-click-to-copy"> -<code class="devsite-terminal">skip_initramfs rootwait ro init=/init root="/dev/dm-0 dm=system none ro,0 1 \ - android-verity <public-key-id> <path-to-system-partition>"</code> -</pre> - -<p> - The <code><public-key-id></code> value is the ID of the public key used - to verify the verity table signature (see - <a href="/security/verifiedboot/dm-verity.html">dm-verity</a>). -</p> - -<h4> - To add the .X509 certificate containing the public key to the system keyring: -</h4> - -<ol> - <li> - Copy the .X509 certificate formatted in the <code>.der</code> - format to the root of the <code>kernel</code> directory. Use the following - <code>openssl</code> command to convert from <code>.pem</code> to - <code>.der</code> format (if the .X509 certificate is formatted in - <code>.pem</code> format): -<pre class="devsite-terminal devsite-click-to-copy"> -openssl x509 -in <x509-pem-certificate> -outform der -out <x509-der-certificate> -</pre> - </li> - <li> - Once copied to the kernel build root, build the <code>zImage</code> to - include the certificate as part of the system keyring. This can be verified - from the following <code>procfs</code> entry (requires - <code>KEYS_CONFIG_DEBUG_PROC_KEYS</code> to be enabled): -<pre class="devsite-click-to-copy"> -angler:/# cat /proc/keys - -1c8a217e I------ 1 perm 1f010000 0 0 asymmetri -Android: 7e4333f9bba00adfe0ede979e28ed1920492b40f: X509.RSA 0492b40f [] -2d454e3e I------ 1 perm 1f030000 0 0 keyring -.system_keyring: 1/4 -</pre> - </li> -</ol> - -<p> - Successful inclusion of the .X509 certificate indicates the presence of the - public key in the system keyring. The highlighted portion denotes the public - key ID. -</p> - -<p> - As the next step, replace the space with ‘#’ and pass it as - <code><public-key-id></code> in the kernel command line. For example, in the - above case, the following is passed in the place of - <code><public-key-id></code>: - <code>Android:#7e4333f9bba00adfe0ede979e28ed1920492b40f</code> -</p> - -<h3 id="recovery">Recovery</h3> - -<p> - The recovery RAM disk is now contained in the <code>boot.img</code> file. When - going into recovery, the bootloader <strong>cannot</strong> put the - <code>skip_initramfs</code> option on the kernel command line. -</p> -<p> - For non-A/B updates, the recovery partition contains the code used to apply - updates. A/B updates are applied by <code>update_engine</code> running in the regular - booted system image. There is still a recovery mode used to implement factory - data reset and sideloading of update packages, which is where the name - "recovery" came from. The code and data for recovery mode is stored in the - regular boot partition now, in a ramdisk. So to boot into the system image, - the bootloader tells the kernel to skip the ramdisk; otherwise we'll boot - into recovery mode. Recovery mode is small (and much of it was already on the - boot partition), so the boot partition doesn't increase in size.</p> - -<h3 id="build-variables">Build variables</h3> - -<p>To implement A/B updates, you need a new A/B-capable bootloader and have -`AB_OTA_UPDATER := true` in your board configuration and list the -partitions to which A/B applies.</p> -<h5>Must define for the A/B target:</h5> - -<ul> - <li><code>AB_OTA_UPDATER := true</code></li> - <li> - <code>AB_OTA_PARTITIONS := \</code><br/> - <code> boot \</code><br/> - <code> system \</code><br/> - <code> vendor</code><br/> - and other partitions updated through <code>update_engine</code> (radio, - bootloader, etc.) - </li> - <li> - <code>BOARD_BUILD_SYSTEM_ROOT_IMAGE := true</code> - </li> - <li><code>TARGET_NO_RECOVERY := true</code></li> - <li> - <code>BOARD_USES_RECOVERY_AS_BOOT := true</code> - </li> - <li> - <code>PRODUCT_PACKAGES += \</code><br/> - <code> update_engine \</code><br/> - <code> update_verifier</code> - </li> - </ul> -<p>For an example, see:<br> -<code><a href="https://android.googlesource.com/device/google/marlin/+/android-7.1.0_r1/device-common.mk">/device/google/marlin/+/android-7.1.0_r1/device-common.mk</a></code></p> - -<p>Optionally, conduct the post-install (but pre-reboot) dex2oat step described -within the <a href="#compilation">Compilation</a> section.</p> - -<h5>Optionally define for debug builds:</h5> - -<ul> - <li> - <code>PRODUCT_PACKAGES_DEBUG += update_engine_client</code> - </li> -</ul> - -<h5>Cannot define for the A/B target:</h5> +<p>The update process starts when an OTA package (referred to in code as a +<em>payload</em>) is available for downloading. Policies in the device may defer +the payload download and application based on battery level, user activity, +charging status, or other policies. In addition, because the update runs in the +background, users might not know an update is in progress. All of this means the +update process might be interrupted at any point due to policies, unexpected +reboots, or user actions.</p> + +<p>Optionally, metadata in the OTA package itself indicates the update can be +streamed; the same package can also be used for non-streaming installation. The +server may use the metadata to tell the client it's streaming so the client will +hand off the OTA to <code>update_engine</code> correctly. Device manufacturers +with their own server and client can enable streaming updates by ensuring the +server identifies the update is streaming (or assumes all updates are streaming) +and the client makes the correct call to <code>update_engine</code> for +streaming. Manufacturers can use the fact that the package is of the streaming +variant to send a flag to the client to trigger hand off to the framework side +as streaming.</p> + +<p>After a payload is available, the update process is as follows:</p> +<table> +<tr> +<th>Step</th> +<th>Activities</th> +</tr> +<tr> +<td>1</td> +<td>The current slot (or "source slot") is marked as successful (if not already +marked) with <code>markBootSuccessful()</code>.</td> +</tr> +<tr> +<td>2</td> +<td>The unused slot (or "target slot") is marked as unbootable by calling the +function <code>setSlotAsUnbootable()</code>. The current slot is always marked +as successful at the beginning of the update to prevent the bootloader from +falling back to the unused slot, which will soon have invalid data. If the +system has reached the point where it can start applying an update, the current +slot is marked as successful even if other major components are broken (such as +the UI in a crash loop) as it is possible to push new software to fix these +problems. +<br><br> +The update payload is an opaque blob with the instructions to update to the new +version. The update payload consists of the following: <ul> - <li><code>BOARD_RECOVERYIMAGE_PARTITION_SIZE</code></li> - <li><code>BOARD_CACHEIMAGE_PARTITION_SIZE</code></li> - <li><code>BOARD_CACHEIMAGE_FILE_SYSTEM_TYPE</code></li> +<li><em>Metadata</em>. A relatively small portion of the update payload, the +metadata contains a list of operations to produce and verify the new version on +the target slot. For example, an operation could decompress a certain blob and +write it to specific blocks in a target partition, or read from a source +partition, apply a binary patch, and write to certain blocks in a target +partition.</li> +<li><em>Extra data</em>. As the bulk of the update payload, the extra data +associated with the operations consists of the compressed blob or binary patch +in these examples.</li> </ul> +</td> +</tr> +<tr> +<td>3</td> +<td>The payload metadata is downloaded.</td> +</tr> +<tr> +<td>4</td> +<td>For each operation defined in the metadata, in order, the associated data +(if any) is downloaded to memory, the operation is applied, and the associated +memory is discarded.</td> +</tr> +<tr> +<td>5</td> +<td>The whole partitions are re-read and verified against the expected hash. +</td> +</tr> +<tr> +<td>6</td> +<td>The post-install step (if any) is run. In the case of an error during the +execution of any step, the update fails and is re-attempted with possibly a +different payload. If all the steps so far have succeeded, the update succeeds +and the last step is executed.</td> +</tr> +<tr> +<td>7</td> +<td>The <em>unused slot</em> is marked as active by calling +<code>setActiveBootSlot()</code>. Marking the unused slot as active doesn't mean +it will finish booting. The bootloader (or system itself) can switch the active +slot back if it doesn't read a successful state.</td> +</tr> +<tr> +<td>8</td> +<td>Post-installation (described below) involves running a program from the +"new update" version while still running in the old version. If defined in the +OTA package, this step is <strong>mandatory</strong> and the program must return +with exit code <code>0</code>; otherwise, the update fails.</td> +</tr> +</table> -<h3 id="partitions">Partitions</h3> -<p> - A/B devices do not need a recovery partition or cache partition because - Android no longer uses these partitions. The data partition is now used for - the downloaded OTA package, and the recovery image code is on the boot - partition. - All partitions that are A/B-ed should be named as follows (slots are always - named <code>a</code>, <code>b</code>, etc.): <code>boot_a</code>, - <code>boot_b</code>, <code>system_a</code>, <code>system_b</code>, - <code>vendor_a</code>, <code>vendor_b</code>. -</p> - -<p>For non-A/B updates, the cache partition was used to store downloaded OTA -packages and to stash blocks temporarily while applying updates. There was -never a good way to size the cache partition: how large it needed to be -depended on what updates you wanted to apply. The worst case would be a cache -partition as large as the system image. With A/B updates there's no need to -stash blocks (because you're always writing to a partition that isn't currently -used) and with streaming A/B there's no need to download the whole OTA package -before applying it.</p> - -<h3 id="fstab">Fstab</h3> - -<p> - The <code>slotselect</code> argument <strong>must</strong> be on the line for - the A/B-ed partitions. For example: -</p> - -<pre class="devsite-click-to-copy"> -<path-to-block-device>/vendor /vendor ext4 ro -wait,verify=<path-to-block-device>/metadata,slotselect -</pre> - -<p> - Please note that there should be no partition named <code>vendor</code> but - instead the partition <code>vendor_a</code> or <code>vendor_b</code> will be - selected and mounted on the <code>/vendor</code> mount point. -</p> - -<h3 id="kernel-slot-arguments">Kernel slot arguments</h3> - -<p> - The current slot suffix should be passed either through a specific DT node - (<code>/firmware/android/slot_suffix</code>) or through the - <code>androidboot.slot_suffix</code> command line argument. -</p> +<aside class="note"><strong>Note:</strong> Steps 3 and 4 take most of the update +time as they involve writing and downloading large amounts of data, and are +likely to be interrupted for reasons of policy or reboot.</aside> -<p>By default, fastboot will flash just slot 'a' on an A/B device, and set the -current slot to 'a'. An update package can contain images for slot 'b' too, in -which case they will also be flashed. A new '--slot' option lets you ask -fastboot to use slot 'b' instead of slot 'a', and the '--set-active' option -lets you set that slot as active too. There's also a new 'fastboot set_active' -command. See 'fastboot --help' for more details.</p> +<h3 id="post-installation">Post-installation</h3> -<p> - Optionally, if the bootloader implements fastboot, the following commands and - variables should be supported: -</p> +<p>For every partition where a post-install step is defined, +<code>update_engine</code> mounts the new partition into a specific location and +executes the program specified in the OTA relative to the mounted partition. For +example, if the post-install program is defined as +<code>usr/bin/postinstall</code> in the system partition, this partition from +the unused slot will be mounted in a fixed location (such as +<code>/postinstall_mount</code>) and the +<code>/postinstall_mount/usr/bin/postinstall</code> command is executed.</p> -<h4>Commands</h4> -<ul> - <li> - <code>set_active <slot></code> —Sets the current active slot to - the given slot. This must also clear the unbootable flag for that slot, and - reset the retry count to default values. - </li> -</ul> +<p>For post-installation to succeed, the old kernel must be able to:</p> -<h4>Variables</h4> <ul> - <li> - <code>has-slot:<partition-base-name-without-suffix></code> - —Returns “yes” if the given partition supports slots, “no” otherwise. - </li> - <li> - <code>current-slot</code> —Returns the slot suffix that will be booted from - next. - </li> - <li> - <code>slot-count</code> —Returns an integer representing the number of - available slots. Currently, two slots are supported so this value is - <code>2</code>. - </li> - <li> - <code>slot-successful:<slot-suffix></code> —Returns "yes" if the given - slot has been marked as successfully booting, "no" otherwise. - </li> - <li> - <code>slot-unbootable:<slot-suffix></code> —Returns “yes” if the given - slot is marked as unbootable, "no" otherwise. - </li> - <li> - <code>slot-retry-count:<slot suffix></code> —Number of retries remaining to - attempt to boot the given slot. - </li> - <li> - These variables should all appear under the following: - <code>fastboot getvar all</code> - </li> +<li><strong>Mount the new filesystem format</strong>. The filesystem type cannot +change unless there's support for it in the old kernel, including details such +as the compression algorithm used if using a compressed filesystem (i.e. +SquashFS).</li> +<li><strong>Understand the new partition's post-install program format</strong>. +If using an Executable and Linkable Format (ELF) binary, it should be compatible +with the old kernel (e.g. a 64-bit new program running on an old 32-bit kernel +if the architecture switched from 32- to 64-bit builds). Unless the loader +(<code>ld</code>) is instructed to use other paths or build a static binary, +libraries will be loaded from the old system image and not the new one.</li> </ul> -<h3 id="ota-package-generation">OTA package generation</h3> - -<p> - The <a href="/devices/tech/ota/tools.html">OTA package tools</a> - follow the same commands as the commands for non-A/B devices. The - <code>target_files.zip</code> file must be generated by defining the build - variables for the A/B target. The OTA package tools automatically identify and - generate packages in the format for the A/B updater. -</p> - -<p> - For example, use the following to generate a full OTA: -</p> - -<pre class="devsite-terminal devsite-click-to-copy"> -./build/tools/releasetools/ota_from_target_files \ - dist_output/tardis-target_files.zip ota_update.zip -</pre> - -<p> - Or, generate an incremental OTA: -</p> - -<pre class="devsite-terminal devsite-click-to-copy"> -./build/tools/releasetools/ota_from_target_files \ - -i PREVIOUS-tardis-target_files.zip \ - dist_output/tardis-target_files.zip incremental_ota_update.zip -</pre> - -<h2 id="configuration">Configuration</h2> - -<h3 id="config-partitions">Partitions</h3> - -<p> - The Update Engine can update any pair of A/B partitions defined in the same - disk. -</p> - -<p> - A pair of partitions has a common prefix (such as <code>system</code> or - <code>boot</code>) and per-slot suffix (such as <code>_a</code>). The list of - partitions for which the payload generator defines an update is configured by - the <code>AB_OTA_PARTITIONS</code> make variable. For example, if a pair of - partitions <code>bootloader_a</code> and <code>booloader_b</code> are included - (<code>_a</code> and <code>_b</code> are the slot suffixes), these partitions - can be updated by specifying the following on the product or board - configuration: -</p> - -<pre class="devsite-click-to-copy"> -AB_OTA_PARTITIONS := \ - boot \ - system \ - bootloader -</pre> - -<p> - All the partitions updated by the Update Engine must not be modified by the - rest of the system. During incremental or <em>delta</em> updates, the binary - data from the current slot is used to generate the data in the new slot. Any - modification may cause the new slot data to fail verification during the - update process, and therefore fail the update. -</p> - -<h3 id="post-install">Post-install</h3> - -<p> - The post-install step can be configured differently for each updated partition - using a set of key-value pairs. -</p> - -<p> - To run a program located at <code>/system/usr/bin/postinst</code> in a - new image, specify the path relative to the root of the filesystem in the - system partition. For example, <code>usr/bin/postinst</code> is - <code>system/usr/bin/postinst</code> (if not using a RAM disk). Additionally, - specify the filesystem type to pass to the <code>mount(2)</code> system call. - Add the following to the product or device <code>.mk</code> files (if - applicable): -</p> - -<pre class="devsite-click-to-copy"> -AB_OTA_POSTINSTALL_CONFIG += \ - RUN_POSTINSTALL_system=true \ - POSTINSTALL_PATH_system=usr/bin/postinst \ - FILESYSTEM_TYPE_system=ext4 -</pre> - -<h3 id="compilation">Compilation</h3> -<p>Minimally, you must compile ahead of time odex files for system_server and -its dependencies (because system_server isn't allowed to JIT for security -reasons); but anything else is optional.</p> - -<p>Compiling apps in the background for A/B updates requires the following two - additions to the product's device configuration (in the product's device.mk):</p> - -<ol> - <li>Include the native components in the build. This ensures the compilation - script and binaries are compiled and included in the system image. -<pre class="devsite-click-to-copy"> - # A/B OTA dexopt package - PRODUCT_PACKAGES += otapreopt_script -</pre> - </li> - <li>Connect the compilation script to <code>update_engine</code> such that it - is run as a post-install step. -<pre class="devsite-click-to-copy"> - # A/B OTA dexopt update_engine hookup - AB_OTA_POSTINSTALL_CONFIG += \ - RUN_POSTINSTALL_system=true \ - POSTINSTALL_PATH_system=system/bin/otapreopt_script \ - FILESYSTEM_TYPE_system=ext4 \ - POSTINSTALL_OPTIONAL_system=true - </pre> - </li> -</ol> - -<p>See <a href="/devices/tech/dalvik/configure.html#other_odex">First -boot installation of DEX_PREOPT files</a> to install the preopted files in the -unused second system partition.</p> - -<h2>Frequently asked questions </h2> +<p>For example, you could use a shell script as a post-install program +(interpreted by the old system's shell binary with a <code>#!</code> marker at +the top), then set up library paths from the new environment for executing a +more complex binary post-install program. Alternatively, you could run the +post-install step from a dedicated smaller partition to enable the filesystem +format in the main system partition to be updated without incurring backward +compatibility issues or stepping-stone updates; this would allow users to update +directly to the latest version from a factory image.</p> + +<p>The new post-install program is limited by the SELinux policies defined in +the old system. As such, the post-install step is suitable for performing tasks +required by design on a given device or other best-effort tasks (i.e. updating +the A/B-capable firmware or bootloader, preparing copies of databases for the +new version, etc.). The post-install step is <strong>not suitable</strong> for +one-off bug fixes before reboot that require unforeseen permissions.</p> + +<p>The selected post-install program runs in the <code>postinstall</code> +SELinux context. All the files in the new mounted partition will be tagged with +<code>postinstall_file</code>, regardless of what their attributes are after +rebooting into that new system. Changes to the SELinux attributes in the new +system won't impact the post-install step. If the post-install program needs +extra permissions, those must be added to the post-install context.</p> + +<h2 id=faq>Frequently asked questions</h2> <h3>Has Google used A/B OTAs on any devices?</h3> -<p>Yes. The marketing name for this feature is <em>seamless updates</em>. The -Pixel and Pixel XL phones from October 2016 shipped with A/B. Additionally, all -Chromebooks use the same <code>update_engine</code> implementation of A/B. The -necessary platform code implementation is public in Android 7.1 and later.</p> +<p>Yes. The marketing name for A/B updates is <em>seamless updates</em>. Pixel +and Pixel XL phones from October 2016 shipped with A/B, and all Chromebooks use +the same <code>update_engine</code> implementation of A/B. The necessary +platform code implementation is public in Android 7.1 and higher.</p> <h3>Why are A/B OTAs better?</h3> -<p>As the name of seamless updates implies, A/B OTAs provide a -better user experience when taking updates. Measurements from monthly -security updates show this feature has already proven a success: As of May -2017, 95% of Pixel owners are running the latest security update after a month -compared to 87% of Nexus users, and the Pixel users update sooner than Nexus -users would. Failures to update blocks during an OTA no longer result in a -device that won't boot; until the new system image has successfully booted, -Android retains the ability to fall back to the previous working system image.</p> +<p>A/B OTAs provide a better user experience when taking updates. Measurements +from monthly security updates show this feature has already proven a success: As +of May 2017, 95% of Pixel owners are running the latest security update after a +month compared to 87% of Nexus users, and Pixel users update sooner than Nexus +users. Failures to update blocks during an OTA no longer result in a device that +won't boot; until the new system image has successfully booted, Android retains +the ability to fall back to the previous working system image.</p> <h3>How did A/B affect the 2016 Pixel partition sizes?</h3> -<p>See the following table for the shipping A/B configuration versus the internally-tested non-A/B configuration:</p> +<p>The following table contains details on the shipping A/B configuration versus +the internally-tested non-A/B configuration:</p> <table> <tbody> <tr> <th>Pixel partition sizes</th> - <th>A/B</th> - <th>Non-A/B</th> + <th width="33%">A/B</th> + <th width="33%">Non-A/B</th> </tr> <tr> <td>Bootloader</td> @@ -907,34 +426,31 @@ Android retains the ability to fall back to the previous working system image.</ </tbody> </table> -<p>Therefore, A/B updates require an increase of only 320 MiB in flash. -Savings of 32MiB come from removing the recovery partition, while another -100MiB is preserved by removing the cache partition.</p> - -<p>This roughly balanced out the cost of the B partitions for the bootloader, the -boot partition, and the radio partition. The vendor partition doubled in size. -(This was the vast majority of the size increase.) Pixel's A/B system image is -half the size of the original non-A/B system image.</p> +<p>A/B updates require an increase of only 320 MiB in flash, with a savings of +32MiB from removing the recovery partition and another 100MiB preserved by +removing the cache partition. This balances the cost of the B partitions for +the bootloader, the boot partition, and the radio partition. The vendor +partition doubled in size (the vast majority of the size increase). Pixel's +A/B system image is half the size of the original non-A/B system image. +</p> -<p>So for Pixel, the A/B and non-A/B variants tested internally (only A/B -shipped), the space used differed by only 320MiB. On a 32GiB device, this is -just under 1%. For a 16GiB device this would be less than 2%, and for an 8GiB -device almost 4% (assuming all three devices had the same system image).</p> +<p>For the Pixel A/B and non-A/B variants tested internally (only A/B shipped), +the space used differed by only 320MiB. On a 32GiB device, this is just under +1%. For a 16GiB device this would be less than 2%, and for an 8GiB device almost +4% (assuming all three devices had the same system image).</p> <h3>Why didn't you use SquashFS?</h3> -<p>Android did experiment with SquashFS but wasn't able to achieve the -performance desired for a high-end device. Android doesn't use or recommend -SquashFS for handheld devices.</p> - -<p> - SquashFS provided about 50% size savings on the system partition, but the - overwhelming majority of the files that compressed well were the precompiled - .odex files. Those files had very high compression ratios (approaching 80%), - but the compression ratio for the rest of the system partition was much - lower.</p> +<p>We experimented with SquashFS but weren't able to achieve the performance +desired for a high-end device. We don't use or recommend SquashFS for handheld +devices.</p> -<p>And there were serious concerns about performance with SquashFS in N:</p> +<p>More specifically, SquashFS provided about 50% size savings on the system +partition, but the overwhelming majority of the files that compressed well were +the precompiled .odex files. Those files had very high compression ratios +(approaching 80%), but the compression ratio for the rest of the system +partition was much lower. In addition, SquashFS in Android 7.0 raised the +following performance concerns:</p> <ul> <li>Pixel has very fast flash compared to earlier devices but not a huge @@ -943,32 +459,32 @@ SquashFS for handheld devices.</p> <li>I/O changes that perform well on an artificial benchmark run on an unloaded system sometimes don't work well on real-world use cases under real-world load (such as crypto on Nexus 6).</li> - <li>Benchmarking showed 85% regressions in some places. As SquashFS matures - and adds features to reduce CPU impact (such as a whitelist of - commonly-accessed files that shouldn't be compressed), the Android team - will continue to evaluate SquashFS and then offer recommendations to - device manufacturers.</li> -</ul> + <li>Benchmarking showed 85% regressions in some places.</li> + </ul> + +<p>As SquashFS matures and adds features to reduce CPU impact (such as a +whitelist of commonly-accessed files that shouldn't be compressed), we will +continue to evaluate it and offer recommendations to device manufacturers.</p> <h3>How did you halve the size of the system partition without SquashFS?</h3> <p>Applications are stored in .apk files, which are actually ZIP archives. Each .apk file has inside it one or more .dex files containing portable Dalvik -bytecode. An .odex file (optimized .dex) lives separately from the apk file -and can contain machine code specific to the device. If an odex file is +bytecode. An .odex file (optimized .dex) lives separately from the .apk file +and can contain machine code specific to the device. If an .odex file is available, Android can run applications at ahead-of-time compiled speeds without having to wait for the code to be compiled each time the application is -launched. An odex file isn't strictly necessary: Android can actually run the +launched. An .odex file isn't strictly necessary: Android can actually run the .dex code directly via interpretation or Just-In-Time (JIT) compilation, but an -odex file provides the best combination of launch speed and run-time speed if +.odex file provides the best combination of launch speed and run-time speed if space is available.</p> -<p>If you look at the installed-files.txt from a Nexus 6P N MR1 build, where -the total system image size is 2628MiB (2755792836 bytes), the breakdown of the -largest contributors to overall system image size by file type looks like -this:</p> +<p>Example: For the installed-files.txt from a Nexus 6P running Android 7.1 with +a total system image size of 2628MiB (2755792836 bytes), the breakdown of the +largest contributors to overall system image size by file type is as follows: +</p> -<table class="style0"> +<table> <tbody> <tr> <td>.odex</td> @@ -1004,75 +520,72 @@ this:</p> </table> <p>These figures are similar for other devices too, so on Nexus/Pixel -devices, odex files take up roughly half of the system partition. This meant -that we could continue to use ext4 but write the odex files to the B partition +devices, .odex files take up approximately half the system partition. This meant +we could continue to use ext4 but write the .odex files to the B partition at the factory and then copy them to <code>/data</code> on first boot. The -actual storage used on Marlin/Sailfish with ext4 A/B is identical to SquashFS -A/B, because if we'd used SquashFS we would have shipped the preopted odex -files on system_a instead of system_b.</p> - -<h3>Doesn't copying odex files to /data mean that the space saved on system - is lost on data?</h3> - -<p>Not exactly. On Pixel, most of the space taken by odex files are for apps. -These typically exist on <code>/data</code> anyway. Since apps take Google -Play updates, the apk and odex files on the system image are unused for -most of the life of the device. So these files can be excluded entirely and -replaced by small profile-driven odex files when the user actually uses each -app and requiring no space for apps the user doesn't use. (This was discussed - at Google I/O 2016 in <a - href="https://www.youtube.com/watch?v=fwMM6g7wpQ8">The Evolution of Art</a>.</p> - -<p>These are the key reasons why comparison is difficult:</p> +actual storage used with ext4 A/B is identical to SquashFS A/B, because if we +had used SquashFS we would have shipped the preopted .odex files on system_a +instead of system_b.</p> + +<h3>Doesn't copying .odex files to /data mean the space saved on /system is +lost on /data?</h3> + +<p>Not exactly. On Pixel, most of the space taken by .odex files is for apps, +which typically exist on <code>/data</code>. These apps take Google Play +updates, so the .apk and .odex files on the system image are unused for most of +the life of the device. Such files can be excluded entirely and replaced by +small, profile-driven .odex files when the user actually uses each app (thus +requiring no space for apps the user doesn't use). For details, refer to the +Google I/O 2016 talk <a href="https://www.youtube.com/watch?v=fwMM6g7wpQ8">The +Evolution of Art</a>.</p> + +<p>The comparison is difficult for a few key reasons:</p> <ul> - <li>Apps updated by Google Play have always had their odex files on <code>/data</code> as soon as they receive their first update.</li> -<li>Apps that the user doesn't run don't need an odex file at all.</li> -<li>Profile-driven compilation generates smaller odex files than ahead-of-time - compilation (because the former optimizes only performance-critical - code).</li> +<li>Apps updated by Google Play have always had their .odex files on +<code>/data</code> as soon as they receive their first update.</li> +<li>Apps the user doesn't run don't need an .odex file at all.</li> +<li>Profile-driven compilation generates smaller .odex files than ahead-of-time +compilation (because the former optimizes only performance-critical code).</li> </ul> -<p>The <a href="/devices/tech/dalvik/configure.html">Configuring ART</a> -documentation explains the tuning options available to the OEM.</p> +<p>For details on the tuning options available to OEMs, see +<a href="/devices/tech/dalvik/configure.html">Configuring ART</a>.</p> -<h3>Aren't there two copies of the odex files on /data?</h3> +<h3>Aren't there two copies of the .odex files on /data?</h3> -<p>It's a little more complicated than that...<br> -After the new system image has been written, the new version of dex2oat is run -against the new dex files to generate the new odex files. This happens while -the old system is still running, and so the old and new odex files are both on -<code>/data</code> at the same time.</p> +<p>It's a little more complicated ... After the new system image has been +written, the new version of dex2oat is run against the new .dex files to +generate the new .odex files. This occurs while the old system is still running, +so the old and new .odex files are both on <code>/data</code> at the same time. +</p> -<p>The code in OtaDexoptService (<code><a - href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/OtaDexoptService.java#200">frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/OtaDexoptService.java#200</a></code>) +<p>The code in OtaDexoptService +(<code><a href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/OtaDexoptService.java#200" class="external">frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/OtaDexoptService.java#200</a></code>) calls <code>getAvailableSpace</code> before optimizing each package to avoid over-filling <code>/data</code>. Note that <em>available</em> here is still conservative: it's the amount of space left <em>before</em> hitting the usual system low space threshold (measured as both a percentage and a byte count). So -if <code>/data</code> is full, there won't be two copies of every odex file.<br> -The same code also has a BULK_DELETE_THRESHOLD: if the device gets that close -to filling the available space (as just described), the odex files belonging to +if <code>/data</code> is full, there won't be two copies of every .odex file. +The same code also has a BULK_DELETE_THRESHOLD: If the device gets that close +to filling the available space (as just described), the .odex files belonging to apps that aren't used are removed. That's another case without two copies of -every odex file.</p> - -<p>In the worst case where <code>/data</code> is completely full, the update waits -until the device has rebooted into the new system and no longer need the old -system's odex files.</p> - -<p>The PackageManager handles this: (<code><a - href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/PackageManagerService.java#7215">frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/PackageManagerService.java#7215</a></code>).</p> - -<p>Once the new system has successfully booted, <code>installd</code> (<code><a - href="https://android.googlesource.com/platform/frameworks/native/+/nougat-mr1-release/cmds/installd/commands.cpp#2192">frameworks/native/+/nougat-mr1-release/cmds/installd/commands.cpp#2192</a></code>) -can remove the odex files that were used by the old system, returning the +every .odex file.</p> + +<p>In the worst case where <code>/data</code> is completely full, the update +waits until the device has rebooted into the new system and no longer needs the +old system's .odex files. The PackageManager handles this: +(<code><a href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/PackageManagerService.java#7215" class="external">frameworks/base/+/nougat-mr1-release/services/core/java/com/android/server/pm/PackageManagerService.java#7215</a></code>). After the new system has +successfully booted, <code>installd</code> +(<code><a href="https://android.googlesource.com/platform/frameworks/native/+/nougat-mr1-release/cmds/installd/commands.cpp#2192" class="external">frameworks/native/+/nougat-mr1-release/cmds/installd/commands.cpp#2192</a></code>) +can remove the .odex files that were used by the old system, returning the device back to the steady state where there's only one copy.</p> -<p>So to return to the original question: it is possible that -<code>/data</code> contains two copies of all the odex files, but (a) only -temporarily and (b) only if you had plenty of free space on <code>/data</code> -anyway. Except during an update, there's only one copy. And as part of ART's -general robustness features, it will never fill <code>/data</code> with odex -files anyway (because that would be a problem on a non-A/B system too).</p> +<p>So, while it is possible that <code>/data</code> contains two copies of all +the .odex files, (a) this is temporary and (b) only occurs if you had plenty of +free space on <code>/data</code> anyway. Except during an update, there's only +one copy. And as part of ART's general robustness features, it will never fill +<code>/data</code> with .odex files anyway (because that would be a problem on a +non-A/B system too).</p> <h3>Doesn't all this writing/copying increase flash wear?</h3> @@ -1083,31 +596,30 @@ flash wear rates should be similar.</p> <h3>Does flashing two system partitions increase factory flashing time?</h3> -<p>Pixel didn't increase in system image size (it merely divided the space -across two partitions). So no, factory flashing time did not grow.</p> +<p>No. Pixel didn't increase in system image size (it merely divided the space +across two partitions).</p> -<h3>Doesn't keeping odex files on B make rebooting after factory data reset slow?</h3> +<h3>Doesn't keeping .odex files on B make rebooting after factory data reset +slow?</h3> -<p>Yes. If you've actually used a device and taken an OTA and then perform a -factory data reset, the first reboot will be slower than it would otherwise be -(taking 1m40s vs 40s on a Pixel XL just tested) because the odex files will -have been lost from B after the first OTA and so can't be copied to -<code>/data</code>. That's the trade-off.</p> +<p>Yes. If you've actually used a device, taken an OTA, and performed a factory +data reset, the first reboot will be slower than it would otherwise be (1m40s vs +40s on a Pixel XL) because the .odex files will have been lost from B after the +first OTA and so can't be copied to <code>/data</code>. That's the trade-off.</p> -<p>Factory data reset should be a rare operation - certainly -compared to regular boot - so the time taken is less important. (This doesn't -affect users or reviewers who get their device from the factory, because in -that case the B partition is available.) Thanks to the JIT compiler, we also -don't need to recompile <em>everything</em>, so it's not as bad as you might -think. It's also possible to mark apps as requiring ahead-of-time compilation -using <code>coreApp="true"</code> in the manifest: (<code><a - href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/packages/SystemUI/AndroidManifest.xml#23">frameworks/base/+/nougat-mr1-release/packages/SystemUI/AndroidManifest.xml#23</a></code>)</p> +<p>Factory data reset should be a rare operation when compared to regular boot +so the time taken is less important. (This doesn't affect users or reviewers who +get their device from the factory, because in that case the B partition is +available.) Use of the JIT compiler means we don't need to recompile +<em>everything</em>, so it's not as bad as you might think. It's also possible +to mark apps as requiring ahead-of-time compilation using +<code>coreApp="true"</code> in the manifest: +(<code><a href="https://android.googlesource.com/platform/frameworks/base/+/nougat-mr1-release/packages/SystemUI/AndroidManifest.xml#23" class="external">frameworks/base/+/nougat-mr1-release/packages/SystemUI/AndroidManifest.xml#23</a></code>). +This is currently used by <code>system_server</code> because it's not allowed to +JIT for security reasons.</p> -<p>This is currently used by system_server because it's not allowed to JIT for -security reasons.</p> - -<h3>Doesn't keeping odex files on /data rather than /system make rebooting - after an OTA slow?</h3> +<h3>Doesn't keeping .odex files on /data rather than /system make rebooting +after an OTA slow?</h3> <p>No. As explained above, the new dex2oat is run while the old system image is still running to generate the files that will be needed by the new system. The @@ -1117,13 +629,13 @@ update isn't considered available until that work has been done.</p> <p>32GiB works well as it was proven on Pixel, and 320MiB out of 16GiB means a reduction of 2%. Similarly, 320MiB out of 8GiB a reduction of 4%. Obviously -A/B, would not be the recommended choice on devices with 4GiB, as the 320MiB +A/B would not be the recommended choice on devices with 4GiB, as the 320MiB overhead is almost 10% of the total available space.</p> <h3>Does AVB2.0 require A/B OTAs?</h3> -<p>No. Android <a href="/security/verifiedboot/">Verified Boot</a> has always required block-based -updates, but not necessarily A/B updates.</p> +<p>No. Android <a href="/security/verifiedboot/">Verified Boot</a> has always +required block-based updates, but not necessarily A/B updates.</p> <h3>Do A/B OTAs require AVB2.0?</h3> @@ -1141,18 +653,18 @@ actually successfully booted the new image, rollback protection doesn't consider it to be the current system image.</p> <h3>If you're installing an update while the system is running, isn't that - slow?</h3> +slow?</h3> <p>With non-A/B updates, the aim is to install the update as quickly as possible because the user is waiting and unable to use their device while the update is applied. With A/B updates, the opposite is true; because the user is -still using their device, as little impact as possible is the goal, so the update -is deliberately slow. Android also (via logic in the Java system update client, in -Google’s case GmsCore, the core package provided by GMS) try to choose a time -when the users aren't using their devices at all. The platform supports -pausing/resuming the update, and the client can use that to pause the update if -the user starts to use the device and resume it when the device is idle -again.</p> +still using their device, as little impact as possible is the goal, so the +update is deliberately slow. Via logic in the Java system update client (which +for Google is GmsCore, the core package provided by GMS), Android also attempts +to choose a time when the users aren't using their devices at all. The platform +supports pausing/resuming the update, and the client can use that to pause the +update if the user starts to use the device and resume it when the device is +idle again.</p> <p>There are two phases while taking an OTA, shown clearly in the UI as <em>Step 1 of 2</em> and <em>Step 2 of 2</em> under the progress bar. Step 1 @@ -1191,22 +703,18 @@ currently running system. The update can simply be retried later.</p> <p>In Google's A/B implementation, the platform APIs and <code>update_engine</code> provide the mechanism while GmsCore provides the -policy. That is, the platform knows <em>how</em> to apply an A/B update, and -all that code is in AOSP (as mentioned above); but it's GmsCore that decides +policy. That is, the platform knows <em>how</em> to apply an A/B update and all +that code is in AOSP (as mentioned above); but it's GmsCore that decides <em>what</em> and <em>when</em> to apply.</p> -<p>If you’re not using GmsCore, you can write your own replacement using the same -platform APIs. The platform Java API for controlling <code>update_engine</code> -is <code>android.os.UpdateEngine</code>:</p> -<code><a class="external-link" target="_blank" - href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/os/UpdateEngine.java">frameworks/base/core/java/android/os/UpdateEngine.java</a></code> - -<p>Callers can provide an <code>UpdateEngineCallback</code> to be notified of -status updates:</p> -<code><a class="external-link" target="_blank" - href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/os/UpdateEngineCallback.java">frameworks/base/+/master/core/java/android/os/UpdateEngineCallback.java</a></code> - -<p>See the reference files for the core classes to use the interface.</p> +<p>If you’re not using GmsCore, you can write your own replacement using the +same platform APIs. The platform Java API for controlling +<code>update_engine</code> is <code>android.os.UpdateEngine</code>: +<code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/os/UpdateEngine.java" class="external-link">frameworks/base/core/java/android/os/UpdateEngine.java</a></code>. +Callers can provide an <code>UpdateEngineCallback</code> to be notified of status +updates: +<code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/os/UpdateEngineCallback.java" class="external-link">frameworks/base/+/master/core/java/android/os/UpdateEngineCallback.java</a></code>. +Refer to the reference files for the core classes to use the interface.</p> <h3>Which systems on a chip (SoCs) support A/B?</h3> @@ -1215,8 +723,8 @@ status updates:</p> <tbody> <tr> <td></td> -<td><strong>N Release</strong></td> -<td><strong>OC Release</strong></td> +<td><strong>Android 7.x Release</strong></td> +<td><strong>Android 8.x Release</strong></td> </tr> <tr> <td><strong>Qualcomm</strong></td> @@ -1231,8 +739,8 @@ status updates:</p> </tbody> </table> -<p>Please check with your SoC contacts for more details on their schedules. If -there are SoCs not listed here, please reach out your SoC directly. </p> +<p>For details on schedules, check with your SoC contacts. For SoCs not listed +above, reach out to your SoC directly.</p> </body> </html> diff --git a/en/devices/tech/ota/images/ab-updates-state-machine.png b/en/devices/tech/ota/images/ab-updates-state-machine.png Binary files differindex a125efdc..04b7952c 100644 --- a/en/devices/tech/ota/images/ab-updates-state-machine.png +++ b/en/devices/tech/ota/images/ab-updates-state-machine.png diff --git a/en/devices/tech/ota/reduce_size.html b/en/devices/tech/ota/reduce_size.html index 227c40df..16957626 100644 --- a/en/devices/tech/ota/reduce_size.html +++ b/en/devices/tech/ota/reduce_size.html @@ -22,154 +22,183 @@ --> -<p>This page describes build changes added to AOSP to reduce unnecessary file changes between - builds. Device implementers who maintain their own build system can use this information as a - guide for reducing over-the-air (OTA) update size.</p> - -<p>Android OTAs occasionally contain changed files that do not correspond to code changes but are - instead artifacts of the build system. This can occurs when the same code built at different - times, from different directories, or on different machines, produces a large number of changed - files. These excess files not only increase the size of an OTA, but make it difficult to - determine which code is changed in the OTA.</p> - -<p>To make the contents of an OTA more transparent, AOSP includes build system changes designed to - reduce OTA size by eliminating unnecessary file changes between builds. The aim is to reduce - the size of OTAs to include only the files that relate to the patches contained in the OTA. AOSP - also includes a <a href="#the_build_diff_tool">build diff tool</a> that filters out common - build-related file changes and provides a cleaner build file diff.</p> - -<p>The build system can create unnecessary file diffs in several ways. The following sections - discuss some of these issues and solutions, providing examples of fixes in AOSP when possible).</p> +<p>This page describes build changes added to AOSP to reduce unnecessary file +changes between builds. Device implementers who maintain their own build system +can use this information as a guide for reducing over-the-air (OTA) update size. +</p> + +<p>Android OTAs occasionally contain changed files that do not correspond to +code changes but are instead artifacts of the build system. This can occur when +the same code built at different times, from different directories, or on +different machines produces a large number of changed files. These excess files +not only increase the size of an OTA, but make it difficult to determine which +code is changed in the OTA.</p> + +<p>To make the contents of an OTA more transparent, AOSP includes build system +changes designed to reduce OTA size by eliminating unnecessary file changes +between builds. The aim is to reduce the size of OTAs to include only the files +that relate to the patches contained in the OTA. AOSP also includes a +<a href="#the_build_diff_tool">build diff tool</a>, which filters out common +build-related file changes and provides a cleaner build file diff, and a +<a href="#block-mapping-tool">block mapping tool</a>, which helps you keep block +allocation consistent.</p> + +<p>The build system can create unnecessary file diffs in several ways. The +following sections discuss some of these issues and solutions, providing +examples of fixes in AOSP when possible.</p> <h2 id=file_order>File order</h2> -<p><strong>Problem</strong>: Filesystems don’t guarantee a file order when asked for a list of files - in a directory, though it’s commonly the same for the same checkout. Tools such as <code>ls</code> - sort the results by default, but the wildcard function used by commands such as <code>find</code> - and <code>make</code> do not. Before using these tools, you must sort the outputs.</p> +<p><strong>Problem</strong>: Filesystems don’t guarantee a file order when asked +for a list of files in a directory, though it’s commonly the same for the same +checkout. Tools such as <code>ls</code> sort the results by default, but the +wildcard function used by commands such as <code>find</code> and +<code>make</code> do not. Before using these tools, you must sort the outputs. +</p> -<p><strong>Solution</strong>: Users of tools such as <code>find</code> and <code>make</code> with - wildcard must sort the output of these commands before using them. Use of - <code>$(wildcard )</code> or <code>$(shell find )</code> in Android.mk files should also be - sorted. Some tools, such as Java, do sort inputs so verify sorting is necessary.</p> +<p><strong>Solution</strong>: Users of tools such as <code>find</code> and +<code>make</code> with wildcard must sort the output of these commands before +using them. Use of <code>$(wildcard)</code> or <code>$(shell find)</code> in +<code>Android.mk</code> files should also be sorted. Some tools, such as Java, +do sort inputs so first verify sorting is necessary.</p> -<p><strong>Examples:</strong> Many instances were fixed in the core build system using the builtin - <code>all-*-files-under</code> macro, which includes <code>all-cpp-files-under</code> (as several - definitions were spread out in other makefiles). For details, refer to the following CLs:</p> +<p><strong>Examples:</strong> Many instances were fixed in the core build system +using the builtin <code>all-*-files-under</code> macro, which includes +<code>all-cpp-files-under</code> (as several definitions were spread out in +other makefiles). For details, refer to the following CLs:</p> <ul> - <li><a href="https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f">https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f</a> - <li><a href="https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410">https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410</a> - <li><a href="https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653">https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653</a> - <li><a href="https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c">https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c</a> + <li><a href="https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f" class="external">https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f</a> + <li><a href="https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410" class="external">https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410</a> + <li><a href="https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653" class="external">https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653</a> + <li><a href="https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c" class="external">https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c</a> </ul> <h2 id=build_directory>Build directory</h2> -<p><strong>Problem:</strong> Changing the directory in which things are built can cause the binaries - to be different. Most paths in the Android build are relative paths so <code>__FILE__</code> in - C/C++ isn’t a problem. However, the debug symbols encode the full pathname by default, and the - <code>.note.gnu.build-id</code> is generated from hashing the pre-stripped binary, so it will - change if the debug symbols change.</p> +<p><strong>Problem:</strong> Changing the directory in which things are built +can cause the binaries to be different. Most paths in the Android build are +relative paths so <code>__FILE__</code> in C/C++ isn’t a problem. However, the +debug symbols encode the full pathname by default, and the +<code>.note.gnu.build-id</code> is generated from hashing the pre-stripped +binary, so it will change if the debug symbols change.</p> -<p><strong>Solution:</strong> AOSP now makes debug paths relative. For details, refer to CL: - <a href="https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02">https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02</a>.</p> +<p><strong>Solution:</strong> AOSP now makes debug paths relative. For details, +refer to CL: <a href="https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02" class="external">https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02</a>. +</p> <h2 id=timestamps>Timestamps</h2> -<p><strong>Problem:</strong> Timestamps in the build output result in unnecessary file changes. This - is likely to happen in the following locations:</p> +<p><strong>Problem:</strong> Timestamps in the build output result in +unnecessary file changes. This is likely to happen in the following +locations:</p> <ul> - <li><code> __DATE__/__TIME__/__TIMESTAMP__ </code> macros in C or C++ code.</li> + <li><code>__DATE__/__TIME__/__TIMESTAMP__</code> macros in C or C++ code.</li> <li>Timestamps embedded in zip-based archives.</li> </ul> -<p><strong>Solutions/Examples:</strong> To remove timestamps from the build output, use the - instructions in the sections below.</p> +<p><strong>Solutions/Examples:</strong> To remove timestamps from the build +output, use the instructions in the sections below.</p> <h3 id=date_time_timestamp_in_c_c>__DATE__/__TIME__/__TIMESTAMP__ in C/C++</h3> -<p>These macros always produce different outputs for different builds, so they shouldn’t be used. - Here are a few options on how to eliminate these macros:</p> +<p>These macros always produce different outputs for different builds, so they +shouldn’t be used. Here are a few options on how to eliminate these macros:</p> <ul> <li>Just remove them, they often aren’t necessary. For an example, refer to - <a href="https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f">https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f</a></li> - <li>To uniquely identify the running binary, read the build-id from the ELF header.</li> - <li>To know when the OS was built, read the <code>ro.build.date</code> (should work for everything - except incremental builds, which may not update this date). For an example, refer to - <a href="https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84">https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84</a></li> + <a href="https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f" class="external">https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f</a>.</li> + <li>To uniquely identify the running binary, read the build-id from the ELF + header.</li> + <li>To know when the OS was built, read the <code>ro.build.date</code> (should + work for everything except incremental builds, which may not update this + date). For an example, refer to + <a href="https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84" class="external">https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84</a>.</li> </ul> - <p class="note"><strong>Note:</strong>We turned on <code>-Werror=date-time</code> so using - timestamps is a build error.</p> +<aside class="note"><strong>Note:</strong> Android 7.0 turned on +<code>-Werror=date-time</code>, so using timestamps is a build error.</aside> -<h3 id=embedded_timestamps_in_zip-based_archives_zip_jar>Embedded timestamps in archives (zip, jar)</h3> +<h3 id=embedded_timestamps_in_zip-based_archives_zip_jar>Embedded timestamps in +archives (zip, jar)</h3> -<p>We fixed the problem of embedded timestamps in zip archives by adding <code>-X</code> to all uses - of the <code>zip</code> command, so the UID/GID of the builder and the extended Unix timestamp are - not embedded in the zip file.</p> +<p>Android 7.0 fixed the problem of embedded timestamps in zip archives by +adding <code>-X</code> to all uses of the <code>zip</code> command, so the +UID/GID of the builder and the extended Unix timestamp are not embedded in the +zip file.</p> -<p>A new tool, <code>ziptime</code> (located in <code> - <a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/">/platform/build/+/master/tools/ziptime/</a></code>) - resets the normal timestamps in the zip headers. For details, refer to the - <a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/README.txt">README - file</a>.</p> +<p>A new tool, <code>ziptime</code> (located in +<code><a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/" class="external">/platform/build/+/master/tools/ziptime/</a></code>) +resets the normal timestamps in the zip headers. For details, refer to the +<a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/README.txt" class="external">README +file</a>.</p> -<p>The <code>signapk</code> tool sets timestamps for the APK files that may vary depending on the - server timezone. For details, refer to the CL - <a href="https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028">https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028</a>.</p> +<p>The <code>signapk</code> tool sets timestamps for the APK files that may vary +depending on the server timezone. For details, refer to the CL +<a href="https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028" class="external">https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028</a>. +</p> <h2 id=version_strings>Version strings</h2> -<p><strong>Problem:</strong> APK version strings often had the BUILD_NUMBER appended to the - hardcoded version. Even if nothing else changed in the APK, the APK would still be different.</p> +<p><strong>Problem:</strong> APK version strings often had the +<code>BUILD_NUMBER</code> appended to the hardcoded version. Even if nothing +else changed in the APK, the APK would still be different.</p> -<p><strong>Solution:</strong> Remove the build number from the APK version string.</p> +<p><strong>Solution:</strong> Remove the build number from the APK version +string.</p> <p><strong>Examples:</strong></p> <ul> - <li><a href="https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27">https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27</a></li> - <li><a href="https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c">https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c</a></li> + <li><a href="https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27" class="external">https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27</a></li> + <li><a href="https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c" class="external">https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c</a></li> </ul> <h2 id=consistent_build_tools>Consistent build tools</h2> -<p><strong>Problem:</strong> Tools that generate installed files must be consistent (the same input - should always produce the same output).</p> +<p><strong>Problem:</strong> Tools that generate installed files must be +consistent (the same input should always produce the same output).</p> -<p><strong>Solutions/Examples:</strong> Changes were required in the following build tools:</p> +<p><strong>Solutions/Examples:</strong> Changes were required in the following +build tools:</p> <ul> - <li><strong>NOTICE file creator</strong>. The NOTICE file creator needed the changes. Refer to CL: - <a href="https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64">https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64</a></li> - <li><strong>Java Android Compiler Kit (Jack)</strong>. The Jack toolchain required an update to - handle an occasional change in generated constructor ordering. Refer to CL: - <a href="https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b">https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b</a></li> - <li><strong>ART AOT compiler (dex2oat)</strong>. The ART compiler binary required an update to - create a deterministic image. Refer to CL: - <a href="https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9">https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9</a></li> - <li><strong>The libpac.so file (V8)</strong>Every build creates a different - <code>/system/lib/libpac.so</code> file because the V8 snapshot changes for each build. The - solution is to remove the snapshot. Refer to CL: - <a href="https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29">https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29</a></li> - <li><strong>Application pre-dexopt’d (.odex) files</strong>. The pre-dexopt’d (.odex) files - contained uninitialized padding on 64-bit systems. Refer to CL: - <a href="https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029">https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029</a></li> + <li><strong>NOTICE file creator</strong>. The NOTICE file creator needed the + changes. Refer to CL: + <a href="https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64" class="external">https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64</a>. + </li> + <li><strong>Java Android Compiler Kit (Jack)</strong>. The Jack toolchain + required an update to handle an occasional change in generated constructor + ordering. Refer to CL: + <a href="https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b" class="external">https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b</a>. + </li> + <li><strong>ART AOT compiler (dex2oat)</strong>. The ART compiler binary + required an update to create a deterministic image. Refer to CL: + <a href="https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9" class="external">https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9</a>. + </li> + <li><strong>The libpac.so file (V8)</strong>. Every build creates a different + <code>/system/lib/libpac.so</code> file because the V8 snapshot changes for + each build. The solution is to remove the snapshot. Refer to CL: + <a href="https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29" class="external">https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29</a>. + </li> + <li><strong>Application pre-dexopt’d (.odex) files</strong>. The pre-dexopt’d + (.odex) files contained uninitialized padding on 64-bit systems. Refer to CL: + <a href="https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029" class="external">https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029</a>. + </li> </ul> <h2 id=the_build_diff_tool>Using the build diff tool</h2> -<p>For cases where it is not possible to eliminate build-related file changes, we supply a build - diff tool, - <code><a href="https://android.googlesource.com/platform/build/+/master/tools/releasetools/target_files_diff.py">target_files_diff.py</a></code> - for use in comparing two file packages. This tool performs a recursive diff between two builds, - excluding common build-related file changes, such as:</p> +<p>For cases where it is not possible to eliminate build-related file changes, +AOSP includes a build diff tool, +<code><a href="https://android.googlesource.com/platform/build/+/master/tools/releasetools/target_files_diff.py" class="external">target_files_diff.py</a></code> +for use in comparing two file packages. This tool performs a recursive diff +between two builds, excluding common build-related file changes, such as:</p> <ul> - <li>Expected changes in the build output (for example, due to a build number change).</li> + <li>Expected changes in the build output (for example, due to a build number + change).</li> <li>Changes due to known issues in the current build system.</li> </ul> @@ -179,8 +208,34 @@ target_files_diff.py dir1 dir2 </pre> -<p><code>dir1</code> and <code>dir2</code> are base directories that contain the extracted target - files for each build.</p> +<p><code>dir1</code> and <code>dir2</code> are base directories that contain the +extracted target files for each build.</p> + +<h2 id=block-mapping-tool>Keeping block allocation consistent</h2> +<p>In an non-A/B OTA, one of the factors that contribute to the time is block +moves. For a given file, although its contents remain the same between two +builds, the actual blocks that hold the data might have changed. As a result, +the updater performs unnecessarily I/O to move the blocks around during an OTA. +</p> + +<p>To address this issue, in Android 7.0 we extended the +<code>make_ext4fs</code> tool that tries to keep the block allocation consistent +across builds. <code>make_ext4fs</code> accepts an optional +<code>-d base_fs</code> flag that attempts to allocate files to the same blocks +when generating an <code>ext4</code> image. You can extract the block mapping +files (i.e. the <code>base_fs</code> map files) from a previous build's target +files zip file (<code>IMAGES/system.map</code> and +<code>IMAGES/vendor.map</code>). The <code>base_fs</code> files can then be +checked in and specified via <code>PRODUCT_SYSTEM_BASE_FS_PATH</code> and +<code>PRODUCT_VENDOR_BASE_FS_PATH</code>. For example,</p> + +<pre class="devsite-click-to-copy"> + PRODUCT_SYSTEM_BASE_FS_PATH := path/to/base_fs_files/base_system.map + PRODUCT_VENDOR_BASE_FS_PATH := path/to/base_fs_files/base_vendor.map +</pre> + +<p>While this doesn’t help reduce the overall OTA package size, it does +improve OTA performance by reducing the amount of I/O.</p> </body> </html> |
