diff options
author | Android Partner Docs <noreply@android.com> | 2017-08-22 10:41:24 -0700 |
---|---|---|
committer | Clay Murphy <claym@google.com> | 2017-08-22 15:01:44 -0700 |
commit | f16c42333aa6b2de30a344dd68246d4a33d93e7d (patch) | |
tree | 311af599312cacb21c888aeae828cae59b0d64a1 /en/devices/architecture/kernel | |
parent | 04426e67ca3ee557a0083f9b3c6ba789021bd7a0 (diff) | |
download | source.android.com-f16c42333aa6b2de30a344dd68246d4a33d93e7d.tar.gz |
Docs: Changes to source.android.com
- 166080694 Devsite localized content from translation request a3d5a7... by Android Partner Docs <noreply@android.com>
- 166079245 Remove duplicate TOC entry to oob-users.html. by mheco <mheco@google.com>
- 166002955 Update builds for Oreo by Android Partner Docs <noreply@android.com>
- 165977566 Fixing bad conversion by hvm <hvm@google.com>
- 165977199 Edit links to point to public source files in AOSP. by cqn <cqn@google.com>
- 165962883 Add codename to CTS downloads page. by gdimino <gdimino@google.com>
- 165955117 Integration of O branch into mainline. by gdimino <gdimino@google.com>
- 165638251 Update July public Android security bulletin to remove QC... by Android Partner Docs <noreply@android.com>
- 165638198 Update June public Android security bulletin to remove QC... by Android Partner Docs <noreply@android.com>
- 165638174 Update May public Android security bulletin to remove QC ... by Android Partner Docs <noreply@android.com>
- 165638096 Update April public Android security bulletin to remove Q... by Android Partner Docs <noreply@android.com>
- 165528993 Update to Keymaster 2 and remove requirements language by daroberts <daroberts@google.com>
- 165511119 Add Bluetooth verification / debug information by cqn <cqn@google.com>
- 165491345 Fixed link broken by file rename. by cqn <cqn@google.com>
- 165381648 Fixed broken image paths and renamed HCI Requirements file. by cqn <cqn@google.com>
- 165365185 Created high-level Bluetooth directory and added HTML ver... by cqn <cqn@google.com>
- 165335694 Devsite localized content from translation request 66a39c... by Android Partner Docs <noreply@android.com>
- 165246927 Update August 2017 bulletin with CVE-2017-0687 by daroberts <daroberts@google.com>
PiperOrigin-RevId: 166080694
Change-Id: I2d3a8d77fa6a66c2099f13ba2e864545328fd17a
Diffstat (limited to 'en/devices/architecture/kernel')
-rw-r--r-- | en/devices/architecture/kernel/hardening.html | 100 | ||||
-rw-r--r-- | en/devices/architecture/kernel/index.html | 80 | ||||
-rw-r--r-- | en/devices/architecture/kernel/lldb-debug.html | 110 | ||||
-rw-r--r-- | en/devices/architecture/kernel/modular-kernels.html | 751 | ||||
-rw-r--r-- | en/devices/architecture/kernel/releases.html | 180 | ||||
-rw-r--r-- | en/devices/architecture/kernel/reqs-interfaces.html | 273 | ||||
-rw-r--r-- | en/devices/architecture/kernel/squashfs.html | 114 |
7 files changed, 1608 insertions, 0 deletions
diff --git a/en/devices/architecture/kernel/hardening.html b/en/devices/architecture/kernel/hardening.html new file mode 100644 index 00000000..28b6d87e --- /dev/null +++ b/en/devices/architecture/kernel/hardening.html @@ -0,0 +1,100 @@ +<html devsite> + <head> + <title>Kernel Hardening</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> +<p> +Android 8.0 added kernel hardening features to help mitigate kernel +vulnerabilities and find bugs in kernel drivers. The features are in <a +href="https://android.googlesource.com/kernel/common/">kernel/common</a> in +branches android-3.18, android-4.4, and android-4.9. +</p> +<h2 id="implementation">Implementation</h2> +<p> +To acquire these features, device manufacturers and SOCs should merge all +hardening patches from <code>kernel/common</code> to their kernel tree and +enable the following kernel configuration options: +</p> +<ul> + <li>Hardened usercopy: <code>CONFIG_HARDENED_USERCOPY=y</code></li> + <li>PAN emulation - arm64: <code>CONFIG_ARM64_SW_TTBR0_PAN=y</code></li> + <li>PAN emulation - arm: <code>CONFIG_CPU_SW_DOMAIN_PAN=y</code></li> + <li>KASLR - 4.4 and later kernels: + <code>CONFIG_RANDOMIZE_BASE=y</code></li> +</ul> +<p> +KASLR also requires bootloader support for passing hardware entropy through +either the device tree node <code>/chosen/kaslr-seed</code> or by implementing +<code>EFI_RNG_PROTOCOL</code>. +</p> +<p> +Also ensure existing hardening features are enabled: +</p> +<ul> + <li>Stack buffer overflow mitigation: + <code>CONFIG_CC_STACKPROTECTOR_STRONG=y</code></li> + <li>Internal memory protection: <code>CONFIG_DEBUG_RODATA=y</code> or + <code>CONFIG_STRICT_KERNEL_RWX=y</code></li> + <li>Restrict user-space access from kernel - x86 (enabled by default): + <code>CONFIG_X86_SMAP=y</code></li> +</ul> +<h2 id="testing">Testing</h2> +<p> +To test your implementation, add <code>CONFIG_LKDTM=y</code> to the kernel +configuration and confirm that each of the following commands lead to a kernel +panic: +</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="# ">echo ACCESS_USERSPACE > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_USERSPACE > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_RO > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_RO_AFTER_INIT > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_KERN > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_STACK > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_RODATA > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_KMALLOC > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_VMALLOC > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo CORRUPT_STACK > /sys/kernel/debug/provoke-crash/DIRECT</code> +</pre> +<p> +<strong>For android-4.9:</strong> +</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="# ">echo USERCOPY_HEAP_SIZE_TO > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo USERCOPY_HEAP_SIZE_FROM > /sys/kernel/debug/provoke-crash/DIRECT</code> +</pre> + +<h2 id="common-issues">Common issues</h2> +<p> +These changes are likely to expose bugs in kernel drivers, which need to be +fixed either by the device manufacturer or the owner of the kernel driver. +</p> +<ul> + <li>Hardened usercopy exposes incorrect bounds checking when copying data + to/from user space. These should be fixed like any other memory corruption bugs.</li> + <li>PAN emulation exposes direct user space access from the kernel, which is not + allowed. Drivers attempting to access user space memory need to be changed to + use the standard <code>copy_to_user()</code>/<code>copy_from_user()</code> + functions instead.</li> +</ul> +</body> +</html> diff --git a/en/devices/architecture/kernel/index.html b/en/devices/architecture/kernel/index.html new file mode 100644 index 00000000..456662f7 --- /dev/null +++ b/en/devices/architecture/kernel/index.html @@ -0,0 +1,80 @@ +<html devsite> + <head> + <title>Kernel</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>The Linux kernel is an extremely important part of the software on nearly +every Android device. This section describes Linux kernel development and +release models (below), +<a href="/devices/architecture/kernel/releases.html">stable and long-term +supported (LTS) kernels</a> (including why all Android devices should use stable +releases instead of cherry picking patches), kernel +<a href="/devices/architecture/kernel/config.html">configuration</a> and +<a href="/devices/architecture/kernel/hardening.html">hardening</a>, +requirements for +<a href="/devices/architecture/kernel/reqs-interfaces.html">interfaces</a> and +the +<a href="/devices/architecture/kernel/modular-kernels.html">modular kernels</a> +(introduced in Android O), kernel +<a href="/devices/architecture/kernel/lldb-debug.html">debugging</a> and +<a href="/devices/architecture/kernel/network_tests.html">network testing</a>, +and <a href="/devices/architecture/kernel/squashfs.html">SquashFS</a>.</p> + +<h2 id="linux-kernel-development">Linux kernel development</h2> +<p> +The Linux kernel is the largest collaborative software project ever. In 2016, +over 4,000 different developers from over 450 different companies contributed to +the project and there were 6 releases, each containing between 12,000 and 16,000 +different changes. At the end of 2016 the size of the Linux kernel was just over +56 thousand files, consisting of 22 million lines of code, build scripts, and +documentation (kernel release 4.9). (For full Linux development statistics, +refer to +<a href="https://kernelnewbies.org/DevelopmentStatistics" class="external">https://kernelnewbies.org/DevelopmentStatistics</a>.) +</p> +<p> +While the Linux kernel contains code for all the different chip architectures +and hardware drivers it supports, an individual system runs only a fraction of +the codebase. An average laptop uses around 2 million lines of kernel code from +5 thousand files to function properly, while the Pixel phone uses 3.2 million +lines of kernel code from 6 thousand files (due to the increased complexity of +an SoC). +</p> + +<h2 id="linux-kernel-releases">Linux kernel releases</h2> +<p>The Linux kernel uses a release model that differs substantially from +standard AOSP releases. With the release of the 2.6 kernel in December of 2003, +the kernel developer community switched from the previous model of having a +separate development and stable kernel branch, and moved to a <em>stable +only</em> branch model. In this model, a new release occurred every 2 to 3 +months, and that release was declared <em>stable</em> and recommended for all +users to run. This change in development model was due to the very long +release cycle prior to the 2.6 kernel (almost 3 years), and the struggle to +maintain two different branches of the codebase at the same time.</p> + +<p>The numbering of the kernel releases began at 2.6.x, where x was an +incrementing number that changed on every release (the value of the number has +no meaning, other than it is newer than the previous kernel release). The kernel +version since then has now moved to 4.x accounting for 2 major version changes. +These version numbers are chosen by the maintainer(s) only to avoid confusion +among users caused by higher minor release numbers.</p> + + </body> +</html> diff --git a/en/devices/architecture/kernel/lldb-debug.html b/en/devices/architecture/kernel/lldb-debug.html new file mode 100644 index 00000000..0c019fd5 --- /dev/null +++ b/en/devices/architecture/kernel/lldb-debug.html @@ -0,0 +1,110 @@ + <html devsite> + <head> + <title>Kernel Enhancements to LLDB/C++ Debugging</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p> +The Android 8.0 release includes kernel enhancements that help developers create +better applications by improving their debugging experience. +</p> +<p> +The arm64 Android kernels support setting watchpoints on memory addresses that +are not 4/8-byte aligned, and reporting all accesses to those addresses. +</p> +<h2 id="implementation">Implementation</h2> +<p> +The feature runs on any ARM 64-bit device. Adding the relevant support for +32-bit hardware/kernels is optional. All necessary kernel modifications have +already been done. +</p> +<p> +This feature is included in the current 3.10, 3.18, 4.4, and 4.9 kernel branches. +To add it to a kernel that does not already include it, cherry pick the +necessary CLs into your kernel build. Choose the patchset according to the +version your kernel is based on, as the patches required some adjustments due to +the kernel codebase evolving over time: +</p> + +<ul> + <li>branch android-3.10: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/bb42e87236d9defb511819991df21d464c1ef2eb">https://android.googlesource.com/kernel/common/+/bb42e87236d9defb511819991df21d464c1ef2eb</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/e8e0018306e1b9426dd70b44f057d1555c212ffb">https://android.googlesource.com/kernel/common/+/e8e0018306e1b9426dd70b44f057d1555c212ffb</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/fc0e9d5d77d9368ab15a06f7e61fff2ae2bc254d">https://android.googlesource.com/kernel/common/+/fc0e9d5d77d9368ab15a06f7e61fff2ae2bc254d</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/482ff2ca45737ee4bab3cf7221f6df1c9e14ea2c">https://android.googlesource.com/kernel/common/+/482ff2ca45737ee4bab3cf7221f6df1c9e14ea2c</a> + </ul> + + <li>branch android-3.18: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/b14849ae202487251b91e9f89af7eda484a80489">https://android.googlesource.com/kernel/common/+/b14849ae202487251b91e9f89af7eda484a80489</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/58a9c1be1966ca89facba6343a5d8781154196b7">https://android.googlesource.com/kernel/common/+/58a9c1be1966ca89facba6343a5d8781154196b7</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/d34d016794f345c5f1dccf4060b1132439098699">https://android.googlesource.com/kernel/common/+/d34d016794f345c5f1dccf4060b1132439098699</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/c82e0fdadfba89cdb50c96af622c2c5ad6cb55eb">https://android.googlesource.com/kernel/common/+/c82e0fdadfba89cdb50c96af622c2c5ad6cb55eb</a> + </ul> + + <li>branch android-4.4: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/91495bc622e9356dd63a7c77b98a21f0e2f5d2b2">https://android.googlesource.com/kernel/common/+/91495bc622e9356dd63a7c77b98a21f0e2f5d2b2</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/7409857a0717fa78dc936ea08099880be893156c">https://android.googlesource.com/kernel/common/+/7409857a0717fa78dc936ea08099880be893156c</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/b48318f371e8a8b98238deac868bc7af8ed8ba4b">https://android.googlesource.com/kernel/common/+/b48318f371e8a8b98238deac868bc7af8ed8ba4b</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/edc166a8714b012a3dd207e437c772ae2a264eca">https://android.googlesource.com/kernel/common/+/edc166a8714b012a3dd207e437c772ae2a264eca</a> + </ul> + + <li>branch android-4.9: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/85c450ac7ac0a69aca4c6eb845e419ae092d8c86">https://android.googlesource.com/kernel/common/+/85c450ac7ac0a69aca4c6eb845e419ae092d8c86</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/67de4de5628a47fa794920745dc7869c97b558d3">https://android.googlesource.com/kernel/common/+/67de4de5628a47fa794920745dc7869c97b558d3</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/32649b0a393303a97fab6003c5757ff7b100331a">https://android.googlesource.com/kernel/common/+/32649b0a393303a97fab6003c5757ff7b100331a</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/d32793b6be517dffa29329a92ffcc688e74bc690">https://android.googlesource.com/kernel/common/+/d32793b6be517dffa29329a92ffcc688e74bc690</a></li> + </ul> +</ul> + +<p> +This feature implements an extension of the <code>ptrace</code> API to support +writing a debugger. +</p> +<h2 id="validation">Validation</h2> +<p> +There is a kernel selftest in the upstream repository, a CTS test, and there +are CTS tests added exercising the new kernel API: +</p> +<p> +<a +href="https://android.googlesource.com/platform/bionic/+/master/tests/sys_ptrace_test.cpp">https://android.googlesource.com/platform/bionic/+/master/tests/sys_ptrace_test.cpp</a> +</p> +</body> +</html> diff --git a/en/devices/architecture/kernel/modular-kernels.html b/en/devices/architecture/kernel/modular-kernels.html new file mode 100644 index 00000000..98ac0528 --- /dev/null +++ b/en/devices/architecture/kernel/modular-kernels.html @@ -0,0 +1,751 @@ +<html devsite> + <head> + <title>Modular Kernel Requirements</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>In Android O, the device kernel splits into System-on-Chip (SoC), device, and +board-specific deliverables. This sets up the kernel and Android such that +Original Device Manufacturers (ODMs) and Original Equipment Manufacturers (OEMs) +can work in isolated board–specific trees for board–specific features, drivers, +etc., enabling them to override common kernel configuration, add new drivers in +the form of kernel modules, etc.</p> + +<p>This page provides details on requirements for:</p> + +<ul> +<li>Platform support for independent SoC and OEM/ODM kernel development. Android +O recommends all board–specific code to be built and shipped as kernel modules +in devices. As a result: + <ul> + <li>All platforms should support either + <a href="https://www.devicetree.org/" class="external">Device Tree</a> or + <a href="http://www.uefi.org/acpi/specs" class="external">Advanced + Configuration and Power Interface (ACPI)</a> to describe all non-discoverable + devices.</li> + <li>For device tree-based platforms, board–specific device nodes should be + added to the kernel device tree as + <a href="/devices/architecture/dto/index.html">overlays</a>.</li> + </ul> +</li> +<li>Application binary interface (ABI)/application programming interface (API) +tests in <a href="/devices/tech/vts/index.html">Vendor Test Suite (VTS)</a> to +ensure a given kernel can run the Android Open Source Project (AOSP) +framework.</li> +<li>Minimum kernel version per Android release and support for generating +<a href="/devices/architecture/vintf/index.html">Android Vendor Interface +(VINTF) kernel objects</a>.</li> +</ul> + +<h2 id="loadable-kernel-modules">Loadable kernel modules</h2> +<p>All SoC kernels should support loadable kernel modules. As a starting point, +the following kernel-config options (or their kernel-version equivalent) have +been added to +<a href="https://android.googlesource.com/kernel/common/+/android-4.4/android/configs/android-base.cfg" class="external">android-base.cfg</a> +in all common kernels and must be enabled in all device kernels:</p> + +<pre class="prettyprint"> +CONFIG_MODULES=y +CONFIG_MODULE_UNLOAD=y +CONFIG_MODVERSIONS=y +</pre> + +<p>All kernel modules are subject to module load/unload testing to ensure the +correctness of the driver/module.</p> + +<aside class="note"><strong>Note:</strong> +<code>CONFIG_MODULE_SRCVERSION_ALL</code> is optional and will not be tested +against.</aside> + +<h3 id="module-signing">Module signing</h3> +<p>Optionally, ODMs can enable module signing in their own kernel configuration +by enabling following kernel config options:</p> + +<pre class="prettyprint"> +CONFIG_MODULE_SIG=y +CONFIG_MODULE_SIG_FORCE=y +</pre> + +<p>On devices required to support verified boot, Android requires the kernel +modules to be in the partitions that have dm-verity enabled. Module signing is +not mandatory and will not be tested against; however, if desired, an ODM can +enable module signing as long as they have the key signing and other +infrastructure required to ensure independent kernel and filesystem OTA updates +in the future.</p> + +<h3 id="file-locations">File locations</h3> +<p>While Android 7.x and earlier do not mandate against kernel modules (and +include support for <code>insmod</code> and <code>rmmod</code>), Android O +recommends the use of kernel modules in the ecosystem. The following table shows +potential board–specific peripheral support required across three Android boot +modes:</p> + +<table> +<tr> +<th>Boot Mode</th> +<th>Storage</th> +<th>Display</th> +<th>Keypad</th> +<th>Battery</th> +<th>PMIC</th> +<th>Touchscreen</th> +<th>NFC, Wi-Fi,<br>Bluetooth</th> +<th>Sensors</th> +<th>Camera</th> +</tr> +<tr> +<td>Recovery</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td>> +<td><span class="compare-no"></span></td> +</tr> +<tr> +<td>Charger</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +</tr> +<tr> +<td>Android</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +</tr> +</table> + +<p>In addition to availability in Android boot modes, kernel modules may also be +categorized by who owns them (the SoC vendor or the ODM). If kernel modules are +being used, requirements for their placement in filesystem are as follows:</p> + +<ul> +<li>All kernels should have built-in support for booting and mounting partitions. +</li> +<li>Kernel modules should be loaded from a read-only partition.</li> +<li>For devices required to have verified boot, kernel modules should be loaded +from verified partitions.</li> +<li>Kernel modules should not be located in <code>/system</code>.</li> +<li>Kernel modules from the SoC vendor that are required for full Android or +Charger modes should be located in <code>/vendor/lib/modules</code>.</li> +<li>If an ODM partition exists, kernel modules from the ODM that are required +for full Android or Charger modes should be located in +<code>/odm/lib/modules</code>. Otherwise, these modules should be located in +<code>/vendor/lib/modules</code>.</li> +<li>Kernel modules from both the SoC vendor and ODM that are required for +Recovery mode should be located in the recovery <code>ramfs</code> at +<code>/lib/modules</code>.</li> +<li>If a kernel module is required for both Recovery mode and full Android or +Charger modes, it should exist both in the recovery <code>rootfs</code> and +either the <code>/vendor</code> or <code>/odm<strong> </strong></code>partitions +(as described above).</li> +<li>Kernel modules used in Recovery mode should not depend on modules located +only in <code>/vendor</code> or <code>/odm</code>, as those partitions are not +mounted in Recovery mode.</li> +<li>SoC vendor kernel modules should not depend on ODM kernel modules.</li> +</ul> + +<p>In Android 7.x and earlier, <code>/vendor</code> and <code>/odm</code> +partitions are <strong>not</strong> mounted early. In Android O, to make module +loading from these partitions possible, provisions have been made to mount +partitions early for both +<a href="https://source.android.com/devices/tech/ota/ab_updates">non-A/B and A/B +devices</a>. This also ensures the partitions are mounted in both Android and +Charger modes.</p> + +<h3 id="android-build-system-support">Android build system support</h3> +<p>In <code>BoardConfig.mk</code>, the Android build defines a +<code>BOARD_VENDOR_KERNEL_MODULES</code> variable that provides a full list of +the kernel modules intended for the vendor image. The modules listed in this +variable are copied into the vendor image at <code>/lib/modules/</code>, and, +after being mounted in Android, appear in <code>/vendor/lib/modules</code> (in +accordance with the above requirements). Example configuration of the vendor +kernel modules:</p> + +<pre class="prettyprint"> +vendor_lkm_dir := device/$(vendor)/lkm-4.x +BOARD_VENDOR_KERNEL_MODULES := \ + $(vendor_lkm_dir)/vendor_module_a.ko \ + $(vendor_lkm_dir)/vendor_module_b.ko \ + $(vendor_lkm_dir)/vendor_module_c.ko +</pre> + +<p>… where a vendor kernel module pre-built repository is mapped into the +Android build at the location listed above.</p> + +<p>The recovery image is likely to contain a subset of the vendor modules. The +Android build defines the variable <code>BOARD_RECOVERY_KERNEL_MODULES</code> +for these modules. Example:</p> + +<pre class="prettyprint"> +vendor_lkm_dir := device/$(vendor)/lkm-4.x +BOARD_RECOVERY_KERNEL_MODULES := \ + $(vendor_lkm_dir)/vendor_module_a.ko \ + $(vendor_lkm_dir)/vendor_module_b.ko +</pre> + +<p>The Android build takes care of running <code>depmod</code> to generate the +required <code>modules.dep</code> files in <code>/vendor/lib/modules</code> and +<code>/lib/modules</code> (<code>recovery ramfs</code>).</p> + +<h3 id="module-loading-&-versioning">Module loading & versioning</h3> +<p>We recommend loading all kernel modules in one pass from +<code>init.rc*</code> by invoking <code>modprobe -a</code>. This avoids the +overhead of repeatedly initializing the C runtime environment for the +<code>modprobe</code> binary. The <code>early-init</code> event can be modified +to invoke <code>modprobe</code>:</p> + +<pre class="prettyprint"> +on early-init + exec u:r:modprobe:s0 -- /vendor/bin/modprobe -a -d \ + /vendor/lib/modules module_a module_b module_c ... +</pre> + +<p>The kernel image may be updated separately from the vendor image, meaning +that kernel modules may be used with kernels other than the one they were +originally compiled against. To allow for this, and to protect against ABI +breakages, module versioning is used. Module versioning is enabled by +<code>CONFIG_MODVERSIONS=y</code> (one of the required kernel configuration +options mentioned above) and is documented in the kernel tree at +<code>Documentation/kbuild/modules.txt</code>.</p> + +<h2 id="mounting-partitions-early-first-stage-mount">Mounting partitions early +(first stage mount)</h2> +<font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: green; color: white"> REQUIRED </font> +<p>All Treble-enabled devices must enable first stage mount to make sure +<code>init</code> can load SELinux policy fragments that are spread across +<code>system</code> and <code>vendor</code> partitions (this also enables +loading of kernel modules as soon as possible after kernel boot).</p> + +<aside class="note"><strong>Note:</strong> For details on SELinux in Android +8.0, see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for +Android 8.0</a>.</aside> + +<p>Android must have access to the filesystem(s) on which the modules reside. To +enable, Android O supports mounting <code>/system</code>, <code>/vendor</code>, +or <code>/odm</code> as early as <code>init</code>'s first stage (i.e before +selinux is initialized). Device makers can use +<a href="/devices/architecture/dto/index.html">device tree overlays</a> to +specify <code>fstab</code> entries for early mounted partitions.</p> + +<aside class="key-point"><strong>Summary of AOSP early mount changes:</strong> +<ul> +<li> +<a href="https://android-review.googlesource.com/#/q/topic:pre-early-mount" class="external">Pre-early +mount v1</a> & +<a href="https://android-review.googlesource.com/#/q/status:merged+project:platform/system/core+branch:master+topic:pre-early-mount-2" class="external">v2</a></li> +<li><a href="https://android-review.googlesource.com/#/c/339471/" class="external">Miscellaneous +partition lookup removal</a></li> +<li> +<a href="https://android-review.googlesource.com/#/q/branch:master+topic:early-mount-support" class="external">Early +mount support</a></li> +<li> +<a href="https://android-review.googlesource.com/#/q/branch:master+topic:early-mount-support" class="external">Early +mount with VBoot 2.0 (AVB)</a></li> +</ul> +</aside> + +<h3 id="early-mounting-partitions-vboot-1-0">Early mounting partitions, VBoot +1.0</h3> +<p>Requirements to early mount partitions with vboot 1.0 include:</p> +<ol> +<li>Device node paths must use their <em>by-name</em> symlinks in +<code>fstab</code> and device tree entries. For example, instead of specifying +partitions using <code>/dev/block/mmcblk0pX</code>, ensure partitions are named +and the device node is <code>/dev/block/…./by-name/{system,vendor,odm}</code>. +</li> +<li>Paths given for <code>PRODUCT_{SYSTEM,VENDOR}_VERITY_PARTITION</code> and +<code>CUSTOM_IMAGE_VERITY_BLOCK_DEVICE</code> in the device configuration for +the product (i.e. in +<code>device/<em>oem</em>/<em>project</em>/device.mk</code>) must match the +corresponding block device nodes specified <em>by-name</em> in the +<code>fstab</code>/device tree entries. Example: +<pre class="prettyprint"> +PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/…./by-name/system +PRODUCT_VENDOR_VERITY_PARTITION := /dev/block/…./by-name/vendor +CUSTOM_IMAGE_VERITY_BLOCK_DEVICE := /dev/block/…./by-name/odm +</pre> +</li> +<li>Entries provided via device tree overlays must not repeat in the +<code>fstab</code> file fragments. For example, when specifying an entry to +mount <code>/vendor</code> in the device tree, the <code>fstab</code> file must +not repeat that entry.</li> +<li>Only <code>/system</code>, <code>/odm</code>, or <code>/vendor</code> can be +mounted early. Android does not include support to mount any other partitions in +<code>init</code> first stage.</li> +<li>Partitions requiring <code>verifyatboot</code> <strong>must not</strong> be +early mounted (doing so is unsupported).</li> +<li>The verity mode/state for verified partitions must be specified in kernel +cmdline using <code>androidboot.veritymode</code> option (existing +requirement).</li> +</ol> + +<h3 id="early-mounting-device-tree-vboot-1-0">Early mounting device tree, VBoot +1.0</h3> +<p>In Android O, <code>init</code> parses the device tree and creates +<code>fstab</code> entries to mount the partition early during its first stage. +An fstab entry takes the form:</p> + +<pre class="prettyprint">src mnt_point type mnt_flags fs_mgr_flags</pre> + +<p>Device tree properties are defined to mimic that format:</p> + +<ul> +<li><code>fstab</code>entries must be under <code>/firmware/android/fstab</code> +in the device tree and must have compatible string set to +<code>android,fstab</code>. </li> +<li>Each node under <code>/firmware/android/fstab</code> is treated as a single +early mount <code>fstab</code> entry. A node must have the following properties +defined: + <ul> + <li><code>dev</code>. Must point to the device node representing the + partition <em>by-name</em>.</li> + <li><code>type</code>. Must be the filesystem type (as in the + <code>fstab</code> files).</li> + <li><code>mnt_flags</code>. Must be the comma-separated list of mount + flags (as in <code>fstab</code> files).</li> + <li><code>fsmgr_flags</code>. Must be the list of Android <code>fs_mgr + flags</code> (as in <code>fstab</code> files). + <ul> + <li>A/B partitions must have <code>slotselect fs_mgr</code>option.</li> + <li>dm-verity enabled partitions must have <code>verify fs_mgr</code> option. + </li> + </ul> + </li> + </ul> +</li> +</ul> + +<h4><strong>Example:</strong> /system and /vendor on N6P</h4> +<p>The following example shows device tree early mount for <code>system</code> +and <code>vendor</code> partitions on Nexus 6P:</p> + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + system { + compatible = "android,system"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/system"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait,verify"; + }; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait"; + }; + }; + }; + }; +}; +</pre> + +<h4><strong>Example:</strong> /vendor on Pixel</h4> +<p>The following example shows device tree early mount for +<code>/vendor</code> on Pixel (remember to add <code>slotselect</code> for +partitions subject to A/B):</p> + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc/624000.ufshc/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,discard"; + fsmgr_flags = "wait,slotselect,verify"; + }; + }; + }; + }; +}; +</pre> + +<h3 id="early-mounting-partitions-vboot-2-0">Early mounting partitions, VBoot +2.0</h3> +<p>VBoot 2.0 is +<a href="https://android.googlesource.com/platform/external/avb/" class="external">Android +Verified Boot (AVB).</a> The requirements to early mount partitions with VBoot +2.0 are:</p> +<ol> +<li>The device node paths must use their <em>by-name</em> symlinks in +<code>fstab</code> and device tree entries. For example, instead of specifying +partitions using <code>/dev/block/mmcblk0pX</code>, ensure the partitions are +named and the device node is +<code>/dev/block/…./by-name/{system,vendor,odm}</code>.</li> +<li>Build system variables (such as +<code>PRODUCT_{SYSTEM,VENDOR}_VERITY_PARTITION</code> and +<code>CUSTOM_IMAGE_VERITY_BLOCK_DEVICE</code>) used for VBoot 1.0 are NOT +required for VBoot 2.0. Instead, new build variables introduced in VBoot 2.0 +(including <code>BOARD_AVB_ENABLE := true</code>) should be defined; for a full +configuration, refer to +<a href="https://android.googlesource.com/platform/external/avb/#Build-System-Integration" class="external">Build-System-Integration +for AVB</a>.</li> +<li>Entries provided via device tree overlays must not repeat in the +<code>fstab</code> file fragments. For example, if you specify an entry to mount +<code>/vendor</code> in the device tree, the <code>fstab</code> file must not +repeat that entry.</li> +<li>Only <code>/system</code>, <code>/odm</code>, or <code>/vendor</code> can be +mounted early. Android does not include support to mount any other partitions in +<code>init</code> first stage.</li> +<li>VBoot 2.0 does not support <code>verifyatboot</code>, regardless of whether +early mount is enabled or not.</li> +<li>The verity mode/state for verified partitions must be specified in kernel +cmdline using <code>androidboot.veritymode</code> option (existing requirement). +Make sure to include the following fixes for AVB: + <ul> + <li> + <a href="https://android-review.googlesource.com/#/q/topic:libavb-api-rev-for-verity-modes+(status:open+OR+status:merged)" class="external">https://android-review.googlesource.com/#/q/topic:libavb-api-rev-for-verity-modes+(status:open+OR+status:merged)</a></li> + <li> + <a href="https://android-review.googlesource.com/#/c/394215/" class="external">https://android-review.googlesource.com/#/c/394215/</a></li> + </ul> +</li> +</ol> + +<h3 id="early-mounting-device-tree-vboot-2-0">Early mounting device tree, VBoot +2.0</h3> +<p>The configuration in device tree for VBoot 2.0 is the same as that in +<a href="#early-mounting-device-tree-vboot-1-0">VBoot 1.0</a>, with the +following exceptions:</p> +<ul> +<li>The <code>fsmgr_flag</code> is switched from <code>verify</code> to +<code>avb</code>.</li> +<li>All partitions with AVB metadata must be in the vbmeta entry in device tree, +even when the partition isn't mounting early (e.g., <code>/boot</code>).</li> +</ul> + +<h4><strong>Example:</strong> /system and /vendor on N5X</h4> +<p>The following example shows device tree early mount for <code>system</code> +and <code>vendor</code> partitions on Nexus 5X. Note that:</p> +<ul> +<li><code>/system</code> is mounted with AVB and <code>/vendor</code> is mounted +without integrity verification.</li> +<li>As the Nexus 5X has no <code>/vbmeta</code> partition, so the top-level +vbmeta resides at the end of <code>/boot</code> partition (for details, refer to +the +<a href="https://android-review.googlesource.com/#/c/344907/" class="external">AOSP +changelist</a>). + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + vbmeta { + compatible = "android,vbmeta"; + parts = "boot,system,vendor"; + }; + fstab { + compatible = "android,fstab"; + system { + compatible = "android,system"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/system"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait,avb"; + }; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait"; + }; + }; + }; + }; +}; +</pre> +</li> +</ul> + +<h4><strong>Example:</strong> /vendor on Pixel</h4> +<p>The following example shows mounting <code>/vendor</code> early on a Pixel. +Note that:</p> + +<ul> +<li>More partitions are specified in the vbmeta entry because those partitions +are +<a href="https://android.googlesource.com/platform/external/avb/#The-VBMeta-struct">protected +by AVB.</a></li> +<li>All AVB partitions must be included, even if only <code>/vendor</code> is +early mounted.</li> +<li>Remember to add <strong><code>slotselect</code></strong> for partitions +subject to A/B. +<pre class="prettyprint"> +/ { + vbmeta { + compatible = "android,vbmeta"; + parts = "vbmeta,boot,system,vendor,dtbo"; + }; + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc/624000.ufshc/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,discard"; + fsmgr_flags = "wait,slotselect,avb"; + }; + }; + }; + }; +}; +</pre> +</li> +</ul> + +<h2 id="device-tree-overlay-support-bootloader">Device tree overlay support +(Bootloader)</h2> +<p> +<a href="https://lkml.org/lkml/2012/11/5/615">Device Tree Overlay</a> (DTO) was +designed to extend the existing +<a href="https://events.linuxfoundation.org/sites/events/files/slides/petazzoni-device-tree-dummies.pdf" class="external">flattened +device-tree (FDT)</a> implementation so that the initial device-tree data in +kernel can be modified by userspace at runtime by loading additional overlay +FDTs that amend the original data. Android does not require runtime updates of +DT blobs from user space, but instead recommends that vendors add the device +tree patching in the bootloader with the help of +<code>libfdt</code>/<code>libufdt</code>.</p> + +<p>In Android 7.x and earlier, Android did not require device tree support and +did not provide recommendations regarding how vendors pass DT blobs to the +kernel or where they store them. Android O recommends such support to keep the +board–specific and SoC-only parts of the kernel separate.</p> + +<h3 id="partitioning-requirements">Partitioning requirements</h3> +<p>Most Android devices today append the DT blob to the kernel at build time, +which the bootloader knows how to read from. As Android has no specific +requirements for how to build/store DT blobs (which is considered as part of the +SoC kernel), the DT blob can be appended to the kernel or stored in a separate +partition. The only assumption is that the bootloader already knows how and +where to load the DT blob from.</p> + +<p>Requirements for Device Tree Overlay support (if used):</p> +<ul> +<li>Device should have new device tree blob for overlay (DTBO) partion per +kernel image for board–specific DT overlay (for details on the partition format, +see <a href="/devices/architecture/dto/partitions.html">DTB/DTBO Partitions</a>. +The assumption is that bootloader already knows where and how to load the +SoC–specific DTB.</li> +<li>Overlay DT partition should be +<a href="https://source.android.com/devices/tech/ota/ab_updates.html">A/B-ed</a> +for A/B devices. For these devices, the recovery kernel is the same as Android +kernel, but the partition must be A/B-ed as it can be updated via OTA.</li> +<li>Partition size is board–specific. + <ul> + <li>The DT overlay partition size depends on the device and the amount of + changes needed on top of the main SoC kernel DT blob.</li> + <li>The size of the DTBO partition is a function of number of changes needed to + make the SoC kernel. Choose a size with room to grow for future updates + (typically, 8MB partition size is more than enough).</li> + </ul> +</li> +</ul> + +<h3 id="bootloader-requirements">Bootloader requirements</h3> +<p>Requirements for bootloader include the following:</p> +<ul> +<li>Bootloader should know how and where (considering the boot slot for A/B +devices) to load the SoC–specific DT blob from in a vendor-specific way. This is +typically extracted from the end of the kernel image as blobs are appended to +the kernel.</li> +<li>Bootloader should know how and where (considering the boot slot for A/B +devices) to load the overlay DT blob from in a vendor-specific way.</li> +<li>Bootloader must patch the main DT blob with the overlay before passing the +combined device tree to the kernel.</li> +</ul> + +<p>For more details about adding support for DTO in bootloader, see +<a href="/devices/architecture/dto/index.html">Device Tree Overlays</a>.</p> + +<h2 id="core-kernel-requirements">Core kernel requirements</h2> +<p>Android O mandates a minimum kernel version and kernel configuration and +checks them both in VTS as well as during an OTA. Android device kernels must +enable the kernel <code>.config</code> support along with the option to read the +kernel configuration at runtime through <code>procfs</code>.</p> + +<h3 id="kernel-config-support">Kernel .config support</h3> +<p>All device kernels must enable the entirety of +<a href="https://android.googlesource.com/kernel/common/+/android-4.4/android/configs/android-base.cfg" class="external">android-base.cfg</a>, +which must include the following kernel–config options (or their kernel–version +equivalent):</p> + +<pre class="prettyprint"> +CONFIG_IKCONFIG=y +CONFIG_IKCONFIG_PROC=y +</pre> + +<h3 id="kernel-version">Kernel version</h3> +<p>Kernel version requirements:</p> +<ul> +<li>All SoCs productized in 2017 must launch with kernel 4.4 or newer.</li> +<li>All other SoCs launching new Android devices running Android O must use +kernel 3.18 or newer.</li> +<li>Regardless of launch date, all SoCs with device launches on Android O remain +subject to kernel changes required to enable Treble.</li> +<li>Older Android devices released prior to Android O but that will be upgraded +to Android O can continue to use their original base kernel version if +desired.</li> +</ul> + +<h3 id="device-tree-support">Device tree support</h3> +<p>Device tree support in the kernel must be enabled and bootloaders must pass +the hardware description in the form of device tree to the kernel (unless the +platform supports ACPI). The device tree must also be available for Android to +read and be able to pass vendor/odm specific parameters to Android. +<code>CONFIG_OF</code> (along with all other device/subsystem specific +<code>CONFIG_OF_*</code> kernel config options) are mandatory.</p> + +<p><code>CONFIG_PROC_DEVICETREE</code> is required on kernels prior to 3.15 so +Android can access vendor/odm specific configuration very early during boot. On +kernels 3.15 and later, the functionality of this option is merged into +<code>CONFIG_OF</code>.</p> + +<pre class="prettyprint"> +CONFIG_OF=y +CONFIG_PROC_DEVICETREE=y (kernels prior to 3.15) +</pre> + +<p>For an example of using device tree to early mount +<code>vendor</code>/<code>odm</code> partitions, refer to the +<a href="https://android-review.googlesource.com/#/c/337310/" class="external">AOSP +changelist</a>.</p> + +<h3 id="debugfs">DebugFS</h3> +<p>The implementation of the vendor interface should not rely on +<code>debugfs</code>. It may be enabled, but VTS testing may be done with +<code>debugfs</code> unmounted.</p> + +<h2 id="beyond-android-o">Beyond Android O</h2> +<p>Android O recommends any board–specific kernel functionality to be in the +form of loadable kernel modules and device–tree overlays. The rest of the kernel +is treated monolithically with respect to Android (whether or not is it is +actually a monolithic kernel, or parts of it are compiled as kernel modules). +</p> + +<p>This monolithic kernel is an SoC kernel that can boot on the SoC vendor's +reference hardware but nothing beyond that. Today, SoC kernels are treated +similar to the common kernel; they are also heavily replicated in board–specific +repos. This distribution model causes them to be fixed differently for the same +bug in each branch, delaying future updates to the kernel due to cherry–picking +at different times or fixing the same bug differently. To counter this, the SoC +kernels must be a separate deliverable, with everyone who uses the SoC +contributing to the same SoC kernel.</p> + +<p>Figure 1 (below) illustrates a common example of how SoC kernels get +fragmented over time, across Android releases, and across ODMs.</p> + +<img src="../images/treble_kernel_current.png"> +<figcaption><strong>Figure 1.</strong> Device kernel replication.</figcaption> + +<p>Figure 1 shows the following:</p> +<ol> +<li>It takes a significant amount of effort and time for everyone to cross-merge +across board–specific branches/tags.</li> +<li>While waiting for the cross-merge, Android device manufacturers patch their +own kernel for bugs/security fixes.</li> +<li>Divergence from the ancestor make future upgrades/merges really +difficult.</li> +</ol> +<p>The proposed model for a common SoC kernel addresses problems created by +upmerging changes (SoC-specific bug fixes, LTS upgrades, security fixes, etc.). +Figure 2 (below) illustrates how the workflow will change in an ideal, +unified–per–SoC–kernel scenario:</p> + +<img src="../images/treble_kernel_treble.png"> +<figcaption><strong>Figure 2.</strong> Android O and higher device +kernels.</figcaption> + +<p>This is intended to solve the problem of fragmented kernel repos by +recommending and working with device manufacturers to stay up to date with the +common SoC kernel. Android O provides all possible options to ODMs to help them +avoid maintaining their own SoC kernels and instead rely on the common SoC +kernel for LTS upgrades/bug fixes/security patches/etc.</p> + +<p>As a start, we want to facilitate all ODMs/vendors using a single kernel +source for an SoC. In the future, we want to move towards a single binary +distribution of kernel per-SoC.</p> + +<h3 id="upstreaming">Upstreaming</h3> +<p>To make updating to newer kernel versions much easier and almost automatic, +and to provide a more secure and reliable platform for ODMs to build a product +with, it is strongly recommended that SoC vendors work to upstream their kernel +changes and get them accepted into the main kernel.org repository. While doing +so requires additional, up front efforts in time and engineering resources, it +is well documented to save both time and money in the long run. It has also been +documented that merged code is of a much higher quality with fewer bugs and +security issues (and usually smaller) than code that has not been reviewed by +the community.</p> + +<p>If full support for the SoC is merged upstream, the community can make needed +API changes as the internal kernel API evolves over time, automatically +extending the longevity of the platform. The kernel can also be automatically +tested for any regressions in development and stable releases by adding the +hardware platform to one of the many community-managed kernel test platforms +(such as +<code><a href="https://kernelci.org" class="external">kernelci.org</a></code>). +</p> + +<p>For help working with the Linux kernel community to upstream your code, refer +to the following resources:</p> +<ul> +<li><code>Documentation/process</code> +(<code>Documentation/development-process</code> in 4.9 and earlier)</li> +<li><code>Documentation/CodingStyle</code></li> +<li><code>Documentation/SubmittingPatches</code></li> +</ul> + +<p>The community uses a minimal review process to accept stand-alone drivers and +filesystems into the staging portion of the kernel, where the community works to +improve code quality.</p> + + </body> +</html> diff --git a/en/devices/architecture/kernel/releases.html b/en/devices/architecture/kernel/releases.html new file mode 100644 index 00000000..612922d9 --- /dev/null +++ b/en/devices/architecture/kernel/releases.html @@ -0,0 +1,180 @@ +<html devsite> + <head> + <title>Stable Kernel Releases & Updates</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p> +The Linux kernel stable release model started in 2005, when it was determined +that the existing kernel development model (a new release every 2-3 months) was +not meeting the needs of most users. Users wanted bugfixes made during those 2-3 +months, and Linux distributions found it difficult to keep kernels up to date +without feedback from the kernel community. In general, attempts to keep +individual kernels secure and with the latest bugfixes was a large and confusing +effort by lots of different individuals. +</p> +<p> +Stable kernel releases are based directly on Linus Torvalds' releases, and are +released every week or so, depending on various external factors (time of year, +available patches, maintainer workload, etc.). The numbering of the stable +releases starts with the number of the kernel release, and an additional number +is added to the end of it. For example, the 4.4 kernel is released by Linus, and +then the stable kernel releases based on this kernel are numbered 4.4.1, 4.4.2, +4.4.3, and so on. This sequence is usually shortened with the number 4.4.y when +referring to a stable kernel release tree. Each stable kernel release tree is +maintained by a single kernel developer, who is responsible for picking the +needed patches for the release and managing the review/release process. +</p> +<p> +Stable kernels are maintained for the length of the current development cycle. +After Linus releases a new kernel, the previous stable kernel release tree is +stopped and users must move to the newer released kernel. +</p> + +<h2 id="long-term-stable-kernels">Long-term stable kernels</h2> +<p> +After a year of this new stable release process, it was determined that many +different users of Linux wanted a kernel to be supported for longer than just a +few months. In response, the Long Term Supported (LTS) kernel release was +created, with the first LTS kernel (2.6.16) released in 2006. Since then, a new +LTS kernel has been selected once a year and kernel community maintains that +kernel for a minimum of 2 years. +</p> +<p>At the time of this writing, the LTS kernels are the 4.4.y and 4.9.y +releases, and a new kernel is released weekly. Due to the needs of some users +and distributions, a few additional older kernels are maintained by kernel +developers at a slower release cycle. Information about all long-term stable +kernels, who is in charge of them, and how long they will be maintained, can be +found on the +<a href="https://www.kernel.org/category/releases.html" class="external">kernel.org +releases</a> page.</p> +<p> +LTS kernel releases average 6-8 patches accepted per day, while the normal +stable kernel releases contain 10-15 patches per day. The number of patches +fluctuates per release given the current time of the corresponding development +kernel release, and other external variables. The older a LTS kernel is, the +less patches are applicable to it as many recent bugfixes are not relevant to +older kernels. However, the older a kernel is, the harder it is to backport the +changes that are needed to be applied, due to the changes in the codebase. So +while there might be a lower number of overall patches being applied, the effort +involved in maintaining a LTS kernel is greater than maintaining the normal +stable kernel. +</p> + +<h2 id="stable-kernel-patch-rules">Stable kernel patch rules</h2> +<p>The rules for what can be added to a stable kernel release have remained +almost identical since its introduction and are summarized below:</p> +<ul> +<li>Must be obviously correct and tested. +<li>Must not be bigger than 100 lines. +<li>Must fix only one thing. +<li>Must fix something that has been reported to be an issue. +<li>Can be a new device id or quirk for hardware, but not add major new +functionality. +<li>Must already be merged into Linus Torvalds' tree.</li> +</ul> +<aside class="note"><strong>Note:</strong> For a full list of the rules for +patches to be accepted into a stable kernel release, refer to the +<code><a href="https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html" class="external">Documentation/process/stable_kernel_rules.rst</a></code> +kernel file.</aside> + +<p>The last rule, "Must already be merged into Linus Torvalds' tree", prevents +the kernel community from losing fixes. The community never wants a fix to go +into a stable kernel release that is not already in Linus Torvalds' tree, so +that anyone who upgrades should never see a regression. This prevents many +problems that other projects who maintain a stable and development branch can +have.</p> + +<h2 id="kernel-updates">Kernel updates</h2> +<p>The Linux kernel community has promised its userbase that no upgrade will +ever break anything that is currently working in a previous release. That +promise still holds true today. Regressions do happen, but those are the highest +priority bugs and are either quickly fixed, or the change that caused the +regression is quickly reverted from the Linux kernel tree.</p> + +<p>This promise holds true for both the incremental stable kernel updates, as +well as the larger major updates that happen every three months. However, the +kernel community can only make this promise for the code that is merged into the +Linux kernel tree. Any code that is merged into a device's kernel that is not in +the <a href="https://www.kernel.org/">kernel.org</a> releases is unknown and +interactions with it can never be planned for, or even considered.</p> + +<p>Devices based on Linux that have large patch sets can have major issues when +updating to newer kernels, because of the large number of changes between each +release (10-14 thousand changes per release). SoC patchsets are especially known +to have issues with updating to newer kernels due to their large size and heavy +modification of architecture specific, and sometimes core, kernel code. As a +result, most SoC vendors are starting to standardize on using the LTS releases +for their devices, enabling those devices to receive bug and security updates +directly from the Linux kernel community.</p> + +<h2 id="security">Security</h2> +<p>When doing kernel releases, the Linux kernel community almost never declares +specific changes as <em>security fixes</em>. This is due to the basic problem of +the difficulty in determining if a bugfix is a security fix or not at the time +of creation. Also, many bugfixes are only determined to be security related +after much time has passed, so the kernel community strongly recommends always +taking all bugfixes that are released.</p> + +<aside class="note"><strong>Note:</strong> For details on Linus Torvalds' +statement on security fixes, refer to the relevant +<a href="http://marc.info/?t=121507404600023&r=4&w=2" class="external">email +thread</a>.</aside> + +<p> +When security problems are reported to the kernel community, they are fixed as +soon as possible and pushed out publically to the development tree and the +stable releases. As described above, the changes are almost never described as +a "security fix", but rather look like any other bugfix for the kernel. This is +done to allow affected parties the ability to update their systems before the +reporter of the problem announces it. +</p> + +<p>For details on reporting security bugs to the kernel community to get +them resolved and fixed as soon as possible, refer to +<a href="https://www.kernel.org/doc/html/latest/admin-guide/security-bugs.html" class="external">Security +bugs</a> in <em>The Linux kernel user's and administrator's guide</em> at +<a href="https://www.kernel.org">www.kernel.org</a>.</p> + +<p> +Because security bugs are not announced to the public by the kernel team, CVE +numbers for Linux kernel-related issues are usually released weeks, months, and +sometimes years after the fix was merged into the stable and development +branches. +</p> +<h3 id="keeping-a-secure-system">Keeping a secure system</h3> +<p>When deploying a device that uses Linux, it is strongly recommended that all +LTS kernel updates be taken by the manufacturer and pushed out to their users +after proper testing shows the update works well. This has several advantages: +</p> +<ul> +<li>Releases have been reviewed by the kernel developers as a whole, not in +individual parts.</li> +<li>It is hard, if not impossible, to determine which patches fix "security" +issues and which do not. Almost every LTS release contains at least one known +security fix, and many yet "unknown".</li> +<li>If testing shows a problem, the kernel developer community will react +quickly to resolve the issue.</li> +<li>Attempts to filter out only the changes you run will result in a kernel +tree that is impossible to merge correctly with future upstream releases.</li> +</ul> + + </body> +</html> diff --git a/en/devices/architecture/kernel/reqs-interfaces.html b/en/devices/architecture/kernel/reqs-interfaces.html new file mode 100644 index 00000000..82349f7b --- /dev/null +++ b/en/devices/architecture/kernel/reqs-interfaces.html @@ -0,0 +1,273 @@ +<html devsite> + <head> + <title>Interface Requirements</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This page describes a subset of the Linux kernel interfaces on which Android +relies to function properly. The presence and correctness of these interfaces is +tested as part of the <a href="/devices/tech/vts/index.html">Vendor Test Suite +(VTS)</a>. This subset will grow over time to contain a larger portion of +Android kernel interfaces.</p> + +<h2 id="system-calls">System calls</h2> +<p>System calls are expected to provide the same signatures and semantics as in +the upstream Linux kernel.</p> + +<p>ARM64 system calls required by bionic per +<code>bionic/libc/SYSCALLS.txt</code>:</p> + +<table> +<tr> +<td class="devsite-click-to-copy">accept4, acct, adjtimex, bind, brk, capget, capset, chdir, chroot, clock_adjtime, clock_getres, clock_gettime, clock_nanosleep, clock_settime, close, connect, delete_module, dup3, dup, epoll_create1, epoll_ctl, epoll_pwait, eventfd2, execve, exit, exit_group, faccessat, fadvise64, fallocate, fchdir, fchmodat, fchmod, fchownat, fchown, fcntl, fdatasync, fgetxattr, flistxattr, flock, fremovexattr, fsetxattr, fstat, newfstatat, fstatfs, fsync, ftruncate, getcpu, getcwd, getdents64, getegid, geteuid, getgid, getgroups, getitimer, getpeername, getpgid, getpid, getppid, getpriority, getresgid, getresuid, getrlimit, getrusage, getsid, getsockname, getsockopt, gettimeofday, getuid, getxattr, init_module, inotify_add_watch, inotify_init1, inotify_rm_watch, ioctl, kill, syslog, lgetxattr, linkat, listen, listxattr, llistxattr, lremovexattr, lseek, lsetxattr, madvise, mincore, mkdirat, mknodat, mlockall, mlock, mmap, mount, mprotect, mremap, msync, munlockall, munlock, munmap, nanosleep, openat, personality, pipe2, ppoll, prctl, pread64, preadv, prlimit64, process_vm_readv, process_vm_writev, pselect6, ptrace, pwrite64, pwritev, quotactl, readahead, readlinkat, read, readv, reboot, recvfrom, recvmmsg, recvmsg, removexattr, renameat, rt_sigaction, rt_sigpending, rt_sigprocmask, rt_sigqueueinfo, rt_sigsuspend, rt_sigtimedwait, sched_getaffinity, sched_getparam, sched_get_priority_max, sched_get_priority_min, sched_getscheduler, sched_rr_get_interval, sched_setaffinity, sched_setparam, sched_setscheduler, sched_yield, sendfile, sendmmsg, sendmsg, sendto, setdomainname, setfsgid, setfsuid, setgid, setgroups, sethostname, setitimer, setns, setpgid, setpriority, setregid, setresgid, setresuid, setreuid, setrlimit, setsid, setsockopt, set_tid_address, settimeofday, setuid, setxattr, shutdown, sigaltstack, signalfd4, socketpair, socket, splice, statfs, swapoff, swapon, symlinkat, sync_file_range, sync, sysinfo, tee, tgkill, timer_create, timer_delete, timerfd_create, timerfd_gettime, timerfd_settime, timer_getoverrun, timer_gettime, timer_settime, times, truncate, umask, umount2, uname, unlinkat, unshare, utimensat, vmsplice, wait4, waitid, write, writev</td> +</tr></table> + +<p>ARM32 system calls required by bionic per +<code>bionic/libc/SYSCALLS.txt</code>:</p> + +<table> +<tr> +<td class="devsite-click-to-copy">accept4, acct, adjtimex, arm_fadvise64_64, bind, brk, cacheflush, capget, capset, chdir, chroot, clock_adjtime, clock_getres, clock_gettime, clock_nanosleep, clock_settime, close, connect, delete_module, dup3, dup, epoll_create1, epoll_ctl, epoll_pwait, eventfd2, execve, exit, exit_group, faccessat, fallocate, fchdir, fchmodat, fchmod, fchownat, fchown32, fcntl64, fdatasync, fgetxattr, flistxattr, flock, fremovexattr, fsetxattr, fstat64, fstatat64, fstatfs64, fsync, ftruncate64, getcpu, getcwd, getdents64, getegid32, geteuid32, getgid32, getgroups32, getitimer, getpeername, getpgid, getpid, getppid, getpriority, getresgid32, getresuid32, ugetrlimit, getrusage, getsid, getsockname, getsockopt, gettimeofday, getuid32, getxattr, init_module, inotify_add_watch, inotify_init1, inotify_rm_watch, ioctl, kill, syslog, lgetxattr, linkat, listen, listxattr, llistxattr, _llseek, lremovexattr, lseek, lsetxattr, madvise, mincore, mkdirat, mknodat, mlockall, mlock, mmap2, mount, mprotect, mremap, msync, munlockall, munlock, munmap, nanosleep, openat, personality, pipe2, ppoll, prctl, pread64, preadv, prlimit64, process_vm_readv, process_vm_writev, pselect6, ptrace, pwrite64, pwritev, quotactl, readahead, readlinkat, read, readv, reboot, recvfrom, recvmmsg, recvmsg, removexattr, renameat, rt_sigaction, rt_sigpending, rt_sigprocmask, rt_sigqueueinfo, rt_sigsuspend, rt_sigtimedwait, sched_getaffinity, sched_getparam, sched_get_priority_max, sched_get_priority_min, sched_getscheduler, sched_rr_get_interval, sched_setaffinity, sched_setparam, sched_setscheduler, sched_yield, sendfile64, sendfile, sendmmsg, sendmsg, sendto, setdomainname, setfsgid, setfsuid, setgid32, setgroups32, sethostname, setitimer, setns, setpgid, setpriority, setregid32, setresgid32, setresuid32, setreuid32, setrlimit, setsid, setsockopt, set_tid_address, settimeofday, set_tls, setuid32, setxattr, shutdown, sigaction, sigaltstack, signalfd4, socketpair, socket, splice, statfs64, swapoff, swapon, symlinkat, sync_file_range2, sync, sysinfo, tee, tgkill, timer_create, timer_delete, timerfd_create, timerfd_gettime, timerfd_settime, timer_getoverrun, timer_gettime, timer_settime, times, truncate64, truncate, umask, umount2, uname, unlinkat, unshare, utimensat, vmsplice, wait4, waitid, write, writev</td> +</tr></table> + +<p>The system calls listed below are made by bypassing bionic:</p> + +<table> + <tr> + <th style="width:20%">All Architectures</th> + <td>gettid, futex, clone, rt_sigreturn, rt_tgsigqueueinfo, restart_syscall, +getrandom, perf_event_open, syncfs, tkill, seccomp</td> + </tr> + <tr> + <th>arm</th> + <td>vfork, sigreturn, pipe, access, stat64, lstat64, open, getdents, eventfd, +epoll_wait, readlink, epoll_create, creat, unlink</td> + </tr> + <tr> + <th>arm64</th> + <td>pivot_root, ioprio_get, ioprio_set</td> + </tr> +</table> + +<aside class="note"><strong>Note:</strong> x86 and x86_64 system calls will be +added in a future release.</aside> + +<h3 id="prctl">prctl</h3> +<p>In addition to the upstream <code>prctl</code> operations for supported +kernel versions, Android relies on additional <code>prctl</code> operations, the +implementation of which can be found in the android-common kernel.</p> + +<pre class="prettyprint"> +PR_SET_TIMERSLACK_PID +PR_SET_VMA +</pre> + +<h2 id="filesystems">Filesystems</h2> +<p>The Linux kernel exports interfaces via several filesystems. Android expects +these interfaces to communicate the same information, in the same format, and +provide the same semantics as in the upstream Linux kernel. For interfaces that +do not exist upstream, the appropriate behavior is dictated by the corresponding +branch of the Android common kernel.</p> + +<h3 id="procfs">procfs</h3> +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/proc/cmdline</code></td> + <td>Read-only file containing command line arguments passed to the kernel. + </td> + </tr> + <tr> + <td><code>/proc/config.gz</code></td> + <td>Read-only file containing kernel build configuration.</td> + </tr> + <tr> + <td><code>/proc/cpuinfo</code></td> + <td>Read-only file containing architecture-specific CPU details.</td> + </tr> + <tr> + <td><code>/proc/kmsg</code></td> + <td>Read-only file showing kernel messages in real time.</td> + </tr> + <tr> + <td><code>/proc/meminfo</code></td> + <td>Read-only file showing memory subsystem details.</td> + </tr> + <tr> + <td><code>/proc/modules</code></td> + <td>Read-only file containing information about loaded kernel modules.</td> + </tr> + <tr> + <td><code>/proc/mounts</code></td> + <td>Symlink to <code>/proc/self/mounts</code>, which is a read-only file + listing information about the mounted filesystems.</td> + </tr> + <tr> + <td><code>/proc/net/xt_qtaguid/ctrl</code></td> + <td>Read-write file providing information about tagged sockets.</td> + </tr> + <tr> + <td><code>/proc/self/maps</code></td> + <td>Read-only file containing the currently mapped memory regions and + permissions.</td> + </tr> + <tr> + <td><code>/proc/stat</code></td> + <td>Read-only file containing various kernel and system statistics.</td> + </tr> + <tr> + <td><code>/proc/sys/kernel/kptr_restrict</code></td> + <td>Read-write file that determines whether kernel pointers are printed in + <code>proc</code> files and other interfaces.</td> + </tr> + <tr> + <td><code>/proc/sys/kernel/randomize_va_space</code></td> + <td>Read-write file that determines the address layout randomization policy + for the system.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_min_addr</code></td> + <td>Read-write file that determines the minimum address than can be + <code>mmap</code>'d.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_rnd_bits</code></td> + <td>Read-write file that specifies the amount of randomness in + <code>mmap</code>'d addresses.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_rnd_compat_bits</code></td> + <td>Read-write file that specifies the amount of randomness in + <code>mmap</code>'d addresses.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/overcommit_memory</code></td> + <td>Read-write file that determines the kernel virtual memory accounting + mode.</td> + </tr> + <tr> + <td><code>/proc/uid_cputime/remove_uid_range</code></td> + <td>Write-only file that, when written, removes UIDs from being shown in + <code>/proc/uid_cputime/show_uid_stat</code>.</td> + </tr> + <tr> + <td><code>/proc/uid_cputime/show_uid_stat</code></td> + <td>Read-only file containing the time a UID's processes spent in user and + kernel space.</td> + </tr> + <tr> + <td><code>/proc/version</code></td> + <td>Read-only file containing a string describing the kernel version.</td> + </tr> + <tr> + <td><code>/proc/vmallocinfo</code></td> + <td>Read-only file containing <code>vmalloc</code>'d ranges.</td> + </tr> + <tr> + <td><code>/proc/zoneinfo</code></td> + <td>Read-only file containing information about memory zones.</td> + </tr> +</table> + +<h3 id="dev">dev</h3> + +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/dev/ashmem</code></td> + <td>Anonymous shared memory device file.</td> + </tr> + <tr> + <td><code>/dev/binder</code></td> + <td>Binder device file.</td> + </tr> + <tr> + <td><code>/dev/hwbinder</code></td> + <td>Hardware binder device file.</td> + </tr> + <tr> + <td><code>/dev/tun</code></td> + <td>Universal TUN/TAP device file.</td> + </tr> + <tr> + <td><code>/dev/xt_qtaguid</code></td> + <td>QTAGUID netfilter device file.</td> + </tr> +</table> + +<h3 id="sysfs">sysfs</h3> +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/sys/devices/system/cpu/online</code></td> + <td>Read-only file showing ranges of CPUs that are currently online.</td> + </tr> + <tr> + <td><code>/sys/kernel/wakeup_reasons/last_resume_reason</code></td> + <td>Read-only file showing a textual description of why the system exited the + last instance of suspend.</td> + </tr> + <tr> + <td><code>/sys/devices/system/cpu/kernel_max</code></td> + <td>Read-only file showing the maximum CPU index supported by the kernel. + </td> + </tr> +</table> + +<h3 id="selinuxfs">selinuxfs</h3> +<p>The framework mounts <code>selinuxfs</code> at <code>/sys/fs/selinux</code>. +</p> + +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/sys/fs/selinux/checkreqprot</code></td> + <td>Read/write file containing a binary flag that determines how selinux + protections are checked on <code>mmap</code> and <code>mprotect</code> calls. + </td> + </tr> + <tr> + <td><code>/sys/fs/selinux/null</code></td> + <td>Read/write null device for use by selinux.</td> + </tr> + <tr> + <td><code>/sys/fs/selinux/policy</code></td> + <td>Read-only file containing the selinux policy in binary form.</td> + </tr> +</table> + +<aside class="note"><strong>Note:</strong> For details on SELinux in Android +8.0, see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for +Android 8.0</a>.</aside> + + </body> +</html> diff --git a/en/devices/architecture/kernel/squashfs.html b/en/devices/architecture/kernel/squashfs.html new file mode 100644 index 00000000..93b97ad0 --- /dev/null +++ b/en/devices/architecture/kernel/squashfs.html @@ -0,0 +1,114 @@ +<html devsite> + <head> + <title>Optimizing SquashFS at the Kernel Level</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + <p> + SquashFS is a compressed read-only filesystem for Linux. The file system is + read-only by design and thus suitable for use on the system partition. Many + Android devices may benefit from using this file system for their system + partition, for example, the following: + </p><ul> + <li>Devices with a low capacity storage such as Android Watch. + <li>Devices with a slow flash storage (compression reduces the number of block + I/Os).</li></ul> + <p> + Unfortunately the performance of SquashFS lags behind ext4. + </p> + <h2 id="optimizations">Optimizations</h2> + <p> + The following optimizations have been implemented to improve the performance of + SquashFS. + </p> + <h3 id="reduce-the-memory-usage-and-memcpy">Reduce the memory usage and + memcpy</h3> + <p> + When reading a block (default 128K), SquashFS tries to grab all the pages + covering this block. + </p> + <p> + If a single page is up-to-date or locked, it falls back to allocating a full + block, submitting a read request, and then copying its content to the pages. + </p> + <p> + This approach is very ineffective; after some time the page cache is likely to + contain pages that are up-to-date even if the adjacent pages are not. + </p> + <p> + The code is now able to handle blocks with holes (=missing pages). This + improves performance in the following ways: + </p><ul> + <li>Reduces the number of <code>memcpy</code> calls + <li>Decreases memory allocations</li></ul> + <h3 id="asynchronous-reads">Asynchronous reads</h3> + <p> + SquashFS still uses the deprecated <code>ll_rw_block()</code> function. There + are two problems with this approach: + </p><ul> + <li>As the name implies, the function waits for the reads to complete before + returning. This is redundant since <code>.readpage()</code> already waits on the + page's lock. Moreover, we need an asynchronous mechanism to efficiently + implement <code>.readpages()</code>. + <li>Merging the read requests entirely depends on the I/O scheduler. + <code>ll_rw_block()</code> simply creates one request per buffer. SquashFS has + more information than the I/O scheduler about what should be merged. Moreover, + merging the request means that we rely less on the I/O scheduler.</li></ul> + <p> + For these reasons, the <code>ll_rw_block()</code> function has been replaced + with <code>submit_bio()</code>. + </p> + <h3 id="readpages-prefetching">Readpages (prefetching)</h3> + <p> + SquashFS does not implement <code>.readpages()</code>, so the kernel repeatedly + calls <code>.readpage()</code>. + </p> + <p> + Now that our read requests are asynchronous, the kernel can truly prefetch pages + using its asynchronous read-ahead mechanism. + </p> + <h3 id="optimize-reading-uncompressed-blocks">Optimize reading uncompressed + blocks</h3> + <p> + Modern systems such as Android contain a lot of files that are already + compressed. As a consequence, the image contains a lot of blocks that can't be + compressed. + </p> + <p> + SquashFS handles compressed and uncompressed blocks using the same logic: when + asked to read a single page, it actually reads a full block (default 128k). + While this is necessary for compressed blocks, it is just a waste of resources + for uncompressed blocks. + </p> + <p> + Instead of reading a full block, SquashFS now just reads what is advised by the + readahead algorithm. + </p> + <p> + This greatly improves the performance of random reads. + </p> + <h2 id="code">Code</h2> + <p> + SquashFS code optimizations are available in AOSP: + </p><ul> + <li><a + href="https://android-review.googlesource.com/#/q/topic:squashfs">https://android-review.googlesource.com/#/q/topic:squashfs</a></li></ul> + </body> +</html> |