diff options
| author | Android Partner Docs <noreply@android.com> | 2017-08-22 10:41:24 -0700 |
|---|---|---|
| committer | Clay Murphy <claym@google.com> | 2017-08-22 15:01:44 -0700 |
| commit | f16c42333aa6b2de30a344dd68246d4a33d93e7d (patch) | |
| tree | 311af599312cacb21c888aeae828cae59b0d64a1 /en/devices/architecture | |
| parent | 04426e67ca3ee557a0083f9b3c6ba789021bd7a0 (diff) | |
| download | source.android.com-f16c42333aa6b2de30a344dd68246d4a33d93e7d.tar.gz | |
Docs: Changes to source.android.com
- 166080694 Devsite localized content from translation request a3d5a7... by Android Partner Docs <noreply@android.com>
- 166079245 Remove duplicate TOC entry to oob-users.html. by mheco <mheco@google.com>
- 166002955 Update builds for Oreo by Android Partner Docs <noreply@android.com>
- 165977566 Fixing bad conversion by hvm <hvm@google.com>
- 165977199 Edit links to point to public source files in AOSP. by cqn <cqn@google.com>
- 165962883 Add codename to CTS downloads page. by gdimino <gdimino@google.com>
- 165955117 Integration of O branch into mainline. by gdimino <gdimino@google.com>
- 165638251 Update July public Android security bulletin to remove QC... by Android Partner Docs <noreply@android.com>
- 165638198 Update June public Android security bulletin to remove QC... by Android Partner Docs <noreply@android.com>
- 165638174 Update May public Android security bulletin to remove QC ... by Android Partner Docs <noreply@android.com>
- 165638096 Update April public Android security bulletin to remove Q... by Android Partner Docs <noreply@android.com>
- 165528993 Update to Keymaster 2 and remove requirements language by daroberts <daroberts@google.com>
- 165511119 Add Bluetooth verification / debug information by cqn <cqn@google.com>
- 165491345 Fixed link broken by file rename. by cqn <cqn@google.com>
- 165381648 Fixed broken image paths and renamed HCI Requirements file. by cqn <cqn@google.com>
- 165365185 Created high-level Bluetooth directory and added HTML ver... by cqn <cqn@google.com>
- 165335694 Devsite localized content from translation request 66a39c... by Android Partner Docs <noreply@android.com>
- 165246927 Update August 2017 bulletin with CVE-2017-0687 by daroberts <daroberts@google.com>
PiperOrigin-RevId: 166080694
Change-Id: I2d3a8d77fa6a66c2099f13ba2e864545328fd17a
Diffstat (limited to 'en/devices/architecture')
85 files changed, 12452 insertions, 20 deletions
diff --git a/en/devices/architecture/configstore/add-class-item.html b/en/devices/architecture/configstore/add-class-item.html new file mode 100644 index 00000000..319462b7 --- /dev/null +++ b/en/devices/architecture/configstore/add-class-item.html @@ -0,0 +1,236 @@ +<html devsite> + <head> + <title>Adding ConfigStore Classes & Items</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>You can add new ConfigStore items</a> (i.e., interface methods) for an +existing interface class. If the interface class is not defined, you must add a +new class before you can add a ConfigStore item for that class. This section +uses the example of a <code>disableInitBlank</code> configuration item for +<code>healthd</code> being added to the <code>IChargerConfigs</code> interface +class.</p> + +<p class=note><strong>Note:</strong> Before continuing, ensure you are familiar +with <a href="/devices/architecture/hidl/index.html">general HIDL concepts</a>, +<a href="/devices/architecture/hidl-cpp/index.html">HIDL C++ development +workflow</a>, <a href="/devices/architecture/hidl/code-style.html">HIDL Code +Style</a>, and <a href="/devices/architecture/configstore/index.html"> +ConfigStore design</a>.</p> + +<h2 id=add-class>Adding interface classes</h2> +<p>If no interface class is defined for the interface method you want to add, +you must first add the interface class before you can add the associated +ConfigStore items.</p> + +<ol> +<li>Create a HAL interface file. The ConfigStore version is 1.0, so define +ConfigStore interfaces in <code>hardware/interfaces/configstore/1.0</code>. For +example, in +<strong><code>hardware/interfaces/configstore/1.0/IChargerConfigs.hal</code></strong>: + +<pre class="devsite-click-to-copy"> +package android.hardware.configstore@1.0; + +interface IChargerConfigs { + // TO-BE-FILLED-BELOW +}; +</pre></li> + +<li>Update <code>Android.bp</code> and <code>Android.mk</code> for ConfigStore +shared library and header files to include the new interface HAL. For example: + +<pre class="devsite-click-to-copy"> +<code class=devsite-terminal>hidl-gen -o hardware/interfaces/configstore/1.0/default -Lmakefile -randroid.hardware:hardware/interfaces -randroid.hidl:system/libhidl/transport android.hardware.configstore@1.0::IChargerConfigs</code> +<code class=devsite-terminal>hidl-gen -o hardware/interfaces/configstore/1.0/default -Landroidbp -randroid.hardware:hardware/interfaces -randroid.hidl:system/libhidl/transport android.hardware.configstore@1.0::IChargerConfigs</code> +</pre> +These commands update <code>Android.bp</code> and <code>Android.mk</code> in +<code>hardware/interfaces/configstore/1.0</code>.</li> + +<li>Generate the C++ stub for implementing the server code. For example: + +<pre class="devsite-terminal devsite-click-to-copy"> +hidl-gen -o hardware/interfaces/configstore/1.0/default -Lc++-impl -randroid.hardware:hardware/interfaces -randroid.hidl:system/libhidl/transport android.hardware.configstore@1.0::IChargerConfigs +</pre> +This command creates two files, <code>ChargerConfigs.h</code> and +<code>ChargerConfigs.cpp</code>, in +<code>hardware/interfaces/configstore/1.0/default</code>.</li> + +<li>Open the .h and .cpp implementation files and remove code related to the +function <code>HIDL_FETCH_<em>name</code></em> (e.g., +<code>HIDL_FETCH_IChargerConfigs</code>). This function is needed for HIDL +passthrough mode, which is unused by ConfigStore.</li> + +<li>Register the implementation to the ConfigStore service. For example, in +<strong><code>hardware/interfaces/configstore/1.0/default/service.cpp</code></strong>: + +<pre class="devsite-click-to-copy"> +#include <android/hardware/configstore/1.0/IChargerConfigs.h> +#include "ChargerConfigs.h" + +using android::hardware::configstore::V1_0::IChargerConfigs; +using android::hardware::configstore::V1_0::implementation::ChargerConfigs; + +int main() { + ... // other code + sp<IChargerConfigs> chargerConfigs = new ChargerConfigs; + status = chargerConfigs->registerAsService(); + LOG_ALWAYS_FATAL_IF(status != OK, "Could not register IChargerConfigs"); + ... // other code +} +</pre></li> + +<li>Modify <code>Android.mk</code> file to add implementation file +(<em>modulename</em>Configs.cpp) to LOCAL_SRC_FILES and to map build flags into +macro definitions. For example, in +<strong><code>hardware/interfaces/configstore/1.0/default/Android.mk</code></strong>: + +<pre class="devsite-click-to-copy"> +LOCAL_SRC_FILES += ChargerConfigs.cpp + +ifeq ($(strip $(BOARD_CHARGER_DISABLE_INIT_BLANK)),true) +LOCAL_CFLAGS += -DCHARGER_DISABLE_INIT_BLANK +endif +</pre></li> + +<li>(Optional) Add manifest entry. If it doesn't exist, default to the "default" +instance name of ConfigStore. For example, in +<strong><code>device/google/marlin/manifest.xml</code></strong>: + +<pre class="devsite-click-to-copy"> + <hal format="hidl"> + <name>android.hardware.configstore</name> + ... + <interface> + <name>IChargerConfigs</name> + <instance>default</instance> + </interface> + </hal> +</pre></li> + +<li>Add the sepolicy rule if needed (i.e., if the client does not have +permissions for making hwbinder calls to the <code>hal_configstore</code>). For +example, in <strong><code>system/sepolicy/private/healthd.te</code></strong>: + +<pre class="devsite-click-to-copy"> +... // other rules +binder_call(healthd, hal_configstore) +</pre></li> +</ol> + + +<h2 id=add-item>Adding new ConfigStore items</h2> +<p>To add a new ConfigStore item:</p> +<ol> +<li>Open the HAL file and add required interface method for the item. (The .hal +files for ConfigStore reside in +<code>hardware/interfaces/configstore/1.0</code>.) For example, in +<strong><code>hardware/interfaces/configstore/1.0/IChargerConfigs.hal</code></strong>: + +<pre class="devsite-click-to-copy"> +package android.hardware.configstore@1.0; + +interface IChargerConfigs { + ... // Other interfaces + disableInitBlank() generates(OptionalBool value); +}; +</pre></li> + +<li>Implement the method in the corresponding interface HAL implementation files +(.h and .cpp). Place default implementations in +<code>hardware/interfaces/configstore/1.0/default</code>. + +<p class=note><strong>Note:</strong> Running <code>hidl-gen</code> with +<code>-Lc++-impl</code> generates skeleton code for the newly added interface +method. However, as it also overwrites implementations for all existing +interface methods, use the <code>-o</code> option appropriately.</p> + +For example, in +<strong><code>hardware/interfaces/configstore/1.0/default/ChargerConfigs.h</code></strong>: + +<pre class="devsite-click-to-copy"> +struct ChargerConfigs : public IChargerConfigs { + ... // Other interfaces + Return<void> disableInitBlank(disableInitBlank_cb _hidl_cb) override; +}; +</pre> + +And in +<strong><code>hardware/interfaces/configstore/1.0/default/ChargerConfigs.cpp</code></strong>: + +<pre class="devsite-click-to-copy"> +Return<void> ChargerConfigs::disableInitBlank(disableInitBlank_cb _hidl_cb) { + bool value = false; +#ifdef CHARGER_DISABLE_INIT_BLANK + value = true; +#endif + _hidl_cb({true, value}); + return Void(); +} +</pre></li> +</ol> + +<h2 id=using>Using ConfigStore items</h2> +<p>To use a ConfigStore item:</p> + +<ol> +<li>Include required header files. For example, in +<strong><code>system/core/healthd/healthd.cpp</code></strong>: + +<pre class="devsite-click-to-copy"> +#include <android/hardware/configstore/1.0/IChargerConfigs.h> +#include <configstore/Utils.h> +</pre></li> + +<li>Access the ConfigStore item using the appropriate template function in +<code>android.hardware.configstore-utils</code>. For example, in +<strong><code>system/core/healthd/healthd.cpp</code></strong>: + +<pre class="devsite-click-to-copy"> +using namespace android::hardware::configstore; +using namespace android::hardware::configstore::V1_0; + +static int64_t disableInitBlank = getBool< + IChargerConfigs, + &IChargerConfigs::disableInitBlank>(false); +</pre> +In this example, the ConfigStore item <code>disableInitBlank</code> is retrieved +and stored to a variable (useful when the variable needs to be accessed multiple +times). The value retrieved from the ConfigStore is cached inside the +instantiated template function so it can be retrieved quickly from the cached +value without contacting the ConfigStore service for later calls to the +instantiated template function. +</li> + +<li>Add the dependency on ConfigStore and <code>configstore-utils</code> library +in <code>Android.mk</code> or <code>Android.bp</code>. For example, in +<strong><code>system/core/healthd/Android.mk</code></strong>: + +<pre class="devsite-click-to-copy"> +LOCAL_SHARED_LIBRARIES := \ + android.hardware.configstore@1.0 \ + android.hardware.configstore-utils \ + ... (other libraries) \ +</pre></li> +</ol> + + </body> +</html> diff --git a/en/devices/architecture/configstore/client.html b/en/devices/architecture/configstore/client.html new file mode 100644 index 00000000..7b691994 --- /dev/null +++ b/en/devices/architecture/configstore/client.html @@ -0,0 +1,166 @@ +<html devsite> + <head> + <title>Client-Side Usage</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>You can refactor conditionally-compiled code to read values dynamically from +the HAL interface. For example:</p> + +<pre class="devsite-click-to-copy"> +#ifdef TARGET_FORCE_HWC_FOR_VIRTUAL_DISPLAYS +//some code fragment +#endif +</pre> + +<p>Framework code can then call an appropriate utility function defined in +<code><configstore/Utils.h></code> depending on its type.</p> + +<h2 id=example>ConfigStore example</h2> +<p>This example shows reading +<code>TARGET_FORCE_HWC_FOR_VIRTUAL_DISPLAYS</code>, defined in ConfigStore HAL +as <code>forceHwcForVirtualDisplays()</code> with return type +<code>OptionalBool</code>:</p> + +<pre class="devsite-click-to-copy"> +#include <configstore/Utils.h> +using namespace android::hardware::configstore; +using namespace android::hardware::configstore::V1_0; + +static bool vsyncPhaseOffsetNs = getBool<ISurfaceFlingerConfigs, + ISurfaceFlingerConfigs::forceHwcForVirtualDisplays>(false); +</pre> + +<p>The utility function (<code>getBool</code> in the example above) contacts the +<code>configstore</code> service to get the handle for the proxy of the +interface function, then retrieves the value by invoking the handle via +HIDL/hwbinder.</p> + +<h2 id=utility-functions>Utility functions</h2> +<p><code><configstore/Utils.h></code> +(<code>configstore/1.0/include/configstore/Utils.h</code>) provides utility +functions for each primitive return type, including +<code>Optional[Bool|String|Int32|UInt32|Int64|UInt64]</code>, as listed +below:</p> + +<table> + +<tr> +<th>Type</th> +<th>Function <em>(template parameters omitted)</em></th> +</tr> + +<tr> +<td><code>OptionalBool</code></td> +<td><code>bool getBool(const bool defValue)</code></td> +</tr> + +<tr> +<td><code>OptionalInt32</code></td> +<td><code>int32_t getInt32(const int32_t defValue)</code></td> +</tr> + +<tr> +<td><code>OptionalUInt32</code></td> +<td><code>uint32_t getUInt32(const uint32_t defValue)</code></td> +</tr> + +<tr> +<td><code>OptionalInt64</code></td> +<td><code>int64_t getInt64(const int64_t defValue)</code></td> +</tr> + +<tr> +<td><code>OptionalUInt64</code></td> +<td><code>uint64_t getUInt64(const uint64_t defValue)</code></td> +</tr> + +<tr> +<td><code>OptionalString</code></td> +<td><code>std::string getString(const std::string &defValue)</code></td> +</tr> + +</table> + +<p><code>defValue</code> is a default value returned when the HAL implementation +does not specify a value for the configuration item. Each function takes two +template parameters:</p> +<ul> +<li><code><strong>I</code></strong>. Interface class name.</li> +<li><code><strong>Func</code></strong>. Member function pointer for getting the +configuration item.</li> +</ul> +<p>Because the configuration value is read-only and does not change, the utility +function internally caches the configuration value. Subsequent calls are +serviced more efficiently using the cached value in the same linking unit.</p> + +<h2 id=utils>Using configstore-utils</h2> +<p>The ConfigStore HAL is designed to be forward-compatible for minor version +upgrades, meaning that when the HAL is up-revisioned and some framework code +uses the newly-introduced items, the ConfigStore service with older minor +version in <code>/vendor</code> can still be used.</p> + +<p>For forward-compatibility, ensure your implementation adheres to the +following guidelines:</p> + +<ol> +<li>New items use the default value when <em>only</em> the old version service +is available. Example: + +<pre class="devsite-click-to-copy"> +service = V1_1::IConfig::getService(); // null if V1_0 is installed +value = DEFAULT_VALUE; + if(service) { + value = service->v1_1API(DEFAULT_VALUE); + } +</pre> + +</li> + +<li>Client uses the earliest interface in which the ConfigStore item was +introduced. Example: + +<pre class="devsite-click-to-copy"> +V1_1::IConfig::getService()->v1_0API(); // NOT ALLOWED + +V1_0::IConfig::getService()->v1_0API(); // OK +</pre> +</li> +<li>New version service can be retrieved for old version interface. In the +following example, if the installed version is v1_1, the v1_1 service must be +returned for getService(): + +<pre class="devsite-click-to-copy"> +V1_0::IConfig::getService()->v1_0API(); +</pre> + +<p class=note><strong>Note:</strong> The +<a href="https://android-review.googlesource.com/c/393736/">current AOSP +implementation</a> satisfies this requirement.</p> +</li> +</ol> + +<p>When the access functions in <code>configstore-utils</code> library are used +for accessing the ConfigStore item, #1 is guaranteed by the implementation and +#2 is guaranteed by compiler errors. For these reasons we strongly recommend +using <code>configstore-utils</code> wherever possible.</p> + + </body> +</html> diff --git a/en/devices/architecture/configstore/index.html b/en/devices/architecture/configstore/index.html new file mode 100644 index 00000000..45dcb9e7 --- /dev/null +++ b/en/devices/architecture/configstore/index.html @@ -0,0 +1,108 @@ +<html devsite> + <head> + <title>Configstore HAL</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>Android O splits the monolithic Android OS into generic (system.img) +and hardware-specific (vendor.img and odm.img) partitions. As a result of this +change, conditional compilation must be removed from modules installed to the +system partition and such modules must now determine the configuration of the +system at runtime (and behave differently depending on that configuration).</p> + +<p>The ConfigStore HAL provides a set of APIs for accessing read-only +configuration items used to configure the Android framework. This page describes +the design of ConfigStore HAL (and why system properties were not used for this +purpose); other pages in this section detail the +<a href="/devices/architecture/configstore/interface.html">HAL interface</a>, +<a href="/devices/architecture/configstore/service.html">service +implementation</a>, and +<a href="/devices/architecture/configstore/client.html">client-side usage</a>, +all using <code>surfaceflinger</code> as an example. For help with ConfigStore +interface classes, see +<a href="/devices/architecture/configstore/add-class-item.html">Adding Interface +Classes & Items</a>.</p> + +<h2 id=system-properties>Why not use system properties?</h2> +<p>We considered using system properties but found several fundamental issues, +including: </p> +<ul> +<li><strong>Length limits on values</strong>. System properties have +tight limits on the length of their values (92 bytes). In addition, as these +limits have been directly exposed to Android apps as C macros, increasing the +length can cause backwards-compatibility issues.</li> +<li><strong>No type support</strong>. All values are essentially strings, and +APIs simply parse the string into an <code>int</code> or <code>bool</code>. +Other compound data types (array, struct, etc.) should be encoded/decoded by +the clients (e.g. "aaa,bbb,ccc" can be decoded as an array of three strings). +</li> +<li><strong>Overwrites</strong>. Because read-only system properties are +implemented as write-once properties, vendors/ODMs that want to override +AOSP-defined read-only values must import their own read-only values prior to +AOSP-defined read-only values, which in turn results in vendor-defined +re-writable values being overridden by AOSP-defined values.</li> +<li><strong>Address space requirements</strong>. System properties take a +relatively large amount of address space in each process. System properties are +grouped in <code>prop_area</code> units with a fixed size of 128KB, all of which +is allocated to a process address space even if only a single system property in +it is being accessed. This can cause problems on 32-bit devices where address +space is precious.</li> +</ul> +<p>We attempted to overcome these limitations without sacrificing compatibility +but continued to be concerned that system properties were not designed to +support accessing read-only configuration items. Eventually we decided that +system properties are better suited for sharing a few dynamically-updated items +across all of Android in real time, and that a need existed for a new system +dedicated to accessing read-only configuration items.</p> + +<h2>ConfigStore HAL design</h2> +<p>The basic design is simple:</p> +<p><img src="../images/treble_configstore_design.png"></p> +<p><strong>Figure 1.</strong> ConfigStore HAL design</p> + +<ul> +<li>Describe build flags (currently used for conditionally compiling the +framework) in HIDL.</li> +<li>Vendors and OEMs provide SoC and device-specific values for build flags by +implementing the HAL service.</li> +<li>Modify the framework to use the HAL service to find the value of a +configuration item at runtime.</li> +</ul> + +<p>Configuration items currently referenced by the framework are included in a +versioned HIDL package (<code>android.hardware.configstore@1.0</code>). Vendors +and/or OEMs provide values to the configuration items by implementing interfaces +in this package, and the framework uses the interfaces when it needs to get a +value for a configuration item.</p> + +<h2 id=security>Security considerations</h2> +<p>Build flags defined in the same interface are affected by same SELinux +policy. If one or more build flags should have different SELinux policies, +<strong>they must be separated to another interface</strong>. This can require +major uprev of <code>android.hardware.configstore package</code> as the +separated interfaces are no longer backwards-compatible.</p> + +<aside class="note"><strong>Note:</strong> For details on Android 8.0 SELinux, +see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for Android +8.0</a>.</aside> + + </body> +</html> diff --git a/en/devices/architecture/configstore/interface.html b/en/devices/architecture/configstore/interface.html new file mode 100644 index 00000000..6c735feb --- /dev/null +++ b/en/devices/architecture/configstore/interface.html @@ -0,0 +1,186 @@ +<html devsite> + <head> + <title>Creating the HAL Interface</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>You must use HIDL to describe all build flags used for conditionally +compiling the framework. Relevant build flags must be grouped and included in a +single <code>.hal</code> file. Using HIDL for specifying configuration items +includes the following benefits:</p> +<ul> +<li>Versioned (to add new config items, vendors/OEMs must explicitly extend the +HAL)</li> +<li>Well-documented</li> +<li>Access control using SELinux</li> +<li>Sanity check for configuration items via +<a href="/devices/tech/test_infra/tradefed/fundamentals/vts">Vendor Test +Suite</a> (range check, inter-dependency check among items, etc.)</li> +<li>Auto-generated APIs in both C++ and Java</li> +</ul> + +<h2 id=identify-flags>Identifying build flags used by the framework</h2> +<p>Start by identifying the build configs used to conditionally compile the +framework, then abandon obsolete configs to make the set smaller. For example, +the following set of build flags are identified for <code>surfaceflinger</code>: +</p> +<ul> +<li><code>TARGET_USES_HWC2</code> (will be obsoleted)</li> +<li><code>TARGET_BOARD_PLATFORM</code></li> +<li><code>TARGET_DISABLE_TRIPLE_BUFFERING</code></li> +<li><code>TARGET_FORCE_HWC_FOR_VIRTUAL_DISPLAYS</code></li> +<li><code>NUM_FRAMEBUFFER_SURFACE_BUFFERS</code></li> +<li><code>TARGET_RUNNING_WITHOUT_SYNC_FRAMEWORK</code></li> +<li><code>VSYNC_EVENT_PHASE_OFFSET_NS</code></li> +<li><code>SF_VSYNC_EVENT_PHASE_OFFSET_NS</code> (will be obsoleted)</li> +<li><code>PRESENT_TIME_OFFSET_FROM_VSYNC_NS</code></li> +<li><code>MAX_VIRTUAL_DISPLAY_DIMENSION</code></li> +</ul> + +<h2 id="create-interface">Creating a HAL interface</h2> +<p>Build configs for a subsystem are accessed via a HAL interface, while +interfaces for giving configuration values are grouped in the HAL package <code>android.hardware.configstore</code> (currently at version 1.0). For example, to +create a HAL interface file for <code>surfaceflinger</code>, in +<strong><code>hardware/interfaces/configstore/1.0/ISurfaceFlingerConfigs.hal</code></strong>: +</p> + +<pre class="devsite-click-to-copy"> +package android.hardware.configstore@1.0; + +interface ISurfaceFlingerConfigs { + // TO-BE-FILLED-BELOW +}; +</pre> + +<p>After creating the <code>.hal</code> file, run +<code>hardware/interfaces/update-makefiles.sh</code> to add the new +<code>.hal</code> file to the <code>Android.bp</code> and +<code>Android.mk</code> files.</p> + +<h2 id="add-functions">Adding functions for build flags</h2> +<p>For each build flag, add a new function to the interface. For example, in +<strong><code>hardware/interfaces/configstore/1.0/ISurfaceFlingerConfigs.hal</code></strong>: +</p> + +<pre class="devsite-click-to-copy"> +interface ISurfaceFlingerConfigs { + disableTripleBuffering() generates(OptionalBool ret); + forceHwcForVirtualDisplays() generates(OptionalBool ret); + enum NumBuffers: uint8_t { + USE_DEFAULT = 0, + TWO = 2, + THREE = 3, + }; + numFramebufferSurfaceBuffers() generates(NumBuffers ret); + runWithoutSyncFramework() generates(OptionalBool ret); + vsyncEventPhaseOffsetNs generates (OptionalUInt64 ret); + presentTimeOffsetFromSyncNs generates (OptionalUInt64 ret); + maxVirtualDisplayDimension() generates(OptionalInt32 ret); +}; +</pre> + +<p>When adding a function:</p> +<ul> +<li><strong>Be concise with names</strong>. Avoid converting makefile variable +names into function names and keep in mind that <code>TARGET_</code> and +<code>BOARD_</code> prefixes are no longer necessary.</li> +<li><strong>Add comments</strong>. Help developers understand the purpose of the +config item, how it changes framework behavior, valid values, etc.</li> +</ul> +<p>Function return types can be +<code>Optional[Bool|String|Int32|UInt32|Int64|UInt64]</code>. Types are defined +in <code>types.hal</code> in the same directory and wrap primitive values with a +field that indicates if the value is specified by the HAL; if not, the default +value is used.</p> + +<pre class="devsite-click-to-copy"> +struct OptionalString { + bool specified; + string value; +}; +</pre> + +<p>When appropriate, define the enum that best represents the type of the +configuration item and use that enum as the return type. In the example above, +the <code>NumBuffers</code> enum is defined to limit the number of valid +values. When defining such custom data types, add a field or a enum value (e.g., +<code>USE_DEFAULT</code>) for denoting if the value is/is not specified by +HAL.</p> + +<p>It is not mandatory for a single build flag to become a single function in +HIDL. Module owners can alternatively aggregate closely-related build flags into +a struct and have a function that returns that struct (doing so can reduce +number of function calls).</p> + +<p>For example, an option for aggregating two build flags into a single struct +in <strong><code>hardware/interfaces/configstore/1.0/ISurfaceFlingerConfigs.hal</code></strong> +is:</p> + +<pre class="devsite-click-to-copy"> + interface ISurfaceFlingerConfigs { + // other functions here + struct SyncConfigs { + OptionalInt64 vsyncEventPhaseoffsetNs; + OptionalInt64 presentTimeoffsetFromSyncNs; + }; + getSyncConfigs() generates (SyncConfigs ret); + // other functions here +}; +</pre> + +<h2 id=alternatives>Alternatives to a single HAL function</h2> + +<p>As an alternative to using a single HAL function for all build flags, the HAL +interface also provides simple functions such as <code>getBoolean(string +key)</code> and <code>getInteger(string key)</code>. The actual +<code>key=value</code> pairs are stored in separate files and the HAL service +provides values by reading/parsing those files.</p> + +<p>While this approach is easy to define, it does not include the benefits +provided by HIDL (enforced versioning, ease of documentation, access control) +and is therefore not recommended.</p> + +<p class=note><strong>Note:</strong> When using simple functions, access +control is almost impossible as HAL cannot identify clients by itself.</p> + +<h2 id=single-multiple>Single vs. multiple interfaces</h2> +<p>The design of the HAL interface for configuration items presents two +choices:</p> + +<ol> +<li>Single interface that covers all configuration items</li> +<li>Multiple interfaces, each of which covers a set of related configuration +items</li> +</ol> +<p>A single interface is easier but can become unmaintainable as more +configuration items are added to the single file. In addition, access control +is not fine-grained, so a process granted access to the interface can read all +configuration items (access to a partial set of configuration items cannot be +granted). Alternatively, if access is not granted, no configuration item can be +read.</p> + +<p>Because of these issues, Android uses multiple interfaces with a single HAL +interface for a group of related configuration items. For example, +<code>ISurfaceflingerConfigs</code> for <code>surfaceflinger</code>-related +configuration items, <code>IBluetoothConfigs</code> for Bluetooth-related +configuration items, etc.</p> + + </body> +</html> diff --git a/en/devices/architecture/configstore/service.html b/en/devices/architecture/configstore/service.html new file mode 100644 index 00000000..09874b4d --- /dev/null +++ b/en/devices/architecture/configstore/service.html @@ -0,0 +1,128 @@ +<html devsite> + <head> + <title>Implementing the Service</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>To prepare for the HAL implementation, you can generate basic configstore +interface code then modify it to meet your needs.</p> + +<h2 id=generate-boilerplate>Generating interface code</h2> +<p>To generate boilerplate code for the interface, run <code>hidl-gen</code>. +For example, to generate code for <code>surfaceflinger</code>:</p> + +<pre class="devsite-terminal devsite-click-to-copy"> +hidl-gen -o hardware/interfaces/configstore/1.0/default \ + -Lc++-impl \ + -randroid.hardware:hardware/interfaces \ + -randroid.hidl:system/libhidl/transport \ + android.hardware.config@1.0::ISurfaceFlingerConfigs +</pre> + +<p class="note"><strong>Note:</strong> Don't run <code>hidl-gen</code> with +<code>-Landroidbp-impl</code> as this generates <code>Android.bp</code>. The +module must be built with <code>Android.mk</code> to access build flags.</p> + +<h2 id=modify-androidmk>Modifying Android.mk</h2> +<p>Next, modify <code>Android.mk</code> file to add implementation file +(<code><modulename>Configs.cpp</code>) to <code>LOCAL_SRC_FILES</code> and +to map build flags into macro definitions. For example, you can modify +<code>surfaceflinger</code> in +<strong><code>hardware/interface/configstore/1.0/default/Android.mk</code></strong>: +</p> + +<pre class="devsite-click-to-copy"> +LOCAL_SRC_FILES += SurfaceFlingerConfigs.cpp +ifneq ($(NUM_FRAMEBUFFER_SURFACE_BUFFERS),) + LOCAL_CFLAGS += -DNUM_FRAMEBUFFER_SURFACE_BUFFERS=$(NUM_FRAMEBUFFER_SURFACE_BUFFERS) +endif + +ifeq ($(TARGET_RUNNING_WITHOUT_SYNC_FRAMEWORK),true) + LOCAL_CFLAGS += -DRUNNING_WITHOUT_SYNC_FRAMEWORK +endif +</pre> + +<p>If <code>Android.mk</code> includes several <code>ifeq-endif</code> blocks, +consider moving your code into a new file (i.e., <code>surfaceflinger.mk</code>) +then include that file from <code>Android.mk</code>.</p> + +<h2 id=implement-functions>Implementing functions</h2> +<p>To fill the functions to implement the HAL, call back the +<code>_hidl_cb</code> function with different values (conditioned on build +flags). For example, you can fill the functions for <code>surfaceflinger</code> +in <strong><code>hardware/interfaces/configstore/1.0/default/SurfaceFlingerConfigs.cpp</code></strong>:</p> + +<pre class="devsite-click-to-copy"> +Return<void> SurfaceFlingerConfigs::numFramebufferSurfaceBuffers( + numFramebufferSurfaceBuffers_cb _hidl_cb) { + #if NUM_FRAMEBUFFER_SURFACE_BUFFERS 2 + _hidl_cb(NumBuffers.TWO); + #else if NUM_FRAMEBUFFER_SURFACE_BUFFERS 3 + _hidl_cb(NumBuffers.THREE); + #else + _hidl_cb(NumBuffers.USE_DEFAULT); + #endif +} + +Return<void> SurfaceFlingerConfigs::runWithoutSyncFramework( + runWithoutSyncFramework_cb _hidl_cb) { + #ifdef RUNNING_WITHOUT_SYNC_FRAMEWORK + _hidl_cb({true /* specified */, true /* value */}); + #else + // when macro not defined, we can give any value to the second argument. + // It will simply be ignored in the framework side. + _hidl_cb({false /* specified */, false /* value */}); + #endif +} +</pre> + +<p>Ensure the implementation does not contain a function named +<code>HIDL_FETCH_<interface name></code> (e.g., +<code>HIDL_FETCH_ISurfaceFlingerConfigs</code>). This function is needed for +HIDL passthrough mode, which is unused (and prohibited) by +<code>configstore</code>. ConfigStore must always run in binderized mode.</p> + +<h2 id=register-service>Registering as a service</h2> +<p>Finally, register all interface implementations to the +<code>configstore</code> service. For example, you can register +<code>surfaceflinger</code> implementations in +<strong><code>hardware/interfaces/configstore/1.0/default/service.cpp</code></strong>: +</p> + +<pre class="devsite-click-to-copy"> +configureRpcThreadpool(maxThreads, true); +sp<ISurfaceFlingerConfigs> surfaceFlingerConfigs = new SurfaceFlingerConfigs; +status_t status = surfaceFlingerConfigs->registerAsService(); + +sp<IBluetoothConfigs> bluetoothConfigs = new BluetoothConfigs; +status = bluetoothConfigs->registerAsService(); + +// register more interfaces here +joinRpcThreadpool(); +</pre> + +<h2 id=bootstrapping>Ensuring early access</h2> +<p>To ensure a framework module can get early access the HAL service, the config +HAL service should start as early as possible, just after +<code>hwservicemanager</code> is ready. As the config HAL service does not read +external files, it is expected to be ready quickly after it is launched.</p> + + </body> +</html> diff --git a/en/devices/architecture/dto/compile.html b/en/devices/architecture/dto/compile.html new file mode 100644 index 00000000..65e3cbc2 --- /dev/null +++ b/en/devices/architecture/dto/compile.html @@ -0,0 +1,103 @@ +<html devsite> + <head> + <title>Compiling & Verifying</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>You can use Device Tree Compiler (DTC) to compile the Device Tree Source +files. However, before applying the overlay DT on the target main DT, you should +also verify the result by simulating the behavior of DTO.</p> + +<h2 id=compile>Compiling with DTC</h2> +<p>When using <code>dtc</code> to compile <code>.dts</code>, you must add option +<code>-@</code> to add a <code>__symbols__</code> node in the resulting +<code>.dtbo</code>. The <code>__symbols__</code> node contains a list of all +nodes that are marked with a label, which the DTO library can use for +references.</p> + +<p>Sample command to build main DT <code>.dts</code>:</p> + +<pre class="devsite-terminal"> +dtc -@ -O dtb -o my_main_dt.dtb my_main_dt.dts +</pre> + +<p>Sample command to build the overlay DT <code>.dts</code>:</p> + +<pre class="devsite-terminal"> +dtc -@ -O dtb -o my_overlay_dt.dtbo my_overlay_dt.dts +</pre> + +<p class="note"><strong>Note:</strong> If you encounter the DTC build error: +<code>invalid option --'@'</code>, you might need to update your DTC version. +Upstream of AOSP, the official DTC supports DTO as of +<a href="https://github.com/dgibson/dtc/tree/v1.4.4" class="external">version +1.4.4</a> and most patches are merged after December 2016. For DTO support, we +recommend using the +<code><a href="https://android.googlesource.com/platform/external/dtc/" class="external">external/dtc</code></a> +in AOSP, which is synced with the latest DTC (with DTO patches merged as +needed).</p> + +<h2 id=verify>Verify DTO results on the host</h2> +<p>Verification can help you identify errors that might occur when placing the +overlay DT on the main DT. Before updating the target, you can verify the +result of overlaying DT on the host by simulating the behavior of DTO using +<code>/include/</code> in <code>.dts</code>.</p> + +<p class="note"><strong>Note:</strong> <code>/include/</code> does NOT support +the use of <code>__overlay__</code> in overlay DT sources.</p> + +<p><img src="../images/treble_dto_simulate.png"></p> +<p><strong>Figure 1.</strong> Use syntax <code>/include/</code> to simulate DTO +on the host.</p> + +<ol> +<li>Create a copy of the overlay <code>.dts</code>. In the copy, remove the +first line header. Example: +<pre> +/dts-v1/; +/plugin/; +</pre> +Save the file as <code>my_overlay_dt_wo_header.dts</code> (or any filename you +want).</li> + +<li>Create a copy of the main <code>.dts</code>. In the copy, after the last +line, append the include syntax for the file you created in step 1. For example: +<pre> +/include/ "my_overlay_dt_wo_header.dts" +</pre> +Save the file as <code>my_main_dt_with_include.dts</code> (or any filename you +want).</li> + +<li>Use <code>dtc</code> to compile <code>my_main_dt_with_include.dts</code> to +get the merged DT, which should be the same result as DTO. For example: +<pre class="devsite-terminal"> +dtc -@ -O dtb -o my_merged_dt.dtb my_main_dt_with_include.dts +</pre> +</li> + +<li>Use <code>dtc</code> to dump <code>my_merged_dt.dto</code>. +<pre class="devsite-terminal"> +dtc -O dts -o my_merged_dt.dts my_merged_dt.dtb +</pre> +</li> +</ol> + + </body> +</html> diff --git a/en/devices/architecture/dto/implement.html b/en/devices/architecture/dto/implement.html new file mode 100644 index 00000000..5d10a92c --- /dev/null +++ b/en/devices/architecture/dto/implement.html @@ -0,0 +1,187 @@ +<html devsite> + <head> + <title>Implementing DTOs</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>Implementing DTO involves dividing the device tree, building, partitioning, +and running. After you have a working implementation, you must also maintain +compatibility between the two DTs and determine a strategy for ensuring the +security of each DT partition.</p> + +<h2 id=divide>Dividing the DT</h2> +<p>Start by dividing the device tree into two (2) parts:</p> +<ul> +<li><strong>Main DT</strong>. The SoC-only part and the default configurations, +provided by SoC vendor.</li> +<li><strong>Overlay DT</strong>. The device-specific configurations, provided by +ODM/OEM.</li> +</ul> +<p>After dividing the device trees, you must ensure compatibility between main +DT and overlay DT so that merging main DT and overlay DT results in a complete +DT for the device. For details on DTO format and rules, see +<a href="/devices/architecture/dto/syntax.html">DTO Syntax</a>. For details on +multiple device trees, see +<a href="/devices/architecture/dto/multiple.html">Multiple DTs</a>.</p> + +<h2 id=build>Building main & overlay DTs</h2> +<p>To build the main DT:</p> +<ol> +<li>Compile main DT <code>.dts</code> into <code>.dtb</code> file.</li> +<li>Flash <code>.dtb</code> file into a bootloader runtime-accessible partition +(detailed below).</li> +</ol> + +<p>To build the overlay DT:</p> +<ol> +<li>Compile overlay DT <code>.dts</code> into <code>.dtbo</code> file. While +this file format is the same as the <code>.dtb</code> file formatted as a +flattened device tree, the different file extension distinguishes it from the +main DT. +</li> +<li>Flash <code>.dtbo</code> file into a bootloader runtime-accessible partition +(as detailed below).</li> +</ol> + +<p>For details on compiling with DTC and verifying DTO results on the host, see +<a href="/devices/architecture/dto/compile.html">Compiling & Verifying</a>. +</p> + +<h2 id=partition>Partitioning DTs</h2> +<p>Determine a bootloader runtime-accessible and trusted location in flash +memory to put <code>.dtb</code> and <code>.dtbo</code>.</p> + +<p>Example locations for the main DT:</p> +<ul> +<li>Part of boot partition, appended to the kernel (<code>image.gz</code>).</li> +<li>Separate DT blobs (<code>.dtb</code>) in dedicated partition +(<code>dtb</code>).</li> +</ul> + +<p>Example locations for the overlay DT:</p> + +<div style="width:75%"> +<div class="attempt-left"> +<table><tr><th style="text-align: center;">Unique Partition</th></tr></table> +<figure id="treble_dto_dtbo_partition_1"> +<img src="../images/treble_dto_dtbo_partition_1.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 1.</strong> Put <code>.dtbo</code> into a unique partition, e.g. +<code>dtbo</code> partition. +</figcaption> +</figure> +</div> +<div class="attempt-right"> +<figure id="treble_dto_dtbo_partition_2"> +<table><tr><th style="text-align: center;">ODM Partition</th></tr></table> +<img src="../images/treble_dto_dtbo_partition_2.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 2.</strong> Put <code>.dtbo</code> into <code>odm</code> +partition (do this only if your bootloader has capability to load data from the +filesystem of <code>odm</code> partition). +</figcaption> +</figure> +</div> +</div> + +<p class="note" style="clear:both"><strong>Note:</strong> The size of the +overlay DT partition depends on the device and the amount of changes needed on +top of the main DT blob. Typically, 8 MB is more than enough and allows room to +grow in the future if required.</p> + +<p>For devices that support +<a href="https://source.android.com/devices/tech/ota/ab_updates.html">seamless +(A/B) updates</a>, A/B the main DT and overlay DT partitions:</p> + +<div style="width:75%"> +<div class="attempt-left"> +<table><tr><th style="text-align: center;">Example 1</th></tr></table> +<figure id="treble_dto_dtbo_ab_1"> +<img src="../images/treble_dto_dtbo_ab_1.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 3.</strong> DTBO partition A/B, example 1. +</figcaption> +</figure> +</div> +<div class="attempt-right"> +<table><tr><th style="text-align: center;">Example 2</th></tr></table> +<figure id="treble_dto_dtbo_ab_2"> +<img src="../images/treble_dto_dtbo_ab_2.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 4.</strong> DTBO partition A/B, example 2. +</figcaption> +</figure> +</div> +</div> + +<h2 id=run style="clear:both">Running in bootloader</h2> +<p>To run:</p> + +<figure id=treble_dto_dtbo> +<img src="../images/treble_dto_dtbo.png"> +<figcaption><strong>Figure 5.</strong> Typical runtime implementation for device +tree overlay in bootloader.</figcaption> +</figure> + +<ol> +<li>Load <code>.dtb</code> from storage into memory.</li> +<li>Load <code>.dtbo</code> from storage into memory.</li> +<li>Overlay <code>.dtb</code> with <code>.dtbo</code> to be a merged DT.</li> +<li>Start kernel given the memory address of the merged DT.</li> +</ol> + +<h2 id=maintain>Maintaining compatibility</h2> +<p>The main DTB (from SoC vendor) is treated as an API surface for DTBOs. After +separating the device tree into a SoC-common part and a device-specific part, +you must keep the two parts mutually compatible in the future, including:</p> + +<ul> +<li><strong>DT definition in main DT</strong> <em>(e.g. nodes, properties, +labels)</em>. Any definition change in main DT could trigger changes in overlay +DT. For example, to correct a node name in main DT, define an "alias" label that +maps to the original node name (to avoid the change of overlay DT).</li> +<li><strong>Overlay DT store location</strong> <em>(e.g. partition name, store +format)</em>.</li> +</ul> + +<h2 id=security>Ensuring security</h2> +<p>Bootloader must ensure the DTB/DTBO is secure, unmodified, and uncorrupted. +You could use any solution to secure DTB/DTBO, for example, +<a href="/security/verifiedboot/verified-boot#signature_format">Boot image +signature</a> in VBoot 1.0 or +<a href="https://android.googlesource.com/platform/external/avb/+/master/README.md#The-VBMeta-struct" class="external">AVB HASH footer</a> (VBoot 2.0). +</p> + +<ul> +<li>If DTB/DTBO is in a unique partition, you can add that partition to the +trust chain of AVB. The trust chain starts from a hardware-protected root of +trust and goes to the bootloader, which verifies the integrity and authenticity +of DTB/DTBO partition.</li> +<li>If DTB/DTBO is in an existing partition (such as the <code>odm</code> +partition), that partition should be in the trust chain of AVB. (DTBO partition +could share a public key with <code>odm</code> partition).</li> +</ul> + +<p>For details, refer to <a href="/security/verifiedboot/index.html">Verified +Boot</a>.</p> + + </body> +</html> diff --git a/en/devices/architecture/dto/index.html b/en/devices/architecture/dto/index.html new file mode 100644 index 00000000..3fdf3c28 --- /dev/null +++ b/en/devices/architecture/dto/index.html @@ -0,0 +1,144 @@ +<html devsite> + <head> + <title>Device Tree Overlays</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>A device tree (DT) is a data structure of named nodes and properties that +describe non-discoverable hardware. Operating systems, such as the Linux kernel +used in Android, use DTs to support a wide range of hardware configurations used +by Android-powered devices. Hardware vendors supply their own DT source files, +which Linux then compiles into the Device Tree Blob (DTB) file used by the +bootloader.</p> + +A <a href="https://lkml.org/lkml/2012/11/5/615" class="external">device tree +overlay</a> (DTO) enables a central DTB to be overlaid on the device tree. A +bootloader using DTO can maintain the system-on-chip (SoC) DT and dynamically +overlay a device-specific DT, adding nodes to the tree and making changes to +properties in the existing tree.</p> + +<p>This page details a typical bootloader workflow for loading a DT and provides +a list of common DT terms. Other pages in this section describe how to +<a href="/devices/architecture/dto/implement.html">implement bootloader support +for DTO</a>, how to +<a href="/devices/architecture/dto/compile.html">compile</a>, verify, and +<a href="/devices/architecture/dto/optimize.html">optimize your DTO +implementation</a>, and how to +<a href="/devices/architecture/dto/multiple.html">use multiple DTs</a>. You can +also get details on <a href="/devices/architecture/dto/syntax.html">DTO +syntax</a> and recommended +<a href="/devices/architecture/dto/partition.html">DTO/DTBO partition +formatting</a>.</p> + +<h2 id=load-dt>Loading a device tree</h2> +<p>Loading a device tree in bootloader involves building, partitioning, and +running.</p> + +<figure id="treble_dto_bootloader"> +<img src="../images/treble_dto_bootloader.png"> +<figcaption><strong>Figure 1.</strong> Typical implementation for loading device +tree in bootloader.</figcaption> +</figure> + +<ol> +<li>To build: +<ul> +<li>Use the device tree compiler (<code>dtc</code>) to compile device tree +source (<code>.dts</code>) into a device tree blob (<code>.dtb</code>), +formatted as a flattened device tree.</li> +<li>Flash the <code>.dtb</code> file into a bootloader runtime-accessible +location (detailed below).</li> +</ul> +</li> +<li>To partition, determine a bootloader runtime-accessible and trusted location +in flash memory to put <code>.dtb</code>. Example locations: + +<div style="width:75%"> +<div class="attempt-left"> +<table><tr><th style="text-align: center;">Boot Partition</th></tr></table> +<figure id="treble_dto_partition_1"> +<img src="../images/treble_dto_partition_1.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 2.</strong> Put <code>.dtb</code> in boot partition by appending +to <code>image.gz</code> and passing as "<code>kernel</code>" to +<code>mkbootimg</code>. +</figcaption> +</figure> +</div> +<div class="attempt-right"> +<table><tr><th style="text-align: center;">Unique Partition</th></tr></table> +<figure id="treble_dto_partition_2"> +<img src="../images/treble_dto_partition_2.png" style="display:block; margin:0 auto"> +<figcaption> +<strong>Figure 3.</strong> Put <code>.dtb</code> in an unique partition (e.g. +<code>dtb</code> partition). +</figcaption> +</figure> +</div> +</div> +</li> + +<li style="clear:both">To run: +<ul> +<li>Load <code>.dtb</code> from storage into memory.</li> +<li>Start kernel given the memory address of the loaded DT.</li> +</ul> +</li> +</ol> + +<h2 id=terms>Terminology</h2> +<p>This section uses the following device tree terms:</p> +<table> +<tbody> +<tr> +<th>DT</th> +<td>Device Tree</td> +</tr> +<tr> +<th>DTB</th> +<td>Device Tree Blob</td> +</tr> +<tr> +<th>DTBO</th> +<td>Device Tree Blob for Overlay</td> +</tr> +<tr> +<th>DTC</th> +<td>Device Tree Compiler</td> +</tr> +<tr> +<th>DTO</th> +<td>Device Tree Overlay</td> +</tr> +<tr> +<th>DTS</th> +<td>Device Tree Source</td> +</tr> +<tr> +<th>FDT</th> +<td>Flattened Device Tree, a binary format contained in a <code>.dtb</code> blob +file</td> +</tr> +</tbody> +</table> + + </body> +</html> diff --git a/en/devices/architecture/dto/multiple.html b/en/devices/architecture/dto/multiple.html new file mode 100644 index 00000000..52f47a46 --- /dev/null +++ b/en/devices/architecture/dto/multiple.html @@ -0,0 +1,75 @@ +<html devsite> + <head> + <title>Using Multiple DTs</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Many SoC vendors and ODMs support the use of multiple DTs on a device, +enabling one image to power multiple SKUs/configurations. In such cases, the +bootloader identifies the hardware and loads the corresponding DT at runtime: +</p> + +<p><img src="../images/treble_dto_multiple_dt.png"></p> +<figcaption><strong>Figure 1.</strong> Multiple device trees overlay in +bootloader.</figcaption> + +<p class="note"><strong>Note:</strong> Using multiple DTs is not mandatory.</p> + +<h2 id=setting>Setting up</h2> +<p>To add support for multiple DTs to the DTO model, set up a list of main DTs +and another list of overlay DTs.</p> + +<p><img src="../images/treble_dto_multiple_dt_runtime.png"></p> +<figcaption><strong>Figure 2.</strong> Runtime DTO implementation for multiple +DTs.</figcaption> + +<p>The bootloader should be able to:</p> + +<ul> +<li>read the SoC ID and select the main DT correspondingly, and</li> +<li>read the board ID and select the overlay DT accordingly.</li> +</ul> + +<p>Only one main DT and one overlay DT are selected for use at runtime, and the +selected pair must be compatible.</p> + +<h2 id=partition>Partitioning</h2> +<p>To partition, determine a bootloader runtime-accessible and trusted location +in flash memory to store the DTBs and DTBOs (bootloader must be able to locate +these files in the matching process). Keep in mind that DTBs and DTBOs can not +exist in the same partition. If your DTBs/DTBOs are in the +<code>dtb</code>/<code>dtbo</code> partition, use the table structure and header +format detailed in <a href="/devices/architecture/dto/partitions.html">DTB/DTBO +Partition Format</a>.</p> + +<h2 id=runtime>Running in bootloader</h2> +<p>To run:</p> +<ol> +<li><strong>Identify the SoC</strong> and load the corresponding .dtb from +storage into memory.</li> +<li><strong>Identify the board</strong> and load the corresponding +<code>.dtbo</code> from storage into memory.</li> +<li>Overlay the <code>.dtb</code> with the <code>.dtbo</code> to be a merged +DT.</li> +<li>Start kernel given the memory address of the merged DT.</li> +</ol> + + </body> +</html> diff --git a/en/devices/architecture/dto/optimize.html b/en/devices/architecture/dto/optimize.html new file mode 100644 index 00000000..7502d7ad --- /dev/null +++ b/en/devices/architecture/dto/optimize.html @@ -0,0 +1,299 @@ +<html devsite> + <head> + <title>Optimizing DTOs</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This page details optimizations you can make to your DTO implementation, +describes restrictions against overlaying the root node, and provides sample +implementation instructions and code.</p> + +<h2 id=kernel>Kernel command line</h2> +<p>The original kernel command line in device tree is located in the +<code>chosen/bootargs</code> node. The bootloader must concatenate this location +with other sources of kernel command line:</p> +<pre class="prettyprint"> +/dts-v1/; + +/ { + chosen: chosen { + bootargs = "..."; + }; +}; +</pre> + +<p>DTO <strong>cannot</strong> concatenate values from main DT and overlay DT. +We recommend putting the kernel command line of the main DT in +<code>chosen/bootargs</code> and the kernel command line of the overlay DT in +<code>chosen/bootargs_ext</code>. Bootloader can then concatenate these +locations and pass the result to the kernel.</p> + +<table> +<tr> +<th width="50%">main.dts</th> +<th>overlay.dts</th> +</tr> +<tr> +<td> +<pre class="prettyprint"> +/dts-v1/; + +/ { + chosen: chosen { + bootargs = "..."; + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +/dts-v1/; +/plugin/; + +&chosen { + bootargs_ext = "..."; +}; +</pre> +</td> +</tr> +</table> + +<h2 id=libufdt>libufdt</h2> +<p>While the latest +<code><a href="https://github.com/dgibson/dtc/tree/master/libfdt" class="external">libfdt</code></a> +supports DTO, we recommend using <code>libufdt</code> to implement DTO (source +at +<code><a href="https://android.googlesource.com/platform/system/libufdt/+/refs/heads/master" class="external">platform/system/libufdt</code></a> +in AOSP). <code>libufdt</code> builds a real tree structure (un-flattened device +tree, or <em>ufdt</em>) from the flattened device tree (FDT), so it can improve +the merging of two <code>.dtb</code> files from O(N2) to O(N), where N is the +number of nodes in the tree.</p> + +<h3 id=performance>Performance testing</h3> +<p>In Google's internal testing, using <code>libufdt</code> on 2405 +<code>.dtb</code> and 283 <code>.dtbo</code> DT nodes results in file sizes of +70,618 and 8,566 bytes after compilation. Compared with a +<a href="http://fxr.watson.org/fxr/source/boot/fdt/" class="external">DTO +implementation</a> ported from FreeBSD (124ms runtime), <code>libufdt</code> +DTO runtime is 10ms.</p> + +<p>In performance testing for Pixel devices, we compared <code>libufdt</code> +and <code>libfdt</code>. The number of base nodes effect is similar, but +includes the following differences:</p> +<ul> +<li>500 overlay (append or override) operations have 6~8x time difference</li> +<li>1000 overlay (append or override) operations have 8~10x time difference</li> +</ul> + +<p>Example with appending count set to X:</p> +<p><img src="../images/treble_dto_appending.png"></p> +<figcaption><strong>Figure 1.</strong> Appending count is X.</figcaption> + +<p>Example with overriding count set to X:</p> +<p><img src="../images/treble_dto_overriding.png"></p> +<figcaption><strong>Figure 2.</strong> Overriding count is X.</figcaption> + +<p><code>libufdt</code> is developed with some <code>libfdt</code> APIs and data +structures. When using <code>libufdt</code>, you must include and link +<code>libfdt</code> (however in your code you can use <code>libfdt</code> API to +operate DTB or DTBO).</p> + +<h3 id=api>libufdt DTO API</h3> +<p>The main API to DTO in <code>libufdt</code> is as follows:</p> +<pre class="prettyprint"> +struct fdt_header *ufdt_apply_overlay( + struct fdt_header *main_fdt_header, + size_t main_fdt_size, + void *overlay_fdt, + size_t overlay_size); +</pre> + +<p>The parameter <code>main_fdt_header</code> is the main DT and +<code>overlay_fdt</code> is the buffer containing the contents of a +<code>.dtbo</code> file. The return value is a new buffer containing the merged +DT (or <code>null</code> in case of error). The merged DT is formated in FDT, +which you can pass to the kernel when starting the kernel.</p> + +<p>The new buffer from the return value is created by <code>dto_malloc()</code>, +which you should implement when porting <code>libufdt</code> into bootloader. +For reference implementations, refer to +<code>sysdeps/libufdt_sysdeps_*.c</code>.</p> + +<h2 id=root>Root node restrictions</h2> +<p>You cannot overlay a new node or property into the root node of main DT +because overlay operations rely on labels. Because the main DT must define a +label and the overlay DT assigns the nodes to be overlaid with labels, we +cannot give a label for the root node (and therefore cannot overlay the root +node).</p> + +<p>SoC vendors must define the overlaying ability of main DT; ODM/OEMs can only +append or override nodes with labels defined by the SoC vendor. As a workaround, +you can define a <strong><code>odm</code></strong> node under the root node in +base DT, enabling all ODM nodes in overlay DT to add new nodes. Alternatively, +you could put all SoC-related nodes in the base DT into a +<strong><code>soc</code></strong> node under root node as described below:</p> + +<table> +<tr> +<th width="50%">main.dts</th> +<th>overlay.dts</th> +</tr> +<tr> +<td> +<pre> +/dts-v1/; + +/ { + compatible = "corp,bar"; + ... + + chosen: chosen { + bootargs = "..."; + }; + + /* nodes for all soc nodes */ + soc { + ... + soc_device@0: soc_device@0 { + compatible = "corp,bar"; + ... + }; + ... + }; + + odm: odm { + /* reserved for overlay by odm */ + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +/dts-v1/; +/plugin/; + +/ { +}; + +&chosen { + bootargs_ex = "..."; +}; + +&odm { + odm_device@0 { + ... + }; + ... +}; +</pre> +</td> +</tr> +</table> + +<h2 id=sample>Sample DTO implementation</h2> +<p>The following instructions walk you through a sample implementation of DTO +with <code>libufdt</code> (sample code below).</p> + +<h3 id=sample-instructions>Sample DTO instructions</h3> + +<ol> +<li>Include libraries. To use <code>libufdt</code>, include <code>libfdt</code> +for data structures and APIs: +<pre class="prettyprint"> +#include <libfdt.h> +#include <ufdt_overlay.h> +</pre> +</li> + +<li>Load main DT and overlay DT. Load <code>.dtb</code> and <code>.dtbo</code> +from storage into memory (exact steps depend on your design). At this point, you +should have the buffer and size of <code>.dtb</code>/<code>.dtbo</code>: +<pre class="prettyprint"> +main_size = my_load_main_dtb(main_buf, main_buf_size) +</pre> +<pre class="prettyprint"> +overlay_size = my_load_overlay_dtb(overlay_buf, overlay_buf_size); +</pre> +</li> + +<li>Overlay the DTs: +<ol> + +<li>Use <code>ufdt_install_blob()</code> to get the FDT header for main DT: +<pre class="prettyprint"> +main_fdt_header = ufdt_install_blob(main_buf, main_size); +main_fdt_size = main_size; +</pre> +</li> +<li>Call <code>ufdt_apply_overlay()</code> to DTO to get a merged DT in FDT +format: +<pre class="prettyprint"> +merged_fdt = ufdt_apply_overlay(main_fdt_header, main_fdt_size, + overlay_buf, overlay_size); +</pre> +</li> + +<li>To get the size of <code>merged_fdt</code>, use <code>dtc_totalsize()</code>: +<pre class="prettyprint"> +merged_fdt_size = dtc_totalsize(merged_fdt); +</pre> +</li> + +<li>Pass merged DT to start kernel. When you start the kernel, pass merged DT to +kernel: +<pre class="prettyprint"> +my_kernel_entry(0, machine_type, merged_fdt); +</pre> +</li> +</ol></li></ol> + +<h3 id=sample-code>Sample DTO code</h3> +<pre class="prettyprint"> +#include <libfdt.h> +#include <ufdt_overlay.h> + +… + +{ + struct fdt_header *main_fdt_header; + struct fdt_header *merged_fdt; + + /* load main dtb into memory and get the size */ + main_size = my_load_main_dtb(main_buf, main_buf_size); + + /* load overlay dtb into memory and get the size */ + overlay_size = my_load_overlay_dtb(overlay_buf, overlay_buf_size); + + /* overlay */ + main_fdt_header = ufdt_install_blob(main_buf, main_size); + main_fdt_size = main_size; + merged_fdt = ufdt_apply_overlay(main_fdt_header, main_fdt_size, + overlay_buf, overlay_size); + merged_fdt_size = dtc_totalsize(merged_fdt); + + /* pass to kernel */ + my_kernel_entry(0, machine_type, merged_fdt); +} +</pre> + + </body> +</html> diff --git a/en/devices/architecture/dto/partitions.html b/en/devices/architecture/dto/partitions.html new file mode 100644 index 00000000..bcbea204 --- /dev/null +++ b/en/devices/architecture/dto/partitions.html @@ -0,0 +1,307 @@ +<html devsite> + <head> + <title>DTB/DTBO Partitions</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>If your DTB/DTBO is in a unique partition, e.g. <code>dtb</code> and +<code>dtbo</code> partition, use the following table structure and header +format:</p> + +<p><img src="../images/treble_dto_partition_layout.png"></p> +<figcaption><strong>Figure 1.</strong> Example +<code>dtb</code>/<code>dtbo</code> partition layout (for AVB signature, see +<a href="/devices/architecture/dto/implement.html#security">Security</a>). +</figcaption> + +<h2 id=structures>Data structures</h2> +<p>The <code>dt_table_header</code> is <strong>only</strong> for the +<code>dtb</code>/<code>dtbo</code> partition; you CANNOT append this format +after the end of <code>image.gz</code>. If you have a single DTB/DTBO, you must +still use this format (and the <code>dt_entry_size</code> in +<code>dt_table_header</code> is 1).</p> + +<pre class="prettyprint"> +#define DT_TABLE_MAGIC 0xd7b7ab1e + +struct dt_table_header { + uint32_t magic; // DT_TABLE_MAGIC + uint32_t total_size; // includes dt_table_header + all dt_table_entry + // and all dtb/dtbo + uint32_t header_size; // sizeof(dt_table_header) + + uint32_t dt_entry_size; // sizeof(dt_table_entry) + uint32_t dt_entry_count; // number of dt_table_entry + uint32_t dt_entries_offset; // offset to the first dt_table_entry + // from head of dt_table_header + + uint32_t page_size; // flash page size we assume + uint32_t reserved[1]; // must be zero +}; + +struct dt_table_entry { + uint32_t dt_size; + uint32_t dt_offset; // offset from head of dt_table_header + + uint32_t id; // optional, must be zero if unused + uint32_t rev; // optional, must be zero if unused + uint32_t custom[4]; // optional, must be zero if unused +}; +</pre> + +<p>To read all <code>dt_table_entry</code>, use the <code>dt_entry_size</code>, +<code>dt_entry_count</code>, and <code>dt_entries_offset</code>. Example:</p> +<pre class="prettyprint"> +my_read(entries_buf, + header_addr + header->dt_entries_offset, + header->dt_entry_size * header->dt_entry_count); +</pre> + +<p>The <code>id</code>, <code>rev</code>, <code>custom</code> in +<code>dt_table_entry</code> are optional hardware identifications of the device +tree the bootloader can use to efficiently identify the DTB/DTBO to load. If the +bootloader requires additional information, put it in the DTB/DTBO where +bootloader can read it by parsing DTB/DTBO (see the sample code below).</p> + +<h2 id=sample-code>Sample code</h2> +<p>The following sample code checks the hardware identification in bootloader. +</p> + +<ul> +<li>The <code>check_dtbo()</code> function checks the hardware identification. +It first checks the data in struct <code>dt_table_entry</code> (<code>id</code>, +<code>rev</code>, etc.). If this data is not enough, it loads <code>dtb</code> +data into memory and checks the value in <code>dtb</code>.</li> +<li>The values of <code>my_hw_information</code> and <code>soc_id</code> +properties are parsed in the root node (example in <code>my_dtbo_1.dts</code>). + +<pre class="prettyprint"> +[my_dtbo_1.dts] +/dts-v1/; +/plugin/; + +/ { + /* As DTS design, these properties only for loader, won't overlay */ + compatible = "board_manufacturer,board_model"; + + /* These properties are examples */ + board_id = <0x00010000>; + board_rev = <0x00010001>; + another_hw_information = "some_data"; + soc_id = <0x68000000>; + ... +}; + +&device@0 { + value = <0x1>; + status = "okay"; +}; + + +[my_bootloader.c] +int check_dtbo(const dt_table_entry *entry, uint32_t header_addr) { + ... + if (entry->id != ... || entry->rev != ...) { + ... + } + ... + void * fdt_buf = my_load_dtb(header_addr + entry->dt_offset, entry->dt_size); + int root_node_off = fdt_path_offset(fdt_buf, "/"); + ... + const char *my_hw_information = + (const char *)fdt_getprop(fdt_buf, root_node_off, "my_hw_information", NULL); + if (my_hw_information != NULL && strcmp(my_hw_information, ...) != 0) { + ... + } + const fdt32_t *soc_id = fdt_getprop(fdt_buf, root_node_off, "soc_id", NULL); + if (soc_id != NULL && *soc_id != ...) { + ... + } + ... +} +</pre></li></ul> + +<h2 id=mkdtimg>mkdtimg</h2> +<p><code>mkdtimg</code> is a tool for creating +<code>dtb</code>/<code>dtbo</code> images +(<a href="https://android-review.googlesource.com/#/q/topic:mkdtimg+(status:open+OR+status:merged+OR+status:pending)" class="external">source +code</a> at <code>system/libufdt</code> in AOSP). <code>mkdtimg</code> supports +several commands, including <code>create</code>, <code>cfg_create</code>, and +<code>dump</code>.</p> + +<h3 id=create>create</h3> +<p>Use the <code>create</code> command to create a +<code>dtb</code>/<code>dtbo</code> image:</p> +<pre class="prettyprint"> +$mkdtimg create <image_filename> (<global-option>...) \ + <ftb1_filename> (<entry1_option>...) \ + <ftb2_filename> (<entry2_option>...) \ + ... +</pre> + +<p><code>ftbX_filename</code> generates a <code>dt_table_entry</code> in the +image. <code>entryX_option</code>s are the values to assign to +<code>dt_table_entry</code>. These values can be any of the following:</p> +<pre class="prettyprint"> +--id=<number|path> +--rev=<number|path> +--custom0=<number|path> +--custom1=<number|path> +--custom2=<number|path> +--custom3=<number|path> +</pre> + +<p>Number values can be a 32-bit digit (such as 68000) or a hex number (such as +0x6800). Alternatively, you can specify a path using the format:</p> +<pre class="prettyprint"> +<full_node_path>:<property_name> +</pre> + +<p>For example, <code>/board/:id</code>. <code>mkdtimg</code> reads the value +from the path in the DTB/DTBO file and assigns the value (32-bit) to a relative +property in <code>dt_table_entry</code>. Alternatively, you can give a +<code>global_option</code> as a default option for all entries. The default +value of <code>page_size</code> in <code>dt_table_header</code> is 2048; use +<code>global_option --page_size=<number></code> to assign a different +value.</p> + +<p>Example:</p> +<pre class="prettyprint"> +[board1.dts] +/dts-v1/; +/plugin/; + +/ { + compatible = "board_manufacturer,board_model"; + board_id = <0x00010000>; + board_rev = <0x00010001>; + another_hw_information = "some_data"; + ... +}; + +&device@0 { + value = <0x1>; + status = "okay"; +}; + + +$mkdtimg create dtbo.img --id=/:board_id --custom0=0xabc \ + board1.dtbo \ + board2.dtbo --id=0x6800 \ + board3.dtbo --id=0x6801 --custom0=0x123 +</pre> + +<ul> +<li>First <code>dt_table_entry</code> (<code>board1.dtbo</code>) <code>id</code> +is <code>0x00010000</code> and <code>custom[0]</code> is +<code>0x00000abc</code>.</li> +<li>Second <code>id</code> is <code>0x00006800</code> and +<code>custom[0]</code> is <code>0x00000abc</code>.</li> +<li>Third <code>id</code> is <code>0x00006801</code> and <code>custom[0]</code> +is <code>0x00000123</code>.</li> +<li>All others use the default value (<code>0</code>).</li> +</ul> + +<h3 id=cfg-create>cfg_create</h3> +<p>The <code>cfg_create</code> command creates an image with a config file in +the following format:</p> +<pre class="prettyprint"> +# global options + <global_option> + ... +# entries +<ftb1_filename> # comment + <entry1_option> # comment + ... +<ftb2_filename> + <entry2_option> + ... +... +</pre> + +<p>Options <code>global_option</code> and <code>entryX_option</code> must start +with one or more space characters (these options are the same as +<code>create</code> options, without the <code>--</code> prefix). Empty lines or +lines beginning with <code>#</code> are ignored.</p> + +<p>Example:</p> +<pre class="prettyprint"> +[dtboimg.cfg] +# global options + id=/:board_id + rev=/:board_rev + custom0=0xabc + +board1.dtbo + +board2.dtbo + id=0x6800 # override the value of id in global options + +board2.dtbo + id=0x6801 # override the value of id in global options + custom0=0x123 # override the value of custom0 in global options + + +$mkdtimg cfg_create dtbo.img dtboimg.cfg +</pre> + +<p><code>mkdtimg</code> does not handle alignment for +<code>.dtb</code>/<code>.dtbo</code> files but rather appends them to the image. +When you use <code>dtc</code> to compile <code>.dts</code> to +<code>.dtb</code>/<code>.dtbo</code>, you must add option <code>-a</code>. For +example, adding the option <code>-a 4</code> adds padding so the size of +<code>.dtb</code>/<code>.dtbo</code> aligns to 4 bytes.</p> + +<p>Several DT table entries can share a <code>.dtb</code>/<code>.dtbo</code>. If +you use the same filename for different entries, it stores only one content in +the image with same <code>dt_offset</code> and <code>dt_size</code>. This is +useful when using different hardware with identical DTs.</p> + +<h3 id=dump>dump</h3> +<p>For <code>dtb</code>/<code>dtbo</code> images, use the <code>dump</code> +command to print the information in the image. Example:</p> +<pre class="prettyprint"> +$mkdtimg dump dtbo.img +dt_table_header: + magic = d7b7ab1e + total_size = 1300 + header_size = 32 + dt_entry_size = 32 + dt_entry_count = 3 + dt_entries_offset = 32 + page_size = 2048 + reserved[0] = 00000000 +dt_table_entry[0]: + dt_size = 380 + dt_offset = 128 + id = 00010000 + rev = 00010001 + custom[0] = 00000abc + custom[1] = 00000000 + custom[2] = 00000000 + custom[3] = 00000000 + (FDT)size = 380 + (FDT)compatible = board_manufacturer,board_model +... +</pre> + + </body> +</html> diff --git a/en/devices/architecture/dto/syntax.html b/en/devices/architecture/dto/syntax.html new file mode 100644 index 00000000..64981420 --- /dev/null +++ b/en/devices/architecture/dto/syntax.html @@ -0,0 +1,328 @@ +<html devsite> + <head> + <title>DTO Syntax</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Device Tree Source (DTS) format is a textual representation of a device tree. +The Device Tree Compiler (DTC) processes this format into a binary device tree, +which is the form expected by the Linux kernel.</p> + +<h2 id=reference>Using references</h2> + +<p>The <a href="https://github.com/pantoniou/dtc" class="external">DTC</a> +(Device Tree compiler + overlay patches) project describes the DTS format in +<a href="https://android.googlesource.com/platform/external/dtc/+/refs/heads/master/Documentation/dts-format.txt" class="external">dtc-format.txt</a> +and +<a href="https://android.googlesource.com/platform/external/dtc/+/refs/heads/master/Documentation/manual.txt" class="external">manual.txt</a>. +DTO format and rules are described in +<a href="https://android.googlesource.com/platform/external/dtc/+/refs/heads/master/Documentation/dt-object-internal.txt" class="external">dt-object-internal.txt</a>. +These documents describe how to update the main DT using node +<code>fragment@x</code> and syntax <code>__overlay__</code> in overlay DT. For +example:</p> +<pre class="prettyprint"> +/ { + fragment@0 { + target = <&some_node>; + __overlay__ { + some_prop = "okay"; + ... + }; + }; +}; +</pre> + +<p>However, Google strongly recommends you do <strong>not</strong> use +<code>fragment@x</code> and syntax <code>__overlay__</code>, and instead use the +reference syntax. For example:</p> +<pre class="prettyprint"> +&some_node { + some_prop = "okay"; + ... +}; +</pre> + +<p>Reference syntax is compiled by <code>dtc</code> into the same object as the +above using syntax <code>__overlay__</code>. This syntax does not force you to +number the fragments, enabling you to read and write overlay DTS easily. If your +<code>dtc</code> doesn't support this syntactic sugar, use the +<a href="https://android.googlesource.com/platform/external/dtc" class="external">dtc +in AOSP</a>.</p> + +<h2 id=labels>Using labels</h2> +<p>To allow undefined references to nodes not present at compilation time, the +overlay DT <code>.dts</code> file must have a tag <code>/plugin/</code> in its +header. For example:</p> + +<pre class="prettyprint"> +/dts-v1/; +/plugin/; +</pre> + +<p>From here you can target the nodes to be overlaid using a reference, which is +an absolute node path prefixed with an ampersand (&). For example, for +<code>node@0</code> in the main DT:</p> + +<table> +<tr> +<th width="50%">Define labels in the main DT ...</th> +<th>... then use the labels.</th> +</tr> + +<tr> +<td> +<pre class="prettyprint"> +[my_main_dt.dts] + +/dts-v1/; + +/ { + my_node: node@0 { + status = "disabled"; + + my_child: child@0 { + value = <0xffffffff>; + }; + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +[my_overlay_dt.dts] + +/dts-v1/; +/plugin/; + +&my_node { + status = "okay"; +}; + +&my_child { + value = <0x1>; +}; +</pre> +</td> +</tr> +</table> + +<h2 id=override>Overriding</h2> +<p>If the reference target property exists in the main DT, it is overridden +after DTO; otherwise, it is appended. For example:</p> + +<table> +<tr> +<th width="33%">main.dts</th> +<th width="33%">overlay.dts</th> +<th>Merged Result</th> +</tr> + +<tr> +<td> +<pre class="prettyprint"> +[my_main_dt.dts] + +/dts-v1/; + +/ { + compatible = "corp,foo"; + + my_node: node@0 { + status = "disabled"; + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +[my_overlay_dt.dts] + +/dts-v1/; +/plugin/; + +&my_node { + status = "okay"; +}; +</pre> +</td> + +<td> +<pre class="prettyprint"> +/dts-v1/; + +/ { + compatible = "corp,foo"; + + ... + + node@0 { + linux,phandle = <0x1>; + phandle = <0x1>; + status = "okay"; + }; +}; +</pre> +</td> +</tr> +</table> + +<h2 id=append>Appending</h2> +<p>If the reference target property does not exist in the main DT, it is +appended after DTO. For example:</p> + +<table> +<tr> +<th width="33%">main.dts</th> +<th width="33%">overlay.dts</th> +<th>Merged Result</th> +</tr> + +<tr> +<td> +<pre class="prettyprint"> +[my_main_dt.dts] + +/dts-v1/; + +/ { + compatible = "corp,foo"; + + my_node: node@0 { + status = "okay"; + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +[my_overlay_dt.dts] + +/dts-v1/; +/plugin/; + +&my_node { + new_prop = "bar"; +}; +</pre> +</td> + +<td> +<pre class="prettyprint"> +/dts-v1/; + +/ { + compatible = "corp,foo"; + + ... + + node@0 { + linux,phandle = <0x1>; + phandle = <0x1>; + status = "okay"; + new_prop = "bar"; + }; +}; +</pre> +</td> +</tr> +</table> + +<h2 id="child">Child nodes</h2> +<p>Examples of child node syntax:</p> + +<table> +<tr> +<th width="33%">main.dts</th> +<th width="33%">overlay.dts</th> +<th>Merged Result</th> +</tr> + +<tr> +<td> +<pre class="prettyprint"> +[my_main_dt.dts] + +/dts-v1/; + +/ { + compatible = "corp,foo"; + + my_nodes: nodes { + compatible = "corp,bar"; + + node@0 { + status = "disabled"; + }; + }; +}; +</pre> +</td> + +<td class="alt"> +<pre class="prettyprint"> +[my_overlay_dt.dts] + +/dts-v1/; +/plugin/; + +&my_nodes { + new_prop1 = "abc"; + + node@0 { + status = "okay"; + new_prop2 = "xyz"; + }; +}; +</pre> +</td> + +<td> +<pre class="prettyprint"> +/dts-v1/; + +/ { + compatible = "corp,foo"; + + ... + + nodes { + linux,phandle = <0x1>; + phandle = <0x1>; + compatible = "corp,bar"; + new_prop1 = "abc"; + + node@0 { + linux,phandle = <0x2>; + phandle = <0x2>; + status = "okay"; + new_prop2 = "xyz"; + }; + }; +}; +</pre> +</td> +</tr> +</table> + + </body> +</html> diff --git a/en/devices/architecture/hal-types.html b/en/devices/architecture/hal-types.html new file mode 100644 index 00000000..52cb869f --- /dev/null +++ b/en/devices/architecture/hal-types.html @@ -0,0 +1,154 @@ +<html devsite> + <head> + <title>HAL Types</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>As part of the Android O rearchitecture of the lower layers of the Android OS +to support better modularity, devices running Android O must support binderized +or passthrough HALs:</p> + +<ul> +<li><strong>Binderized HALs</strong>. HALs expressed in HAL interface definition +language (HIDL). These HALs replace both conventional and legacy HALs used in +earlier versions of Android and can serve the HAL in binderized mode. All +devices launching with Android O or later must support binderized HALs only.</li> + +<li><strong>Passthrough HALs</strong>. A HIDL-wrapped conventional or legacy +HAL. These HALs wrap existing HALs and can serve the HAL in binderized and +same-process (passthrough) modes. Devices upgrading to Android O can use +passthrough HALs.</li> +</ul> + +<h2 id=requirements>HAL mode requirements</h2> + +<table> +<tbody> + +<tr> +<th>Device</th> +<th>Passthrough</th> +<th>Binderized</th> +</tr> + +<tr> +<td><em>Launch with Android O</em></td> +<td>HALs listed in <a href="#passthrough">Passthrough HALs</a> must be +passthrough.</td> +<td>All other HALs are binderized (including HALs that are vendor +extensions).</td> +</tr> + +<tr> +<td rowspan=2><em>Upgrade to Android O</em></td> +<td>HALs listed in <a href="#passthrough">Passthrough HALs</a> must be +passthrough.</td> +<td>HALs listed in <a href="#binderized">Binderized HALs</a> must be +binderized.</td> +</tr> + +<tr> +<td colspan=2>All other HALs provided by the vendor image can be in passthrough +OR binderized mode.</td> +</tr> + +</tbody> +</table> + +<h2 id=binderized>Binderized HALs</h2> +<p>Android requires the following HALS to to be binderized on all Android +devices regardless of whether they are launch devices or upgrade devices:</p> + +<ul> +<li><code>android.hardware.biometrics.fingerprint@2.1</code>. Replaces +<code>fingerprintd</code> which is no longer in Android O.</li> +<li><code>android.hardware.configstore@1.0</code>. New in Android O.</li> +<li><code>android.hardware.dumpstate@1.0</code>. The original interface provided +by this HAL could not be shimmed and was changed. Because of this, +<code>dumpstate_board</code> must be re-implemented on a given device (this is +an optional HAL).</li> +<li><code>android.hardware.graphics.allocator@2.0</code>. Required to be +binderized in Android O so file descriptors don't have to be shared between +trusted and untrusted processes.</li> +<li><code>android.hardware.radio@1.0</code>. Replaces the interface provided by +<code>rild</code> which lives in its own process.</li> +<li><code>android.hardware.usb@1.0</code>. New in Android O.</li> +<li><code>android.hardware.wifi@1.0</code>. New in Android O, replaces the +legacy Wi-Fi HAL library that was loaded into <code>system_server</code>.</li> +<li><code>android.hardware.wifi.supplicant@1.0</code>. A HIDL interface over the +existing <code>wpa_supplicant</code> process.</li> +</ul> + +<p class=note><strong>NOTE</strong>: Android provides the following HIDL +interfaces which will always be in binderized mode: +<code>android.frameworks.*</code>, <code>android.system.*</code> , and +<code>android.hidl.*</code> (except for <code>android.hidl.memory@1.0</code> +as described below).</p> + +<h2 id=passthrough>Passthrough HALs</h2> +<p>Android requires the following HALs to be in passthrough mode on all Android +devices regardless of whether they are launch devices or upgrade devices:</p> + +<ul> +<li><code>android.hardware.graphics.mapper@1.0</code>. Maps memory into the +process it lives in.</li> +<li><code>android.hardware.renderscript@1.0</code>. Passes items in the same +process (equivalent to <code>openGL</code>).</li> +</ul> +<p>All HALs not listed above must be binderized for launch devices.</p> + +<h2 id=same-process>Same-Process HALs</h2> +<p>Same-Process HALs (SP-HALs) always open in the same process in which they are +used. They include all HALs not expressed in HIDL as well as some that are +<strong>not</strong> binderized. Membership in the SP-HAL set is controlled only +by Google, with no exceptions.</p> + +<p>SP-HALs include the following:</p> + +<ul> +<li><code>openGL</code></li> +<li><code>Vulkan</code></li> +<li><code>android.hidl.memory@1.0</code> (provided by the Android system, always +passthrough)</li> +<li><code>android.hardware.graphics.mapper@1.0</code>.</li> +<li><code>android.hardware.renderscript@1.0</code></li> +</ul> + +<h2 id=legacy>Conventional & legacy HALs</h2> + +<p>Conventional HALs (deprecated in Android O) are interfaces that conform to a +specific named and versioned application binary interface (ABI). The bulk of +Android system interfaces +(<a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h">camera</a>, +<a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/audio.h">audio</a>, +<a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/sensors.h">sensors</a>, +etc.) are in the form of conventional HALs, which are defined under +<a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware">hardware/libhardware/include/hardware</a>.</p> + +<p>Legacy HALs (also deprecated in Android O) are interfaces that predate +conventional HALs. A few important subsystems (Wi-Fi, Radio Interface Layer, and +Bluetooth) are legacy HALs. While there's no uniform or standardized way to +describe a legacy HAL, anything predating Android O that is not a conventional +HAL is a legacy HAL. Parts of some legacy HALs are contained in +<a href="https://android.googlesource.com/platform/hardware/libhardware_legacy/+/master">libhardware_legacy</a>, +while other parts are interspersed throughout the codebase.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl-cpp/functions.html b/en/devices/architecture/hidl-cpp/functions.html new file mode 100644 index 00000000..5d52b31d --- /dev/null +++ b/en/devices/architecture/hidl-cpp/functions.html @@ -0,0 +1,153 @@ +<html devsite> + <head> + <title>Functions</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Functions in a HIDL interface are mapped to methods in the autogenerated +<code>IFoo</code> C++ class declaration. The name of each function remains the +same in C++; the following sections describe how HIDL arguments and return +values are translated to C++.</p> + +<h2 id=parameters>Function parameters</h2> +<p>The arguments listed in the <code>.hal</code> file map to C++ data types. +Arguments that do not map to a primitive C++ type are passed by const +reference.</p> + +<p>For every HIDL function that has a return value (has a <code>generates</code> +statement), the C++ parameter list for that function has an additional argument: +a callback function that is called with the return values of the HIDL function. +There is <strong>one exception</strong>: If the <code>generates</code> clause +contains a single parameter that directly maps to a C++ primitive, callback +<em>elision</em> is used (the callback is removed and the return value is +returned from the function through a normal <code>return</code> statement).</p> + +<h2 id=return-values>Function return values</h2> +<p>The following functions have return values.</p> + +<h3 id=transport>Transport errors and return type</h3> +<p>The <code>generates</code> statement can result in three types of function +signatures:</p> + +<ul> +<li>For only one return value that is a C++ primitive, the +<code>generates</code> return value is returned by value from the function in a +<code>Return<T></code> object.</li> +<li>For more complicated cases, the <code>generates</code> return value(s) are +returned through the callback parameter provided with the function call itself, +and the function returns <code>Return<void></code>.</li> +<li>For when no <code>generates</code> statement exists, the function returns +<code>Return<void></code>.</li> +</ul> + +<p>RPC calls can occasionally encounter transport errors, e.g. when the server +dies, when transport resources are insufficient to complete the call, or when +the parameters passed do not permit completing the call (such as missing a +required callback function). <code>Return</code> objects store transport error +indications as well as a <code>T</code> value (except +<code>Return<void></code>).</p> + +<p>As the client-side and server-side functions have the same signature, the +server-side function must return a <code>Return</code> type even though its +implementation does not signal transport errors. <code>Return<T></code> +objects are constructed with <code>Return(myTValue)</code> (or can be implicitly +constructed from <code>mTValue</code>, such as in <code>return</code> +statements) and <code>Return<void></code> objects are constructed with +<code>Void()</code>.</p> + +<p><code>Return<T></code> objects have implicit conversion to and from +their <code>T</code> value. The <code>Return</code> object can be checked for +transport errors by calling its <code>isOk()</code> method. This check is not +required; however, if an error occurs and is not checked by the time the +<code>Return</code> object is destroyed, or a <code>T</code> value conversion is +attempted, the client process will be killed and an error logged. If +<code>isOk()</code> indicates a transport error or a call failure due to a logic +error in developer code (such as passing <code>nullptr</code> as a synchronous +callback), then <code>description()</code> can be called on the Return object to +return a string suitable for logging. In such cases, there is no way to +determine how much code may have executed on the server as a result of the +failed call. The method <code>isDeadObject()</code> is also provided. This +method indicates that <code>!isOk()</code> is because the remote object has +crashed or no longer exists. <code>isDeadObject()</code> always implies +<code>!isOk()</code>.</p> + +<h3 id=return-by>Return by value</h3> +<p>If the <code>generates</code> statement maps to a single C++ primitive, no +callback parameter is in the parameter list. Instead, an implementation provides +the return value <code>T</code> in a <code>Return<T></code> object, which +can be implicitly generated from the primitive type <code>T</code>. For +example:</p> + +<pre class="prettyprint"> +Return<uint32_t> someMethod() { + uint32_t return_data = ...; // Compute return_data + return return_data; +}; +</pre> + +<p>The method <code>Return<*>::withDefault</code> is also provided. This +method provides a value in cases where the return value is <code>!isOk()</code>. +This method also automatically marks the return object as okay so the client +process will not be killed.</p> + +<h3 id=return-callback>Return using callback parameter</h3> +<p>A callback can pass the return value of the HIDL function back to the caller. +The prototype of the callback is a <code>std::function</code> object with +parameters (taken from the <code>generates</code> statement) mapped to C++ +types. Its return value is void—the callback itself doesn't return a value.</p> + +<p>The return value of a C++ function with a callback parameter has type +<code>Return<void></code>. The server implementation is responsible only +for providing the return value. As the return values are already transferred +using the callback, the <code>T</code> template parameter is <code>void</code>: +</p> + +<pre class="prettyprint"> +Return<void> someMethod(someMethod_cb _cb); +</pre> + +<p>From their C++ implementation, server implementations should return +<code>Void()</code>, which is a static inlined function returning a +<code>Return<void></code> object. Example of a typical server method +implementation with a callback parameter:</p> + +<pre class="prettyprint"> +Return<void> someMethod(someMethod_cb _cb) { + // Do some processing, then call callback with return data + hidl_vec<uint32_t> vec = ... + _cb(vec); + return Void(); +}; +</pre> + +<h2 id=no-return>Functions without return values</h2> +<p>The C++ signature of a function without a <code>generates</code> statement +will not have a callback parameter in the parameter list. Its return type will +be <code>Return<void>.</code></p> + +<h2 id=oneway>Oneway functions</h2> +<p>Functions marked with the <code>oneway</code> keyword are asynchronous +functions (clients won't block on their execution) and do not have return +values. The C++ signature of a <code>oneway</code> function will not have a +callback parameter in the parameter list, and its C++ return value will be +<code>Return<void></code>.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl-cpp/index.html b/en/devices/architecture/hidl-cpp/index.html new file mode 100644 index 00000000..af685400 --- /dev/null +++ b/en/devices/architecture/hidl-cpp/index.html @@ -0,0 +1,132 @@ +<html devsite> + <head> + <title>HIDL C++</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Android O re-architects the Android OS to define clear interfaces between the +device-independent Android platform and device- and vendor-specific code. +Android already defines many such interfaces in the form of HAL interfaces, +defined as C headers in <code>hardware/libhardware</code>. HIDL replaces these +HAL interfaces with stable, versioned interfaces, which can be client- and +server-side HIDL interfaces in C++ (described below) or +<a href="/devices/architecture/hidl-java/index.html">Java</a>.</p> + +<p>The pages in this section describe C++ implementations of HIDL interfaces, +including details about the files auto-generated from the HIDL <code>.hal</code> +files by the <code>hidl-gen</code> compiler, how these files are packaged, and +how to integrate these files with the C++ code that uses them.</p> + +<h2 id=client-server>Client & server implementations</h2> +<p>HIDL interfaces have client and server implementations:</p> + +<ul> +<li>A <strong>client</strong> of a HIDL interface is the code that uses the +interface by calling methods on it.</li> +<li>A <strong>server</strong> is an implementation of a HIDL interface that +receives calls from clients and returns results (if necessary).</li> +</ul> + +<p>In transitioning from <code>libhardware</code> HALs to HIDL HALs, the HAL +implementation becomes the server and the process calling into the HAL becomes +the client. Default implementations can serve both passthrough and binderized +HALs, and can change over time:</p> + +<p><img src="../images/treble_cpp_legacy_hal_progression.png"></p> +<p><strong>Figure 1.</strong> Development progression for legacy HALs.</p> + +<h2>Creating the HAL client</h2> +<p>Start by including the HAL libraries in the makefile:</p> + +<ul> +<li>Make: <code>LOCAL_SHARED_LIBRARIES += android.hardware.nfc@1.0</code></li> +<li>Soong: <code>shared_libs: [ …, android.hardware.nfc@1.0 ]</code></li> +</ul> + +<p>Next, include the HAL header files:</p> + +<pre class="prettyprint"> +#include <android/hardware/nfc/1.0/IFoo.h> +… +// in code: +sp<IFoo> client = IFoo::getService(); +client->doThing(); +</pre> + +<h2>Creating the HAL server</h2> +<p>To create the HAL implementation, you must have the <code>.hal</code> files +that represent your HAL and have already generated makefiles for your HAL using +<code>-Lmakefile</code> or <code>-Landroidbp</code> on <code>hidl-gen</code> +(<code>./hardware/interfaces/update-makefiles.sh</code> does this for internal +HAL files and is a good reference). When transferring over HALs from +<code>libhardware</code>, you can do a lot of this work easily using c2hal.</p> + +<p>To create the necessary files to implement your HAL:</p> + +<pre class="prettyprint"> +PACKAGE=android.hardware.nfc@1.0 +LOC=hardware/interfaces/nfc/1.0/default/ +make hidl-gen -j64 +hidl-gen -o $LOC -Lc++-impl -randroid.hardware:hardware/interfaces \ + -randroid.hidl:system/libhidl/transport $PACKAGE +hidl-gen -o $LOC -Landroidbp-impl -randroid.hardware:hardware/interfaces \ + -randroid.hidl:system/libhidl/transport $PACKAGE +</pre> + +<p>For the HAL to work in passthrough mode (for legacy devices), you must have +the function <em>HIDL_FETCH_IModuleName</em> residing in +<code>/system/lib(64)?/hw/android.hardware.package@3.0-impl(-$OPTIONAL_IDENTIFIER).so</code> +where <code>$OPTIONAL_IDENTIFIER</code> is a string identifying the passthrough +implementation. The passthrough mode requirements are met automatically by the +above commands, which also create the <code>android.hardware.nfc@1.0-impl</code> +target.</p> + +<p>Next, fill out the stubs with functionality and setup a daemon. Example +daemon code (supporting passthrough):</p> + +<pre class="prettyprint"> +#include <hidl/LegacySupport.h> + +int main(int /* argc */, char* /* argv */ []) { + return defaultPassthroughServiceImplementation<INfc>("nfc"); +} +</pre> + +<p><code>defaultPassthroughServiceImplementation</code> will +<code>dlopen()</code> the provided <code>-impl</code> library and provide it as +a binderized service. Example daemon code (for pure binderized service):</p> + +<pre class="prettyprint"> +int main(int /* argc */, char* /* argv */ []) { + Nfc nfc = new Nfc(); + nfc->registerAsService(); +} +</pre> + +<p>This daemon should live in <code>$PACKAGE + "-service"</code> (for example, +<code>android.hardware.nfc@1.0-service</code>). The +<a href="/security/selinux/device-policy.html">sepolicy</a> for a specific class +of HALs is the attribute <code>hal_<module></code> (for instance, +<code>hal_nfc)</code>. This attribute must be applied to the daemon that runs a +particular HAL (if the same process serves multiple HALs, multiple attributes +can be applied to it).</p> + +</body> +</html> diff --git a/en/devices/architecture/hidl-cpp/interfaces.html b/en/devices/architecture/hidl-cpp/interfaces.html new file mode 100644 index 00000000..5047e8d0 --- /dev/null +++ b/en/devices/architecture/hidl-cpp/interfaces.html @@ -0,0 +1,271 @@ +<html devsite> + <head> + <title>Interfaces</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Every interface defined in a HIDL package has its own autogenerated C++ class +inside its package's namespace. Clients and servers deal with interfaces in +different ways:</p> + +<ul> +<li><strong>Servers</strong> implement interfaces.</li> +<li><strong>Clients</strong> call methods on interfaces.</li> +</ul> + +<p>Interfaces can either be registered by name by the server or passed as +parameters to HIDL-defined methods. For example, framework code may serve an +interface to receive asynchronous messages from the HAL and pass that interface +directly to the HAL without registering it.</p> + +<h2 id=server>Server implementation</h2> +<p>A server implementing the <code>IFoo</code> interface must include the +<code>IFoo</code> header file that was autogenerated:</p> + +<pre class="prettyprint"> +#include <android/hardware/samples/1.0/IFoo.h> +</pre> + +<p>The header is automatically exported by the shared library of the +<code>IFoo</code> interface to link against. Example <code>IFoo.hal</code>:</p> + +<pre class="prettyprint"> +// IFoo.hal +interface IFoo { + someMethod() generates (vec<uint32_t>); + ... +} +</pre> + +<p>Example skeleton for a server implementation of the IFoo interface:</p> + +<pre class="prettyprint"> +// From the IFoo.h header +using android::hardware::samples::V1_0::IFoo; + +class FooImpl : public IFoo { + Return<void> someMethod(foo my_foo, someMethod_cb _cb) { + vec<uint32_t> return_data; + // Compute return_data + _cb(return_data); + return Void(); + } + ... +}; +</pre> + +<p>To make the implementation of a server interface available to a client, you +can:</p> + +<ol> +<li><strong>Register</strong> the interface implementation with the +<code>hwservicemanager</code> (see details below), +<br><br> +OR<br><br> +</li> +<li><strong>Pass</strong> the interface implementation as an argument of an +interface method (for detals, see <a href="#asynchronous">Asynchronous +callbacks</a>).</li> +</ol> + +<p>When registering the interface implementation, the +<code>hwservicemanager</code> process keeps track of registered HIDL interfaces +running on the device by name and version. Servers can register a HIDL interface +implementation by name and clients can request service implementations by name +and version. This process serves the HIDL interface +<code>android.hidl.manager@1.0::IServiceManager</code>.</p> + +<p>Each auto-generated HIDL interface header file (such as <code>IFoo.h</code>) +has a <code>registerAsService()</code> method that can be used to register the +interface implementation with the <code>hwservicemanager</code>. The only +required argument is the name of the interface implementations as clients will +use this name to retrieve the interface from the <code>hwservicemanager</code> +later:</p> + +<pre class="prettyprint"> +::android::sp<IFoo> myFoo = new FooImpl(); +::android::sp<IFoo> myFoo = new FooAnotherImpl(); +myFoo->registerAsService(); +mySecondFoo->registerAsService("another_foo"); +</pre> + +<p>The <code>hwservicemanager</code> treats the combination of +<code>[package@version::interface, instance_name]</code> as unique to enable +different interfaces (or different versions of the same interface) to register +with identical instance names without conflicts. If you call +<code>registerAsService()</code> with the exact same package version, interface, +and instance name, the <code>hwservicemanager</code> drops its reference to the +previously registered service and uses the new one.</p> + +<h2 id=client>Client implementation</h2> +<p>Just as the server does, a client must <code>#include</code> every interface +it refers to:</p> + +<pre class="prettyprint"> +#include <android/hardware/samples/1.0/IFoo.h> +</pre> + +<p>A client can obtain an interface in two ways:</p> + +<ul> +<li>Through <code>I<InterfaceName>::getService</code> (via the +<code>hwservicemanager</code>)</li> +<li>Through an interface method</li> +</ul> + +<p>Each autogenerated interface header file has a static <code>getService</code> +method that can be used to retrieve a service instance from the +<code>hwservicemanager</code>:</p> + +<pre class="prettyprint"> +// getService will return nullptr if the service can't be found +sp<IFoo> myFoo = IFoo::getService(); +sp<IFoo> myAlternateFoo = IFoo::getService("another_foo"); +</pre> + +<p>Now the client has an an <code>IFoo</code> interface, and can call methods to +it as if it were a local class implementation. In reality, the implementation +may run in the same process, a different process, or even on another device +(with HAL remoting). Because the client called <code>getService</code> on an +<code>IFoo</code> object included from version <code>1.0 </code>of the package, +the <code>hwservicemanager</code> returns a server implementation only if that +implementation is compatible with <code>1.0</code> clients. In practice, this +means only server implementations with version <code>1.n</code> (version +<code>x.(y+1)</code> of an interface must extend (inherit from) +<code>x.y</code>).</p> + +<p>Additionally the method <code>castFrom</code> is provided to cast between +different interfaces. This method works by making an IPC call to the remote +interface to make sure the underlying type is the same as the type that is being +requested. If the requested type is unavailable, then <code>nullptr</code> is +returned.</p> + +<pre class="prettyprint"> +sp<V1_0::IFoo> foo1_0 = V1_0::IFoo::getService(); +sp<V1_1::IFoo> foo1_1 = V1_1::IFoo::castFrom(foo1_0); +</pre> + +<h2 id=asynchronous>Asynchronous callbacks</h2> +<p>Many existing HAL implementations talk to asynchronous hardware, which means +they need an asynchronous way to notify clients of new events that have +occurred. A HIDL interface can be used as an asynchronous callback because HIDL +interface functions can take HIDL interface objects as parameters.</p> + +<p>Example interface file <code>IFooCallback.hal</code>:</p> + +<pre class="prettyprint"> +package android.hardware.samples@1.0; +interface IFooCallback { + sendEvent(uint32_t event_id); + sendData(hidl_vec<uint8_t> data); +} +</pre> + +<p>Example new method in <code>IFoo</code> that takes an +<code>IFooCallback</code> parameter:</p> + +<pre class="prettyprint"> +package android.hardware.samples@1.0; +interface IFoo { + struct Foo { + int64_t some_value; + Handle my_handle; + }; + + someMethod(Foo foo) generates (int32_t ret); + another_method() generates (hidl_vec<uint32_t>); + register_callback(IFooCallback callback); +}; +</pre> + +<p>The <em>client</em> using the <code>IFoo</code> interface is the +<em>server</em> of the <code>IFooCallback</code> interface; it provides an +implementation of <code>IFooCallback</code>:</p> + +<pre class="prettyprint"> +class FooCallback : public IFooCallback { + Return<void> sendEvent(uint32_t event_id) { + // process the event from the HAL + } + Return<void> sendData(hidl_vec<uint8_t> data) { + // process data from the HAL + } +}; +</pre> + +<p>It can also simply pass that over an existing instance of the +<code>IFoo</code> interface:</p> +<pre class="prettyprint"> +sp<IFooCallback> myFooCallback = new FooCallback(); +myFoo.registerCallback(myFooCallback); +</pre> + +<p>The server implementing <code>IFoo</code> receives this as an +<code>sp<IFooCallback></code> object. It can store the callback, and call +back into the client whenever it wants to use this interface.</p> + +<h2 id=death>Death recipients</h2> +<p>As service implementations can run in a different process, it can happen +that the process implementing an interface dies while the client stays alive. +Any calls on an interface object hosted in a process that has died will fail +with a transport error (<code>isOK()</code> will return false). The only way to +recover from such a failure is to request a new instance of the service by +calling <code>I<InterfaceName>::getService()</code>. This works only if +the process that crashed has restarted and re-registered its services with the +<code>servicemanager</code> (which is generally true for HAL implementations). +</p> + +<p>Instead of dealing with this reactively, clients of an interface can also +register a <em>death recipient</em> to get a notification when a service dies. +To register for such notifications on a retrieved <code>IFoo</code> interface, a +client can do the following:</p> + +<pre class="prettyprint"> +foo->linkToDeath(recipient, 1481 /* cookie */); +</pre> + +<p>The <code>recipient</code> parameter must be an implementation of the +<code>android::hardware::hidl_death_recipient</code> interface provided by HIDL, +which contains a single method <code>serviceDied()</code> that will be called +from a thread in the RPC threadpool when the process hosting the interface dies: +</p> + +<pre class="prettyprint"> +class MyDeathRecipient : android::hardware::hidl_death_recipient { + virtual void serviceDied(uint64_t cookie, const android::wp<::android::hidl::base::V1_0::IBase>& who) { + // Deal with the fact that the service died + } +} +</pre> + +<p>The <code>cookie</code> parameter contains the cookie that was passed in with +<code>linkToDeath()</code>, whereas the <code>who</code> parameter contains a +weak pointer to the object representing the service in the client. With the +sample call given above, <code>cookie</code> equals 1481, and <code>who</code> +equals <code>foo</code>.</p> + +<p>It's also possible to unregister a death recipient after registering it:</p> + +<pre class="prettyprint"> +foo->unlinkToDeath(recipient); +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl-cpp/packages.html b/en/devices/architecture/hidl-cpp/packages.html new file mode 100644 index 00000000..2c8e096d --- /dev/null +++ b/en/devices/architecture/hidl-cpp/packages.html @@ -0,0 +1,202 @@ +<html devsite> + <head> + <title>Packages</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p class=note><strong>Note:</strong> This section uses sample <code>.hal</code> +files to illustrate how HIDL language constructs map to C++.</p> + +<p>With few exceptions, HIDL interface packages are located in +<code>hardware/interfaces</code> or the <code>vendor/</code> directory. The +<code>hardware/interfaces</code> top-level maps directly to the +<code>android.hardware</code> package namespace; the version is a subdirectory +under the package (not interface) namespace.</p> + +<p>The <code>hidl-gen</code> compiler compiles the <code>.hal</code> files into +a set of a <code>.h</code> and <code>.cpp</code> files. From these autogenerated +files a shared library that client/server implementations link against is built. +The <code>Android.bp</code> file that builds this shared library is +autogenerated by the <code>hardware/interfaces/update-makefiles.sh</code> +script. Every time you add a new package to <code>hardware/interfaces</code>, or +add/remove <code>.hal</code> files to/from an existing package, you must rerun +the script to ensure the generated shared library is up-to-date.</p> + +<p>For example, the <code>IFoo.hal</code> sample file should be located in +<code>hardware/interfaces/samples/1.0</code>. The sample +<code><strong>IFoo.hal</code></strong> file creates an IFoo interface in the +<strong>samples</strong> package:</p> + +<pre class="prettyprint"> +package android.hardware.samples@1.0; +interface IFoo { + struct Foo { + int64_t someValue; + handle myHandle; + }; + + someMethod() generates (vec<uint32_t>); + anotherMethod(Foo foo) generates (int32_t ret); +}; +</pre> + +<h2 id=generated-files>Generated files</h2> + +<p>Autogenerated files in a HIDL package are linked into a single shared +library with the same name as the package (for example, +<code>android.hardware.samples@1.0</code>). The shared library also exports a +single header, <code>IFoo.h</code>, which can be included by clients and +servers. Using the <code>hidl-gen</code> compiler with the <code>IFoo.hal</code> +interface file as an input, binderized mode has the following autogenerated +files:</p> + +<p><img src="../images/treble_cpp_compiler_generated_files.png"></p> +<p><strong>Figure 1.</strong> Files generated by compiler.</p> + +<ul> +<li><code><strong>IFoo.h</code></strong>. Describes the pure <code>IFoo</code> +interface in a C++ class; it contains the methods and types defined in the +<code>IFoo</code> interface in the <code>IFoo.hal</code> file, translated to C++ +types where necessary. <strong>Does not contain</strong> details related to the +RPC mechanism (e.g., <code>HwBinder</code>) used to implement this interface. +The class is namespaced with the package and version, e.g. +<code>::android::hardware::samples::IFoo::V1_0</code>. Both clients and servers +include this header: Clients for calling methods on it and servers for +implementing those methods.</li> +<li><code><strong>IHwFoo.h</code></strong>. Header file that contains +declarations for functions that serialize data types used in the interface. +Developers should never include his header directly (it does not contain any +classes).</li> +<li><code><strong>BpFoo.h</code></strong>. A class that inherits from +<code>IFoo</code> and describes the <code>HwBinder</code> proxy (client-side) +implementation of the interface. Developers should never refer to this class +directly.</li> +<li><code><strong>BnFoo.h</code></strong>.<strong> </strong>A class that holds a +reference to an <code>IFoo</code> implementation and describes the +<code>HwBinder</code> stub (server-side) implementation of the interface. +Developers should never refer to this class directly.</li> +<li><code><strong>FooAll.cpp</code></strong>. A class that contains the +implementations for both the <code>HwBinder</code> proxy and the +<code>HwBinder</code> stub. When a client calls an interface method, the proxy +automatically marshals the arguments from the client and sends the transaction +to the binder kernel driver, which delivers the transaction to the stub on the +other side (which then calls the actual server implementation).</li> +</ul> + +<p>The files are structured similarly to the files generated by +<code>aidl-cpp</code> (for details, see "Passthrough mode" in the +<a href="/devices/architecture/hidl/index.html">HIDL Overview</a>). The only +autogenerated file that is independent of the RPC mechanism used by HIDL is +<code>IFoo.h</code>; all other files are tied to the HwBinder RPC mechanism used +by HIDL. Therefore, client and server implementations <strong>should never +directly refer to anything other than <code>IFoo</code></strong>. To achieve +this, include only <code>IFoo.h</code> and link against the generated shared +library.</p> + +<p class=note><strong>Note</strong>: HwBinder is only one possible transport; +new transports may be added in the future.</p> + +<h2 id=link-libraries>Linking to shared libraries</h2> +<p>A client or server that uses any interface in a package must include the +shared library of that package in <strong>one (1)</strong> of the following +locations:</p> + +<ul> +<li>In <strong>Android.mk</strong>: +<pre class="prettyprint"> +LOCAL_SHARED_LIBRARIES += android.hardware.samples@1.0 +</pre> +</li> + +<li>In <strong>Android.bp</strong>: +<pre class="prettyprint"> +shared_libs: [ + /* ... */ + "android.hardware.samples@1.0", +], +</pre> +</li> +</ul> + +<p>For specific libraries:</p> + +<table> + +<tr> +<th><code>libhidlbase</code></th> +<td>Includes standard HIDL data-types. Unless your interface consists only of +primitives that map directly to C++ primitives, you must also link this library: +<pre class="prettyprint"> +LOCAL_SHARED_LIBRARIES += libhidlbase +</pre> +</td> +</tr> + +<tr> +<th><code>libhidltransport</code></th> +<td>Handles the transport of HIDL calls over different RPC/IPC mechanisms. You +must always link this library: +<pre class="prettyprint"> +LOCAL_SHARED_LIBRARIES += libhidltransport +</pre> +</td> +</tr> + +<tr> +<th><code>libhwbinder</code></th> +<td>You must also link to this library: +<pre class="prettyprint"> +LOCAL_SHARED_LIBRARIES += libhwbinder +</pre> +</td> +</tr> + +<tr> +<th><code>libfmq</code></th> +<td>To use Fast Message Queue IPC, you must also link to this library. +<pre class="prettyprint"> +LOCAL_SHARED_LIBRARIES += libfmq +</pre> +</td> +</tr> + +</tbody> +</table> + +<h2 id=namespaces>Namespaces</h2> +<p>HIDL functions and types such as <code>Return<T></code> and +<code>Void()</code> are declared in namespace <code>::android::hardware</code>. +The C++ namespace of a package is determined by the package name and version. +For example, a package <strong>mypackage</strong> with version 1.2 under +<code>hardware/interfaces</code> has the following qualities:</p> + +<ul> +<li><strong>C++ namespace</strong> is +<code>::android::hardware::mypackage::V1_2</code></li> +<li><strong>Fully qualified name </strong>of <code>IMyInterface</code> in that +package is: <code>::android::hardware::mypackage::V1_2::IMyInterface</code>. +(<code>IMyInterface</code> is an identifier, not part of the namespace).</li> +<li><strong>Types</strong> defined in the package's <code>types.hal</code> file +are identified as: +<code>::android::hardware::mypackage::V1_2::MyPackageType</code></li> +</ul> + + </body> +</html> diff --git a/en/devices/architecture/hidl-cpp/types.html b/en/devices/architecture/hidl-cpp/types.html new file mode 100644 index 00000000..1d93df74 --- /dev/null +++ b/en/devices/architecture/hidl-cpp/types.html @@ -0,0 +1,323 @@ +<html devsite> + <head> + <title>Data Types</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>HIDL data declarations generate C++ standard-layout data structures. These +structures can be placed anywhere that feels natural (on the stack, at file or +global scope, or on the heap) and can be composed in the same fashion. Client +code calls HIDL proxy code passing in const references and primitive types, +while the stub and proxy code hides the details of serialization.</p> + +<p class=note><strong>Note:</strong> At no point is developer-written code +required to explicitly serialize or deserialize data structures.</p> + +<p>The table below maps HIDL primitives to C++ data types:</p> + +<table> +<tbody> +<tr> +<th><strong>HIDL Type</strong></th> +<th><strong>C++ Type</strong></th> +<th><strong>Header/Library</strong></th> +</tr> + +<tr> +<td><pre>enum</pre></td> +<td><pre>enum class</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>uint8_t..uint64_t</pre></td> +<td><pre>uint8_t..uint64_t</pre></td> +<td><pre><stdint.h></pre></td> +</tr> + +<tr> +<td><pre>int8_t..int64_t</pre></td> +<td><pre>int8_t..int64_t</pre></td> +<td><pre><stdint.h></pre></td> +</tr> + +<tr> +<td><pre>float</pre></td> +<td><pre>float</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>double</pre></td> +<td><pre>double</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>vec<T></pre></td> +<td><pre>hidl_vec<T></pre></td> +<td><pre>libhidlbase</pre></td> +</tr> + +<tr> +<td><pre>T[S1][S2]...[SN]</pre></td> +<td><pre>T[S1][S2]...[SN]</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>string</pre></td> +<td><pre>hidl_string</pre></td> +<td><pre>libhidlbase</pre></td> +</tr> + +<tr> +<td><pre>handle</pre></td> +<td><pre>hidl_handle</pre></td> +<td><pre>libhidlbase</pre></td> +</tr> + +<tr> +<td><pre>opaque</pre></td> +<td><pre>uint64_t</pre></td> +<td><pre><stdint.h></pre> +</td> +</tr> + +<tr> +<td><pre>struct</pre></td> +<td><pre>struct</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>union</pre></td> +<td><pre>union</pre></td> +<td></td> +</tr> + +<tr> +<td><pre>fmq_sync</pre></td> +<td><pre>MQDescriptorSync</pre></td> +<td><pre>libhidlbase</pre></td> +</tr> + +<tr> +<td><pre>fmq_unsync</pre></td> +<td><pre>MQDescriptorUnsync</pre></td> +<td><pre>libhidlbase</pre></td> +</tr> + +</tbody> +</table> + +<p>The sections below describe data types in more detail.</p> + +<h2 id=enum>enum</h2> +<p>An enum in HIDL becomes an enum in C++. For example:</p> +<pre class="prettyprint"> +enum Mode : uint8_t { WRITE = 1 << 0, READ = 1 << 1 }; +</pre> + +<p>… becomes:</p> +<pre class="prettyprint"> +enum class Mode : uint8_t { WRITE = 1, READ = 2 }; +</pre> + +<h2 id=bitfield>bitfield<T></h2> +<p><code>bitfield<T></code> (where <code>T</code> is a user-defined enum) +becomes the underlying type of that enum in C++. In the above example, +<code>bitfield<Mode></code> becomes <code>uint8_t</code>.</p> + +<h2 id=vect>vec<T></h2> +<p>The <code>hidl_vec<T></code> class template is part of +<code>libhidlbase</code> and can be used to pass a vector of any HIDL type with +an arbitrary size. The comparable fixed size container is +<code>hidl_array</code>. A <code>hidl_vec<T></code> can also be +initialized to point to an external data buffer of type <code>T</code>, using +the <code>hidl_vec::setToExternal()</code> function.</p> + +<p>In addition to emitting/inserting the struct appropriately in the generated +C++ header, the use of <code>vec<T></code> generates some convenience +functions to translate to/from <code>std::vector</code> and bare <code>T</code> +pointers. If the <code>vec<T></code> is used as a parameter, the function +using it will be overloaded (two prototypes will be generated) to accept and +pass both the HIDL struct and a <code>std::vector<T></code> type for that +parameter.</p> + +<h2 id=arrays>array</h2> +<p>Constant arrays in hidl are represented by the <code>hidl_array</code> class +in <code>libhidlbase</code>. A <code>hidl_array<T, S1, S2, …, +SN></code> represents an N dimensional fixed size array +<code>T[S1][S2]…[SN]</code>.</p> + +<h2 id=string>string</h2> +<p>The <code>hidl_string</code> class (part of <code>libhidlbase</code>) can be +used to pass strings over HIDL interfaces and is defined in +<code>/system/libhidl/base/include/hidl/HidlSupport.h</code>. The first storage +location in the class is a pointer to its character buffer.</p> + +<p><code>hidl_string</code> knows how to convert to and from +<code>std::string and char*</code> (C-style string) using +<code>operator=</code>, implicit casts, and <code>.c_str()</code> function. +HIDL string structs has the appropriate copy constructors and assignment +operators to:</p> + +<ul> +<li>Load the HIDL string from an <code>std::string</code> or a C string.</li> +<li>Create a new <code>std::string</code> from a HIDL string.</li> +</ul> + +<p>In addition, HIDL strings have conversion constructors so C strings +(<code>char *</code>) and C++ strings (<code>std::string</code>) can be used on +methods that take a HIDL string.</p> + +<h2 id=struct>struct</h2> +<p>A <code>struct</code> in HIDL can contain only fixed-size data types and no +functions. HIDL struct definitions map directly to standard-layout +<code>struct</code>s in C++, ensuring that <code>struct</code>s have a +consistent memory layout. A struct can include HIDL types, including +<code>handle</code>, <code>string</code>, and <code>vec<T></code>, that +point to separate variable-length buffers.</p> + +<h2 id=handle>handle</h2> + +<p class=warning><strong>WARNING:</strong> Addresses of any kind (even physical +device addresses) must never be part of a native handle. Passing this +information between processes is dangerous and makes them susceptible to attack. +Any values passed between processes must be validated before being used to look +up allocated memory within a process. Otherwise, bad handles may cause bad +memory access or memory corruption.</p> + +<p>The <code>handle</code> type is represented by the <code>hidl_handle</code> +structure in C++, which is a simple wrapper around a pointer to a +<code>const native_handle_t</code> object (this has been present in Android for +a long time).</p> + +<pre> +typedef struct native_handle +{ + int version; /* sizeof(native_handle_t) */ + int numFds; /* number of file descriptors at &data[0] */ + int numInts; /* number of ints at &data[numFds] */ + int data[0]; /* numFds + numInts ints */ +} native_handle_t; +</pre> + +<p>By default, <code>hidl_handle</code> does <strong>not</strong> take ownership +of the <code>native_handle_t</code> pointer it wraps. It merely exists to safely +store a pointer to a <code>native_handle_t</code> such that it can be used in +both 32- and 64-bit processes.</p> + +<p>Scenarios in which the <code>hidl_handle</code> does own its enclosed file +descriptors include:</p> +<ul> +<li>Following a call to the <code>setTo(native_handle_t* handle, bool +shouldOwn)</code> method with the <code>shouldOwn</code> parameter set to +<code>true</code></li> +<li>When the <code>hidl_handle</code> object is created by copy construction +from another <code>hidl_handle</code> object</li> +<li>When the <code>hidl_handle</code> object is copy-assigned from another +<code>hidl_handle</code> object</li> +</ul> + +<p><code>hidl_handle</code> provides both implicit and explicit conversions +to/from <code>native_handle_t* </code>objects. The main use for the +<code>handle</code> type in HIDL is to pass file descriptors over HIDL +interfaces. A single file descriptor is therefore represented by a +<code>native_handle_t</code> with no <code>int</code>s and a single +<code>fd</code>. If the client and server live in a different process, the RPC +implementation will automatically take care of the file descriptor to ensure +both processes can operate on the same file.</p> + +<p>Although a file descriptor received in a <code>hidl_handle</code> by a +process will be valid in that process, it will not persist beyond the receiving +function (it will be closed once the function returns). A process that wants to +retain persistent access to the file descriptor must <code>dup()</code> the +enclosed file descriptors, or copy the entire <code>hidl_handle</code> object. +</p> + +<h2 id=memory>memory</h2> +<p>The HIDL <code>memory</code> type maps to the <code>hidl_memory</code> class +in <code>libhidlbase</code>, which represents unmapped shared memory. This is +the object that must be passed between processes to share memory in HIDL. To +use shared memory:</p> + +<ol> +<li>Obtain an instance of <code>IAllocator</code> (currently only instance +"ashmem" is available) and use it to allocate shared memory.</li> +<li><code>IAllocator::allocate()</code> returns a <code>hidl_memory</code> +object that can be passed through HIDL RPC and be mapped into a process using +<code>libhidlmemory</code>'s <code>mapMemory</code> function.</li> +<li><code>mapMemory</code> returns a reference to an +<code>sp<IMemory></code> object that can be used to access the memory. +(<code>IMemory</code> and <code>IAllocator</code> are defined in +<code>android.hidl.memory@1.0</code>.)</li> +</ol> + +<p>An instance of <code>IAllocator</code> can be used to allocate memory:</p> +<pre class="prettyprint"> +#include <android/hidl/allocator/1.0/IAllocator.h> +#include <android/hidl/memory/1.0/IMemory.h> +#include <hidlmemory/mapping.h> +using ::android::hidl::allocator::V1_0::IAllocator; +using ::android::hidl::memory::V1_0::IMemory; +using ::android::hardware::hidl_memory; +.... + sp<IAllocator> ashmemAllocator = IAllocator::getService("ashmem"); + ashmemAllocator->allocate(2048, [&](bool success, const hidl_memory& mem) { + if (!success) { /* error */ } + // now you can use the hidl_memory object 'mem' or pass it around + })); +</pre> + +<p>Actual changes to the memory must be done through an <code>IMemory</code> +object, either on the side that created <code>mem</code> or on the side that +receives it over HIDL RPC.</p> + +<pre class="prettyprint"> +// Same includes as above + +sp<IMemory> memory = mapMemory(mem); +void* data = memory->getPointer(); +memory->update(); +// update memory however you wish after calling update and before calling commit +data[0] = 42; +memory->commit(); +// … +memory->update(); // the same memory can be updated multiple times +// … +memory->commit(); +</pre> + +<h2 id=interfaces>interface</h2> +<p>Interfaces can be passed as objects. The word <em>interface</em> can be used +as syntactic sugar for the type <code>android.hidl.base@1.0::IBase</code>; +in addition, the current interface and any imported interfaces will be defined +as a type.</p> + +<p>Variables that hold Interfaces should be strong pointers: +<code>sp<IName></code>. HIDL functions that take interface parameters +will convert raw pointers to strong pointers, causing non-intuitive behavior +(the pointer can be cleared unexpectedly). To avoid problems, always store HIDL +interfaces as a <code>sp<></code>.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl-java/constants.html b/en/devices/architecture/hidl-java/constants.html new file mode 100644 index 00000000..30c55b95 --- /dev/null +++ b/en/devices/architecture/hidl-java/constants.html @@ -0,0 +1,87 @@ +<html devsite> + <head> + <title>Exporting Constants</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>In cases where an interface is not Java-compatible (because it uses unions +for example) it may still be desirable to export the constants (enum values) to +the Java world. This scenario is supported by <code>hidl-gen -Ljava-constants +…</code> which extracts annotated enum declarations from the interface +file(s) in a package and produces a java library named +<code>[PACKAGE-NAME]-V[PACKAGE-VERSION]-java-constants</code>. Annotate each +enum declaration to be exported as follows:</p> + +<pre class="prettyprint"> +@export +enum Foo : int32_t { + SOME_VALUE, + SOME_OTHER_VALUE, +}; +</pre> + +<p>If necessary, the name under which this type is exported to the Java world +can be different from that chosen in the interface declaration by adding the +annotation-parameter <code>name</code>:</p> + +<pre class="prettyprint"> +@export(name="JavaFoo") +enum Foo : int32_t { + SOME_VALUE, + SOME_OTHER_VALUE, +}; +</pre> + +<p>If Java conventions or personal preference ask for a common prefix to be +added to the enum type's values, use the annotation-parameter +<code>value_prefix</code>:</p> + +<pre class="prettyprint"> +// File "types.hal". + +package android.hardware.bar@1.0; + +@export(name="JavaFoo", value_prefix="JAVA_") +enum Foo : int32_t { + SOME_VALUE, + SOME_OTHER_VALUE, +}; +</pre> + +<p>The resulting Java class appears as follows:</p> + +<pre class="prettyprint"> +package android.hardware.bar.V1_0; + +public class Constants { + public final class JavaFoo { + public static final int JAVA_SOME_VALUE = 0; + public static final int JAVA_SOME_OTHER_VALUE = 1; + }; +}; +</pre> + +<p>Finally, Java type declaration for enum types declared in +<code>types.hal</code> are grouped inside a class <code>Constants</code> in the +given package. Enum types declared as children of an interface will be grouped +under that interface's Java class declaration.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl-java/index.html b/en/devices/architecture/hidl-java/index.html new file mode 100644 index 00000000..3b94247e --- /dev/null +++ b/en/devices/architecture/hidl-java/index.html @@ -0,0 +1,186 @@ +<html devsite> + <head> + <title>HIDL Java</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>Android O re-architects the Android OS to define clear interfaces between the +device-independent Android platform and device- and vendor-specific code. +Android already defines many such interfaces in the form of HAL interfaces, +defined as C headers in <code>hardware/libhardware</code>. HIDL replaces these +HAL interfaces with stable, versioned interfaces, which can be in Java +(described below) or as client- and server-side HIDL interfaces in +<a href="/devices/architecture/hidl-cpp/index.html">C++</a>.</p> + +<p>HIDL interfaces are intended to be used primarily from native code, and as a +result HIDL is focused on the auto-generation of efficient code in C++. However, +HIDL interfaces must also be able to be used directly from Java as some Android +subsystems (such as Telephony) will most likely have Java HIDL interfaces.</p> + +<p>The pages in this section describe the Java frontend for HIDL interfaces, +detail how to create, register, and use services, and explain how HALs and HAL +clients written in Java interact with the HIDL RPC system.</p> + +<h2 id=client>Being a client</h2> +<p>To access an interface IFoo in package <code>android.hardware.foo</code> +version 1.0 that is registered as service name <code>foo-bar</code>:</p> + +<ol> +<li>Add libraries: + +<ul> +<li>Add the following to Android.mk: +<pre class="prettyprint">LOCAL_JAVA_LIBRARIES += android.hardware.foo-V1.0-java</pre> + +<strong>OR</strong><br> +</li> + +<li>Add the following to Android.bp: +<pre class="prettyprint"> +shared_libs: [ + /* … */ + "android.hardware.foo-V1.0-java", +], +</pre> +The static version of the library is also available as +<code>android.hardware.foo-V1.0-java-static</code>.</li> +</ul> +</li> +<li>Add the following to your Java file: +<pre class="prettyprint"> +import android.hardware.foo.V1_0.IFoo; +... +IFoo server = IFoo.getService(); // throws exception if not available +IFoo anotherServer = IFoo.getService("second_impl"); +server.doSomething(…); +</pre> +</li> +</ol> + +<h2 id=service>Providing a service</h2> +<p>Framework code in Java may need to serve interfaces to receive asynchronous +callbacks from HALs.</p> + +<p class=warning><strong>Warning</strong>: Do not implement a driver (HAL) in +Java. We strongly recommend you implement drivers in C++.</p> + +<p>For interface <code>IFooCallback</code> in version 1.0 of package +<code>android.hardware.foo</code>, you can implement your interface in Java +using the following steps:</p> + +<ol> +<li>Define your interface in HIDL.</li> +<li>Open <code>/tmp/android/hardware/foo/IFooCallback.java</code> as a +reference.</li> +<li>Create a new module for your Java implementation.</li> +<li>Examine the abstract class +<code>android.hardware.foo.V1_0.IFooCallback.Stub</code>, then write a new class +to extend it and implement the abstract methods.</li> +</ol> + +<h3 id=autogen>Viewing auto-generated files</h3> +<p>To view the automatically-generated files, run:</p> +<pre class="prettyprint"> +hidl-gen -o /tmp -Ljava \ + -randroid.hardware:hardware/interfaces \ + -randroid.hidl:system/libhidl/transport android.hardware.foo@1.0 +</pre> + +<p>These commands generate the directory +<code>/tmp/android/hardware/foo/1.0</code>. For the file +<code>hardware/interfaces/foo/1.0/IFooCallback.hal</code>, this generates the +file <code>/tmp/android/hardware/foo/1.0/IFooCallback.java</code>, which +encapsulates the Java interface, the proxy code, and the stubs (both proxy and +stubs conform to the interface).</p> + +<p><code>-Lmakefile</code> generates the rules that run this command at build +time and allow you to include +<code>android.hardware.foo-V1.0-java(-static)?</code> and link against the +appropriate files. A script that automatically does this for a project full of +interfaces can be found at <code>hardware/interfaces/update-makefiles.sh</code>. +The paths in this example are relative; hardware/interfaces can be a temporary +directory under your code tree to enable you to develop a HAL prior to +publishing it.</p> + +<h2 id=service>Running a service</h2> +<p>The HAL provides an interface <code>IFoo</code>, which must make asynchronous +callbacks to the framework over interface <code>IFooCallback</code>. The +<code>IFooCallback</code> interface is not registered by name as a discoverable +service; instead, <code>IFoo</code> must contain a method such as +<code>setFooCallback(IFooCallback x)</code>.</p> + +<p>To set up <code>IFooCallback</code> from version 1.0 of package +<code>android.hardware.foo</code>, add +<code>android.hardware.foo-V1.0-java</code> to <code>Android.mk</code>. The code +to run the service is:</p> + +<pre class="prettyprint"> +import android.hardware.foo.V1_0.IFoo; +import android.hardware.foo.V1_0.IFooCallback.Stub; +.... +class FooCallback extends IFoo.Stub { + // implement methods +} +.... +// Get the service you will be receiving callbacks from. +// This also starts the threadpool for your callback service. +IFoo server = IFoo.getService(); // throws exception if not available +.... +// This must be a persistent instance variable, not local, +// to avoid premature garbage collection. +FooCallback mFooCallback = new FooCallback(); +.... +// Do this once to create the callback service and tell the "foo-bar" service +server.setFooCallback(mFooCallback); +</pre> + +<h2 id=extensions>Interface extensions</h2> +<p>Assuming a given service implements an interface <code>IFoo</code> across all +devices, it's possible that on a particular device the service may provide +additional capabilities implemented in an interface extension +<code>IBetterFoo</code>, i.e.:</p> + +<pre class="prettyprint"> +interface IFoo { + ... +}; + +interface IBetterFoo extends IFoo { + ... +}; +</pre> + +<p>Calling code aware of the extended interface can use the +<code>castFrom()</code> Java method to safely cast the base interface to the +extended interface:</p> + +<pre class="prettyprint"> +IFoo baseService = Foo.getService(); +IBetterFoo extendedService = IBetterFoo.castFrom(baseService); +if (extendedService != null) { + // The service implements the extended interface. +} else { + // The service only implements the base interface. +} +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl-java/interfaces.html b/en/devices/architecture/hidl-java/interfaces.html new file mode 100644 index 00000000..024b64ce --- /dev/null +++ b/en/devices/architecture/hidl-java/interfaces.html @@ -0,0 +1,151 @@ +<html devsite> + <head> + <title>Interface Methods & Errors</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This section details interface methods and errors.</p> + +<h2 id=void>Void methods</h2> +<p>Methods that do not return results are translated into Java methods that +return <code>void</code>. For example, the HIDL declaration:</p> + +<pre class="prettyprint"> +doThisWith(float param); +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +void doThisWith(float param); +</pre> + +<h2 id=single-result>Single-result methods</h2> +<p>Methods that return a single result are translated into their Java +equivalents also returning a single result. For example, the following:</p> + +<pre class="prettyprint"> +doQuiteABit(int32_t a, int64_t b, + float c, double d) generates (double something); +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +double doQuiteABit(int a, long b, float c, double d); +</pre> + + +<h2 id=multiple-result>Multiple-result methods</h2> +<p>For each method that returns more than one result, a callback class is +generated that supplies all the results in its <code>onValues</code> method. +That callback acts as an additional parameter to the method. For example, the +following:</p> + +<pre class="prettyprint"> +oneProducesTwoThings(SomeEnum x) generates (double a, double b); +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +public interface oneProducesTwoThingsCallback { + public void onValues(double a, double b); +} +void oneProducesTwoThings(byte x, oneProducesTwoThingsCallback cb); +</pre> + +<p>A caller of <code>oneProducesTwoThings()</code> would typically use an +anonymous inner class or lambda to implement the callback locally:</p> + +<pre class="prettyprint"> +someInstanceOfFoo.oneProducesTwoThings( + 5 /* x */, + new IFoo.oneProducesTwoThingsCallback() { + @Override + void onValues(double a, double b) { + // do something interesting with a and b. + ... + }}); +</pre> + +<p>or:</p> + +<pre class="prettyprint"> +someInstanceOfFoo.oneProducesTwoThings(5 /* x */, + (a, b) -> a > 3.0 ? f(a, b) : g(a, b))); +</pre> + +<p>You can also define a class to use as a callback … </p> + +<pre class="prettyprint"> +class MyCallback implements oneProducesTwoThingsCallback { + public void onValues(double a, double b) { + // do something interesting with a and b. + } +} +</pre> + +<p>… and pass an instance of <code>MyCallback</code> as the third parameter to +<code>oneProducesTwoThings()</code>.</p> + +<h2 id=errors>Transport errors and death recipients</h2> +<p>Because service implementations can run in a different process, in some cases +the client can stay alive even when the process implementing an interface dies. +Calls on an interface object hosted in a dead process fail with a transport +error (a runtime exception thrown by the called method). Recovery from such a +failure is possible by requesting a new instance of the service by calling +<code>I<InterfaceName>.getService()</code>. However, this method works +only if the process that crashed has restarted and re-registered its services +with the servicemanager (which is generally true for HAL implementations).</p> + +<p>Clients of an interface can also register a <em>death recipient</em> to get a +notification when a service dies. Transport errors can still occur if a call is +made just as the server dies. To register for such notifications on a retrieved +<code>IFoo</code> interface, a client can do the following:</p> + +<pre class="prettyprint"> +foo.linkToDeath(recipient, 1481 /* cookie */); +</pre> + +<p>The <code>recipient</code> parameter must be an implementation of the +interface <code>HwBinder.DeathRecipient</code> provided by HIDL. The interface +contains a single method <code>serviceDied()</code> that is called when the +process hosting the interface dies.</p> + +<pre class="prettyprint"> +final class DeathRecipient implements HwBinder.DeathRecipient { + @Override + public void serviceDied(long cookie) { + // Deal with service going away + } +} +</pre> + +<p>The <code>cookie</code> parameter contains the cookie that was passed with +the call to <code>linkToDeath()</code>. It's also possible to unregister a death +recipient after registering it using:</p> + +<pre class="prettyprint"> +foo.unlinkToDeath(recipient); +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl-java/types.html b/en/devices/architecture/hidl-java/types.html new file mode 100644 index 00000000..8cc2d258 --- /dev/null +++ b/en/devices/architecture/hidl-java/types.html @@ -0,0 +1,161 @@ +<html devsite> + <head> + <title>Data Types</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>Given a HIDL interface file, the Java HIDL backend generates Java interfaces, +Stub, and Proxy code. It supports all scalar HIDL types +([<code>u</code>]<code>int</code>{<code>8,16,32,64}_t, float, double,</code> and +<code>enum</code>s), as well as strings, interfaces, struct types, and arrays +and vectors of supported HIDL types. The Java HIDL backend <strong>does NOT +support union types, native handles, shared memory, and fmq types</strong>.</p> + +<p>As Java runtime does not support the concept of unsigned integers natively, +all unsigned types (and enums based on them) are silently treated as their +signed equivalents, i.e. <code>uint32_t</code> becomes an <code>int</code> in +the Java interface. No value conversion is performed; the implementor on the +Java side must use the signed values as if they were unsigned.</p> + +<h2 id=enum>Enums</h2> +<p>Enums do not generate Java enum classes but are instead translated into inner +classes containing a static constant definition for each enum case. If the enum +class derives from some other enum class, it inherits that class's storage type. +Enumerations based on an unsigned integer type are rewritten into their signed +equivalent.</p> + +<p>For example, a <code>SomeBaseEnum</code> with a type of +<code>uint8_t</code>:</p> + +<pre class="prettyprint"> +enum SomeBaseEnum : uint8_t { foo = 3 }; +enum SomeEnum : SomeBaseEnum { + quux = 33, + goober = 127 +}; +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +public final class SomeBaseEnum { public static final byte foo = 3; } +public final class SomeEnum { + public static final byte foo = 3; + public static final byte quux = 33; + public static final byte goober = 127; +} +</pre> + +<p>And:</p> + +<pre class="prettyprint"> +enum SomeEnum : uint8_t { + FIRST_CASE = 10, + SECOND_CASE = 192 +}; +</pre> + +<p>… is rewritten as:</p> + +<pre class="prettyprint"> +public final class SomeEnum { + static public final byte FIRST_CASE = 10; // no change + static public final byte SECOND_CASE = -64; +} +</pre> + +<h2 id=string>Strings</h2> +<p><code>String</code>s in Java are utf-8 or utf-16 but are converted to utf-8 +as the common HIDL type when transported. Additionally, a <code>String</code> +must not be null when passed into HIDL.</p> + +<h2 id=array-vect>Arrays and vectors</h2> +<p>Arrays are translated into Java arrays and vectors are translated into +<code>ArrayList<T></code> where T is the appropriate object type, possibly +wrapping scalar types such as <code>vec<int32_t> => +ArrayList<Integer></code>). For example:</p> + +<pre class="prettyprint"> +takeAnArray(int32_t[3] array); +returnAVector() generates (vec<int32_t> result); +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +void takeAnArray(int[] array); +ArrayList<Integer> returnAVector(); +</pre> + +<h2 id=struct>Structures</h2> +<p>Structures are translated into Java classes with a similar layout. For +example:</p> + +<pre class="prettyprint"> +struct Bar {vec<bool> someBools; + }; + struct Foo { + int32_t a; + int8_t b; + float[10] c; + Bar d; + }; +</pre> + +<p>… becomes:</p> + +<pre class="prettyprint"> +class Bar { + public final ArrayList<Boolean> someBools = new ArrayList(); +}; +class Foo { + public int a; + public byte b; + public final float[] c = new float[10]; + public final Bar d = new Bar(); +} +</pre> + +<h2 id=declared>Declared types</h2> +<p>Each top-level type declared in <code>types.hal</code> gets its own .java +output file (as required by Java). For example, the following +<code>types.hal</code> file results in two extra files being created (Foo.java +and Bar.java):</p> + +<pre class="prettyprint"> +struct Foo { + ... +}; + +struct Bar { + ... + + struct Baz { + }; + + ... +}; +</pre> + +<p>The definition of Baz lives in a static inner class of Bar (in Bar.java).</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/binder-ipc.html b/en/devices/architecture/hidl/binder-ipc.html new file mode 100644 index 00000000..811bc3b7 --- /dev/null +++ b/en/devices/architecture/hidl/binder-ipc.html @@ -0,0 +1,315 @@ +<html devsite> + <head> + <title>Using Binder IPC</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>This page describes changes to the binder driver in Android O, provides +details on using binder IPC, and lists required SELinux policy.</p> + +<h2 id=binder-changes>Changes to binder driver</h2> + +<p>Starting in Android O, the Android framework and HALs now communicate with +each other using binder. As this communication dramatically increases binder +traffic, Android O includes several improvements designed to keep binder IPC +fast. SoC vendors and OEMs integrating the latest version of the driver should +review the list of these improvements, relevant SHAs for the 3.18, 4.4, and 4.9 +kernels, and required userspace changes.</p> + +<h3 id=contexts>Multiple binder domains (contexts)</h3> +<em>In common-3.10, common-3.18, common-4.4, common-4.9, and upstream</em> +<p> +To cleanly split the binder traffic between framework (device-independent) and +vendor (device-specific) code, Android O introduces the concept of a <em>binder +context</em>. Each binder context has its own device node and its own context +(service) manager. You can access the context manager only through the device +node to which it belongs and, when passing a binder node through a certain +context, it is accessible from that same context only by another process, thus +completely isolating the domains from each other. For details on using, see +<a href="#vndbinder">vndbinder</a> and +<a href="#vndservicemanager">vndservicemanager</a>. +</p> + + +<h3 id=scatter>Scatter-gather</h3> +<em>In common-3.10, common-3.18, common-4.4, common-4.9, and upstream</em> +<p> +In previous releases of Android, every piece of data in a binder call was copied +three times: +</p> +<ul> +<li>Once to serialize it into a +<code><a href="https://developer.android.com/reference/android/os/Parcel.html">Parcel</a></code> +in the calling process</li> +<li>Once in the kernel driver to copy the <code>Parcel</code> to the target +process</li> +<li>Once to unserialize the <code>Parcel</code> in the target process</li> +</ul> + +<p> +Android O uses +<a href="https://en.wikipedia.org/wiki/Vectored_I/O">scatter-gather +optimization</a> to reduce the number of copies from 3 to 1. Instead of +serializing data in a <code>Parcel</code> first, data remains in its original +structure and memory layout and the driver immediately copies it to the target +process. After the data is in the target process, the structure and memory +layout is the same and the data can be read without requiring another copy. +</p> + +<h3 id=locking>Fine-grained locking</h3> +<em>In common-3.18, common-4.4, common-4.9, and upstream</em> + +<p> +In previous Android releases, the binder driver used a global lock to protect +against concurrent access to critical data structures. While there was minimal +contention for the lock, the main problem was that if a low-priority thread +obtained the lock and then got preempted, it could seriously delay +higher-priority threads needing to obtain the same lock. This caused jank in the +platform. +</p> +<p> +Initial attempts to resolve this problem involved disabling preemption while +holding the global lock. However, this was more of a hack than a true solution, +and was eventually rejected by upstream and discarded. Subsequent attempts +focused on making locking more fine-grained, a version of which has been running +on Pixel devices since January 2017. While the majority of those changes were +made public, substantial improvements were made in future versions. +</p> +<p> +After identifying small issues in the fine-grained locking implementation, we +devised an improved solution with a different locking architecture and submitted +the changes in the 3.18, 4.4, and 4.9 common branches. We continue to test this +implementation on a large number of different devices; as we are unaware of any +outstanding issues, this is the recommended implementation for devices shipping +with Android O. +</p> +<p class="note"><strong>Note:</strong> We strongly encourage budgeting +sufficient testing hours for fine-grained locking. +</p> +<h3 id=rt-priority>Real-time priority inheritance</h3> +<em>In common-3.18, common-4.4, common-4.9 (upstream coming soon)</em> + +<p> +The binder driver has always supported nice priority inheritance. As an +increasing number of processes in Android run at real-time priority, in some +cases it now makes sense that if a real-time thread makes a binder call, the +thread in the process that handles that call also runs at real-time priority. To +support these use cases, Android O now implements real-time priority inheritance +in the binder driver. +</p> +<p> +In addition to transaction-level priority inheritance, <em>node priority +inheritance</em> allows a node (binder service object) to specify a minimum +priority at which calls into this node should be executed. Previous versions of +Android already supported node priority inheritance with nice values, but +Android O adds support for real-time scheduling policies node inheritance. +</p> +<p class="note"><strong>Note:</strong> The Android performance team found that +real-time priority inheritance caused unwanted side-effects in the framework +binder domain (<code>/dev/binder</code>), so real-time priority inheritance is +<strong>disabled</strong> for that domain. +</p> + +<h3 id=userspace>Userspace changes</h3> +<p> +Android O includes all userspace changes required to work with the current +binder driver in the common kernel with one exception: The original +implementation to disable real-time priority inheritance for +<code>/dev/binder</code> used an +<a href="https://android.googlesource.com/kernel/msm/+/868f6ee048c6ff51dbd92353dd5c68bea4419c78" class="external">ioctl</a>. Subsequent development switched control of priority +inheritance to a more fine-grained method that is per binder mode (and not per +context). Thus, the ioctl is not in the Android common branch and is instead +<a href="https://android-review.googlesource.com/#/c/421861/" class="external">submitted +in our common kernels</a>. +</p> +<p> +The effect of this change is that real-time priority inheritance is disabled by +default for <em>every</em> node. The Android performance team has found it +beneficial to enable real-time priority inheritance for all nodes in the +<code>hwbinder</code> domain. To achieve that same effect, cherry-pick +<a href="https://android-review.googlesource.com/#/c/440359/" class="external">this +change</a> in userspace. +</p> +<h3 id=shas>SHAs for common kernels</h3> +<p> +To obtain necessary changes to the binder driver, sync to the SHAs below (or +later): +</p> +<ul> +<li>Common-3.18<br> +cc8b90c121de ANDROID: binder: don't check prio permissions on restore.</li> +<li>Common-4.4<br> +76b376eac7a2 ANDROID: binder: don't check prio permissions on restore.</li> +<li>Common-4.9<br> +ecd972d4f9b5 ANDROID: binder: don't check prio permissions on restore.</li> +</ul> + +<h2 id=ipc>Using binder IPC</h2> +<p>Historically, vendor processes have used binder interprocess communication +(IPC) to communicate. In Android O, the <code>/dev/binder</code> device node +becomes exclusive to framework processes, meaning vendor processes no longer +have access to it. Vendor processes can access <code>/dev/hwbinder</code>, but +must convert their AIDL interfaces to use HIDL. For vendors who want to continue +using AIDL interfaces between vendor processes, Android supports binder IPC as +described below.</p> + +<h3 id=vndbinder>vndbinder</h3> +<p>Android O supports a new binder domain for use by vendor services, accessed +using <code>/dev/vndbinder</code> instead of <code>/dev/binder</code>. With the +addition of <code>/dev/vndbinder</code>, Android now has the following three +IPC domains:</p> + +<table> + <tr> + <th>IPC Domain</th> + <th>Description</th> + </tr> + <tr> + <td><code>/dev/binder</code></td> + <td>IPC between framework/app processes with AIDL interfaces</td> + </tr> + <tr> + <td><code>/dev/hwbinder</code></td> + <td>IPC between framework/vendor processes with HIDL interfaces + <br>IPC between vendor processes with HIDL interfaces</td> + </tr> + <tr> + <td><code>/dev/vndbinder</code></td> + <td>IPC between vendor/vendor processes with AIDL Interfaces</td> + </tr> +</table> + +<p>For <code>/dev/vndbinder</code> to appear, ensure the kernel configuration +item <code>CONFIG_ANDROID_BINDER_DEVICES</code> is set to +<code>"binder,hwbinder,vndbinder"</code> (this is the default in Android's +common kernel trees).</p> + +<p>Normally, vendor processes don't open the binder driver directly and instead +link against the <code>libbinder</code> userspace library, which opens the +binder driver. Adding a method for <code>::android::ProcessState()</code> +selects the binder driver for <code>libbinder</code>. Vendor processes should +call this method <strong>before</strong> calling into <code>ProcessState,</code> +<code>IPCThreadState</code>, or before making any binder calls in general. To +use, place the following call after the <code>main()</code> of a vendor process +(client and server):</p> + +<pre class="prettyprint">ProcessState::initWithDriver("/dev/vndbinder");</pre> + +<h3 id=vndservicemanager>vndservicemanager</h3> +<p>Previously, binder services were registered with <code>servicemanager</code>, +where they could be retrieved by other processes. In Android O, +<code>servicemanager</code> is now used exclusively by framework and app +processes and vendor processes can no longer access it.</p> + +<p>However, vendor services can now use <code>vndservicemanager</code>, a new +instance of <code>servicemanager</code> that uses <code>/dev/vndbinder</code> +instead of <code>/dev/binder</code> and which is built from the same sources as +framework <code>servicemanager</code>. Vendor processes do not need to make +changes to talk to <code>vndservicemanager</code>; when a vendor process opens +/<code>dev/vndbinder</code>, service lookups automatically go to +<code>vndservicemanager</code>.</p> + +<p>The <code>vndservicemanager</code> binary is included in Android's default +device makefiles.</p> + +<h2 id=selinux>SELinux policy</h2> +<p>Vendor processes that want to use binder functionality to communicate with +each other need the following:</p> +<ol> +<li>Access to <code>/dev/vndbinder</code>.</li> +<li>Binder <code>{transfer, call}</code> hooks into +<code>vndservicemanager</code>.</li> +<li><code>binder_call(A, B)</code> for any vendor domain A that wants to call +into vendor domain B over the vendor binder interface.</li> +<li>Permission to <code>{add, find}</code> services in +<code>vndservicemanager</code>.</li> +</ol> + +<p>To fulfill requirements 1 and 2, use the <code>vndbinder_use()</code> +macro:</p> + +<pre class="prettyprint">vndbinder_use(some_vendor_process_domain);</pre> + +<p>To fulfill requirement 3, the <code>binder_call(A, B)</code> for vendor +processes A and B that need to talk over binder can stay in place, and doesn't +need renaming.</p> + +<p>To fulfill requirement 4, you must make changes in the way service names, +service labels, and rules are handled.</p> + +<p>For details on SELinux, see +<a href="https://source.android.com/security/selinux/">Security-Enhanced Linux +in Android</a>. For details on SELinux in Android 8.0, see +<a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for Android +8.0</a>.</p> + +<h3 id=names>Service names</h3> +<p>Previously, vendor processes registered service names in a +<code>service_contexts</code> file and added corresponding rules for accessing +that file. Example <code>service_contexts</code> file from +<code>device/google/marlin/sepolicy</code>:</p> + +<pre class="devsite-click-to-copy"> +AtCmdFwd u:object_r:atfwd_service:s0 +cneservice u:object_r:cne_service:s0 +qti.ims.connectionmanagerservice u:object_r:imscm_service:s0 +rcs u:object_r:radio_service:s0 +uce u:object_r:uce_service:s0 +vendor.qcom.PeripheralManager u:object_r:per_mgr_service:s0 +</pre> + +<p>In Android O, <code>vndservicemanager</code> loads the +<code>vndservice_contexts</code> file instead. Vendor services migrating to +<code>vndservicemanager</code> (and which are already in the old +<code>service_contexts</code> file) should be added to the new +<code>vndservice_contexts</code> file.</p> + +<h3 id=labels>Service labels</h3> +<p>Previously, service labels such as <code>u:object_r:atfwd_service:s0</code> +were defined in a <code>service.te</code> file. Example:</p> + +<pre class="prettyprint">type atfwd_service, service_manager_type;</pre> + +<p>In Android O, you must change the type to +<code>vndservice_manager_type</code> and move the rule to the +<code>vndservice.te</code> file. Example:</p> + +<pre class="prettyprint">type atfwd_service, vndservice_manager_type;</pre> + +<h3 id=rules>Servicemanager rules</h3> +<p>Previously, rules granted domains access to add or find services from +<code>servicemanager</code>. Example:</p> + +<pre class="prettyprint"> +allow atfwd atfwd_service:service_manager find; +allow some_vendor_app atfwd_service:service_manager add; +</pre> + +<p>In Android O, such rules can stay in place and use the same class. Example: +</p> + +<pre class="prettyprint"> +allow atfwd atfwd_service:service_manager find; +allow some_vendor_app atfwd_service:service_manager add; +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl/code-style.html b/en/devices/architecture/hidl/code-style.html new file mode 100644 index 00000000..0746ff36 --- /dev/null +++ b/en/devices/architecture/hidl/code-style.html @@ -0,0 +1,718 @@ +<html devsite> + <head> + <title>Code Style Guide</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>The HIDL code style resembles C++ code in the Android framework, with 4-space +indents and mixed-case filenames. Package declarations, imports, and docstrings +are similar to those in Java, with slight modifications.</p> + +<p>The following examples for <code>IFoo.hal</code> and <code>types.hal</code> +illustrate HIDL code styles and provide quick links to details on each style +(<code>IFooClientCallback.hal</code>, <code>IBar.hal</code>, and +<code>IBaz.hal</code> have been omitted).</p> + +<table> +<tr><th><code>hardware/interfaces/foo/1.0/IFoo.hal</code></th></tr> +<tr><td><pre class="prettyprint"> +/* + * (License Notice) + */ + +<a href="#package-names">package</a> android.hardware.foo@<a href="#versions">1.0</a>; + +<a href="#imports">import</a> android.hardware.bar@1.0::IBar; + +import IBaz; +import IFooClientCallback; + +/** + * IFoo is an interface that… + */ +interface <a href="#interface-names">IFoo</a> { + + /** + * This is a <a href="#comments">multiline docstring</a>. + * <a href="#return">@return</a> result 0 if successful, nonzero otherwise. + */ + <a href="#function-declarations">foo() generates (FooStatus result);</a> + + /** + * Restart controller by power cycle. + * <a href="#param">@param</a> bar callback interface that… + * @return result 0 if successful, nonzero otherwise. + */ + <a href="#functions">powerCycle</a>(IBar bar) generates (FooStatus <a href="#functions">result</a>); + + /** <a href="#comments">Single line docstring</a>. */ + baz(); + + + /** + * The bar function. + * <a href="#param">@param</a> <a href="#functions">clientCallback</a> callback after function is called + * @param baz related baz object + * @param data input data blob + */ + bar(IFooClientCallback clientCallback, + IBaz baz, + FooData data); + +}; +</pre> +</td></tr></table> + +<table> +<tr><th><code>hardware/interfaces/foo/1.0/types.hal</code></th></tr> +<tr><td><pre class="prettyprint"> +/* + * (License Notice) + */ + +package android.hardware.foo@1.0; + +<a href="#comments">/** Replied status. */</a> +<a href="#enum-declarations">enum Status : int32_t</a> { + <a href="#enum-values">OK</a>, + ERR_ARG, <a href="#comments">// invalid arguments</a> + ERR_UNKNOWN = -1, // note, no transport related errors +}; + +<a href="#struct-declarations">struct ArgData</a> { + <a href="#array-declarations">int32_t[20] someArray;</a> + <a href="#vectors">vec<uint8_t> data;</a> +}; +</pre> +</td></tr></table> + +<h2 id=naming>Naming conventions</h2> +<p>Function names, variable names, and filenames should be descriptive; avoid +over-abbreviation. Treat acronyms as words (e.g., use <code>INfc</code> instead +of <code>INFC</code>.</p> + +<h3 id=dir-structure>Directory structure and file naming</h3> +<p>The directory structure should appear as follows:</p> +<ul> +<li><code><var>ROOT-DIRECTORY</var></code> + <ul> + <li><code><var>MODULE</var></code> + <ul> + <li><code><var>SUBMODULE</var></code> (optional, could be more than one + level) + <ul> + <li><code><var>VERSION</var></code> + <ul> + <li><code>Android.mk</code></li> + <li><code>I<var>INTERFACE_1</var>.hal</code></li> + <li><code>I<var>INTERFACE_2</var>.hal</code></li> + <li><code>…</code></li> + <li><code>I<var>INTERFACE_N</var>.hal</code></li> + <li><code>types.hal</code> (optional)</li> + </ul> + </li> + </ul> + </li> + </ul> + </li> + </ul> +</li> +</ul> + +<p>Where:</p> +<ul> +<li><code><var>ROOT-DIRECTORY</var></code> is: + <ul> + <li><code>hardware/interfaces</code> for core HIDL packages.</li> + <li><code>vendor/<var>VENDOR</var>/interfaces</code> for vendor packages, + where <code><var>VENDOR</var></code> refers to an SoC vendor or an + OEM/ODM.</li> + </ul> + </li> +<li><code><var>MODULE</var></code> should be one lowercase word that describes +the subsystem (e.g. <code>nfc</code>). If more than one word is needed, use +nested <code><var>SUBMODULE</var></code>. There can be more than one level of +nesting.</li> +<li><code><var>VERSION</var></code> should be the exact same version +(major.minor) as described in <a href="#versions">Versions</a>.</li> +<li><code>I<var>INTERFACE_X</var></code> should be the interface name with +<code>UpperCamelCase</code>/<code>PascalCase</code> (e.g. <code>INfc</code>) +as described in <a href="#interface-names">Interface names</a>.</li> +</ul> + +<p>Example:</p> +<ul> +<li><code>hardware/interfaces</code> + <ul> + <li><code>nfc</code> + <ul> + <li><code>1.0</code> + <ul> + <li><code>Android.mk</code></li> + <li><code>INfc.hal</code></li> + <li><code>INfcClientCallback.hal</code></li> + <li><code>types.hal</code></li> + </ul> + </li> + </ul> + </li> + </ul> +</li> +</ul> + +<p class="note"><strong>Note:</strong> All files must have non-executable +permissions (in Git). + +<h3 id=package-names>Package names</h3> +<p>Package names must use the following <a href="#fqn">fully-qualified name +(FQN)</a> format (referred to as <code><var>PACKAGE-NAME</var></code>):</p> + +<pre class="prettyprint"> +<var>PACKAGE</var>.<var>MODULE</var>[.<var>SUBMODULE</var>[.<var>SUBMODULE</var>[…]]]@<var>VERSION</var> +</pre> + +<p>Where:</p> +<ul> +<li><code><var>PACKAGE</var></code> is the package that maps to the +<code><var>ROOT-DIRECTORY</var></code>. In particular, +<code><var>PACKAGE</var></code> is: + <ul> + <li><code>android.hardware</code> for core HIDL packages (mapping to + <code>hardware/interfaces</code>).</li> + <li><code>vendor.<var>VENDOR</var>.hardware</code> for vendor packages, where + <code><var>VENDOR</var></code> refers to an SoC vendor or an OEM/ODM (mapping + to <code>vendor/<var>VENDOR</var>/interfaces</code>).</li> + </ul> +<li><code><var>MODULE</var>[.<var>SUBMODULE</var>[.<var>SUBMODULE</var>[…]]]@<var>VERSION</var></code> +are the exact same folder names in the structure described in +<a href="#dir-structure">Directory structure</a>.</li> +<li>Package names should be lowercase. If they are more than one word long, the +words should either be used as submodules or written in <code>snake_case</code>. +</li> +<li>No spaces are allowed.</li> +</ul> + +<p>The FQN is always used in package declarations.</p> + +<h3 id=versions>Versions</h3> +<p> +Versions should have the following format: +</p> + +<pre class="prettyprint"><var>MAJOR</var>.<var>MINOR</var></pre> + +<p>Both the <var>MAJOR</var> and the <var>MINOR</var> version should be a single +integer. HIDL uses <a href="http://semver.org/" class="external">semantic +versioning</a> rules.</p> + +<h3 id=imports>Imports</h3> +<p>An import has one of the following three formats:</p> +<ul> +<li>Whole-package imports: <code>import <var>PACKAGE-NAME</var>;</code></li> +<li>Partial imports: <code>import +<var>PACKAGE-NAME</var>::<var>UDT</var>;</code> (or, if the imported +type is in the same package,<code>import <var>UDT</var>;</code></li> +<li>Types-only imports: <code>import <var>PACKAGE-NAME</var>::types;</code></li> +</ul> + +<p>The <code><var>PACKAGE-NAME</var></code> follows the format in +<a href="#package-names">Package names</a>. The current package's +<code>types.hal</code> (if it exists) is automatically imported (do not import +it explicitly).</p> + +<h4 id=fqn>Fully qualified names (FQNs)</h4> +<p>Use fully qualified names for a user-defined type import only when necessary. +Omit <code><var>PACKAGE-NAME</var></code> if the import type is in the same +package. An FQN must not contain spaces. Example of a fully qualified name:</p> + +<pre class="prettyprint">android.hardware.nfc@1.0::INfcClientCallback</pre> + +<p>In another file under <code>android.hardware.nfc@1.0</code>, refer to the +above interface as <code>INfcClientCallback</code>. Otherwise, use only the +fully qualified name.</p> + +<h4 id=grouping>Grouping and ordering imports</h4> +<p>Use an empty line after package declaration (before the imports). Each import +should occupy a single line and should not be indented. Group imports in the +following order:</p> + +<ol> +<li>Other <code>android.hardware</code> packages (use fully qualified names). +</li> +<li>Other <code>vendor.<var>VENDOR</var></code> packages (use fully qualified +names). + <ul> + <li>Each vendor should be a group.</li> + <li>Order vendors alphabetically.</li> + </ul> + </li> +<li>Imports from other interfaces in the same package (use simple names).</li> +</ol> + +<p>Use an empty line between groups. Inside each group, sort imports +alphabetically. Example:</p> + +<pre class="prettyprint"> +import android.hardware.nfc@1.0::INfc; +import android.hardware.nfc@1.0::INfcClientCallback; + +// Importing the whole module. +import vendor.barvendor.bar@3.1; + +import vendor.foovendor.foo@2.2::IFooBar; +import vendor.foovendor.foo@2.2::IFooFoo; + +import IBar; +import IFoo; +</pre> + +<h3 id=interface-names>Interface names</h3> +<p>Interface names must start with an <code>I</code>, followed by an +<code>UpperCamelCase</code>/<code>PascalCase</code> name. An interface with name +<code>IFoo</code> must be defined in the file <code>IFoo.hal</code>. This file +can contain definitions only for the <code>IFoo</code> interface (the interface +<code>I<var>NAME</var></code> should be in <code>I<var>NAME</var>.hal</code> ). +</p> + +<h3 id=functions>Functions</h3> +<p>For function names, arguments, and return variable names, use +<code>lowerCamelCase</code>. Example:</p> + +<pre class="prettyprint"> +open(INfcClientCallback clientCallback) generates (int32_t retVal); +oneway pingAlive(IFooCallback cb); +</pre> + +<h3 id=struct-union>Struct/union field names</h3> +<p>For struct/union field names, use <code>lowerCamelCase</code>. Example:</p> + +<pre class="prettyprint"> +struct FooReply { + vec<uint8_t> replyData; +} +</pre> + +<h3 id=type-names>Type names</h3> +<p>Type names refer to struct/union definitions, enum type definitions, and +<code>typedef</code>s. For these name, use +<code>UpperCamelCase</code>/<code>PascalCase</code>. Examples:</p> + +<pre class="prettyprint"> +enum NfcStatus : int32_t { + /*...*/ +}; +struct NfcData { + /*...*/ +}; +</pre> + +<h3 id=enum-values>Enum values</h3> +<p>Enum values should be <code>UPPER_CASE_WITH_UNDERSCORES</code>. When passing +enum values as function arguments and returning them as function returns, use +the actual enum type (not the underlying integer type). Example:</p> + +<pre class="prettyprint"> +enum NfcStatus : int32_t { + HAL_NFC_STATUS_OK = 0, + HAL_NFC_STATUS_FAILED = 1, + HAL_NFC_STATUS_ERR_TRANSPORT = 2, + HAL_NFC_STATUS_ERR_CMD_TIMEOUT = 3, + HAL_NFC_STATUS_REFUSED = 4 +}; +</pre> + +<p class="note"><strong>Note:</strong> The underlying type of an enum type is +explicitly declared after the colon. As it is not compiler dependent, using the +actual enum type is clearer.</p> + +<p>For fully qualified names for enum values, a <strong>colon</strong> is used +between the enum type name and the enum value name:</p> + +<pre class="prettyprint"> +<var>PACKAGE-NAME</var>::<var>UDT</var>[.<var>UDT</var>[.<var>UDT</var>[…]]:<var>ENUM_VALUE_NAME</var> +</pre> + +<p>There must not be spaces inside a fully qualified name. Use a fully qualified +name only when necessary and omit unnecessary parts. Example:</p> + +<pre class="prettyprint"> +android.hardware.foo@1.0::IFoo.IFooInternal.FooEnum:ENUM_OK +</pre> + +<h2 id=comments>Comments</h2> +<p>For a single line comment, both <code>// </code>and <code>/** */</code> are +fine.</p> +<pre class="prettyprint"> +// This is a single line comment +/* This is also single line comment */ +/** This is documentation comment */ +</pre> + +<ul> +<li>Use <code>//</code> mainly for: + <ul> + <li>trailing comments</li> + <li>Comments that will not be used for generated documentation</li> + <li>TODOs</li> + </ul> +</li> +<li>Use <code>/** */</code> mainly for Function documents/"docstrings" used for +generated documentation. Example: +<pre class="prettyprint"> +/** Replied status */ +enum FooStatus { + OK = 0, // no error + ERR_TRANSPORT = 1, // transport level error + ERR_ARG = 2 // invalid args +} +</pre> +<li>Multi-line comments should start a new line with <code>/**</code>, use +<code>*</code> at the beginning of each line, and place <code>*/</code> on the +last line all on its own (asterisks should align). Example: +<pre class="prettyprint"> +/** + * My multi-line + * comment + */ +</pre> +</li> +<li>Licensing notice and changelogs should start a new line with <code>/*</code> +(a single asterisk), use <code>*</code> at the beginning of each line, and place +<code>*/</code> on the last line all on its own (asterisks should align). +Example: +<pre class="prettyprint"> +/* + * Copyright (C) 2017 The Android Open Source Project + * ... + */ + +/* + * Changelog: + * ... + */ +</pre> +</li> +</ul> + +<h3 id=file-comments>File comments</h3> +<p>Start each file with the appropriate licensing notice. For core HALs, this +should be the AOSP Apache license in +<a href="https://android.googlesource.com/platform/development/+/master/docs/copyright-templates/c.txt" class="external"><code>development/docs/copyright-templates/c.txt</code></a>. +Remember to update the year and use <code>/* */</code> style multi-line comments +as explained above.</p> + +<p>You can optionally place an empty line after the license notice, followed +by a changelog/versioning information. Use <code>/* */</code> style +multi-line comments as explained above, place the empty line after the +changelog, then follow with the package declaration.</p> + +<h3 id=todo-comments>TODO comments</h3> +<p>TODOs should include the string <code>TODO</code> in all caps followed by a +colon. Example:</p> +<pre>// TODO: remove this code before foo is checked in.</pre> + +<p class="alert">TODO comments are allowed only during development; they must +not exist in published interfaces.</p> + +<h3 id=docstrings>Interface/Function comments (docstrings)</h3> +<p>Use <code>/** */</code> for multi-line and single line docstrings. Do not use +<code>//</code> for docstrings.</p> + +<p>Docstrings for interfaces should describe general mechanisms of the +interface, design rationale, purpose, etc. Docstrings for functions should be +specific to the function (package-level documentation goes in a README file in +the package directory).</p> + +<pre class="prettyprint"> +/** + * IFooController is the controller for foos. + */ +interface IFooController { + /** + * Opens the controller. + * @return status HAL_FOO_OK if successful. + */ + open() generates (FooStatus status); + + /** Close the controller. */ + close(); +}; +</pre> + +<p>You must add <code>@param</code>s and <code>@return</code>s for each +parameter/return value:</p> + +<ul> +<li id=param><code>@param</code> must be added for each parameter. It should be +followed by the name of the parameter then the docstring.</li> +<li id=return><code>@return</code> must be added for each return value. It +should be followed by the name of the return value then the docstring.</li> +</ul> +<p>Example:</p> + +<pre class="prettyprint"> +/** + * Explain what foo does. + * @param arg1 explain what arg1 is + * @param arg2 explain what arg2 is + * @return ret1 explain what ret1 is + * @return ret2 explain what ret2 is + */ +foo(T arg1, T arg2) generates (S ret1, S ret2); +</pre> + +<h2>Formatting</h2> +<p>General formatting rules include:</p> +<ul> +<li><strong>Line length</strong>. Each line of text should be at most +<strong>80</strong> columns long.</li> +<li><strong>Whitespaces</strong>. No trailing whitespace on lines; empty lines +must not contain whitespaces.</li> +<li><strong>Spaces vs. tabs</strong>. Use only spaces.</li> +<li><strong>Indent size</strong>. Use <strong>4</strong> spaces for blocks and +<strong>8</strong> spaces for line wraps</li> +<li><strong>Bracing</strong>. Except for <a href="#annotations">annotation +values</a>, an <strong>open</strong> brace goes on the same line as preceding +code but a <strong>close</strong> brace and the following semicolon occupies +the entire line. Example: +<pre class="prettyprint"> +interface INfc { + close(); +}; +</pre> +</li> +</ul> + +<h3 id=package-declarations>Package declaration</h3> +<p>Package declaration should be at the top of the file after the license +notice, should occupy the entire line, and should not be indented. Packages are +declared using the following format (for name formatting, see +<a href="#package-names">Package names</a>):</p> + +<pre class="prettyprint"> +package <var>PACKAGE-NAME</var>; +</pre> + +<p>Example:</p> + +<pre class="prettyprint"> +package android.hardware.nfc@1.0; +</pre> + +<h3 id=function-declarations>Function declarations</h3> +<p>Function name, parameters, <code>generates</code>, and return values should +be on the same line if they fit. Example:</p> +<pre class="prettyprint"> +interface IFoo { + /** ... */ + easyMethod(int32_t data) generates (int32_t result); +}; +</pre> + +<p>If they don't fit on the same line, attempt to put parameters and return +values in the same indent level and distinguish <code>generate</code> to help +the reader quickly see the parameters and return values. Example:</p> + +<pre class="prettyprint"> +interface IFoo { + suchALongMethodThatCannotFitInOneLine(int32_t theFirstVeryLongParameter, + int32_t anotherVeryLongParameter); + anEvenLongerMethodThatCannotFitInOneLine(int32_t theFirstLongParameter, + int32_t anotherVeryLongParameter) + generates (int32_t theFirstReturnValue, + int32_t anotherReturnValue); + superSuperSuperSuperSuperSuperSuperLongMethodThatYouWillHateToType( + int32_t theFirstVeryLongParameter, // 8 spaces + int32_t anotherVeryLongParameter + ) generates ( + int32_t theFirstReturnValue, + int32_t anotherReturnValue + ); + // method name is even shorter than 'generates' + foobar(AReallyReallyLongType aReallyReallyLongParameter, + AReallyReallyLongType anotherReallyReallyLongParameter) + generates (ASuperLongType aSuperLongReturnValue, // 4 spaces + ASuperLongType anotherSuperLongReturnValue); +} +</pre> + +<p>Additional details:</p> +<ul> +<li>An open parenthesis is always on the same line as the function name.</li> +<li>No spaces between the function name and the open parenthesis.</li> +<li>No spaces between the parentheses and parameters <em>except</em> when there +are line feeds between them.</li> +<li>If <code>generates</code> is on the same line as the previous closing +parenthesis, use a preceding space. If <code>generates</code> is on the same +line as the next open parenthesis, follow with a space.</li> +<li>Align all parameters and return values (if possible).</li> +<li>Default indentation is 4 spaces.</li> +<li>Wrapped parameters are aligned to the first parameters on the previous line, +otherwise they have an 8-space indent.</li> +</ul> + +<h3 id=annotations>Annotations</h3> +<p>Use the following format for annotations:</p> + +<pre class="prettyprint"> +@annotate(keyword = value, keyword = {value, value, value}) +</pre> + +<p>Sort annotations in alphabetical order, and use spaces around equal signs. +Example:</p> + +<pre class="prettyprint"> +@callflow(key = value) +@entry +@exit +</pre> + +<p>Ensure an annotation occupies the entire line. Examples:</p> +<pre class="prettyprint"> +// Good +@entry +@exit + +// Bad +@entry @exit +</pre> + +<p>If annotations cannot fit on the same line, indent with 8 spaces. +Example:</p> + +<pre class="prettyprint"> +@annotate( + keyword = value, + keyword = { + value, + value + }, + keyword = value) +</pre> + +<p>If the entire value array cannot fit in the same line, put line breaks after +open braces <code>{</code> and after each comma inside the array. Place closing +parenthesis immediately after the last value. Do not put the braces if there is +only one value.</p> + +<p>If the entire value array can fit in the same line, do not use spaces after +open braces and before closing braces and use one space after each comma. +Examples:</p> + +<pre class="prettyprint"> +// Good +@callflow(key = {"val", "val"}) + +// Bad +@callflow(key = { "val","val" }) +</pre> + +<p>There must NOT be empty lines between annotations and the function +declaration. Examples:</p> +<pre class="prettyprint"> +// Good +@entry +foo(); + +// Bad +@entry + +foo(); +</pre> + +<h3 id=enum-declarations>Enum declarations</h3> +<p>Use the following rules for enum declarations:</p> +<ul> +<li>If enum declarations are shared with another package, put the declarations +in <code>types.hal</code> rather than embedding inside an interface.</li> +<li>Use a space before and after the colon, and space after the underlying type +before the open brace.</li> +<li>The last enum value may or may not have an extra comma.</li> +</ul> + +<h3 id=struct-declarations>Struct declarations</h3> +<p>Use the following rules for struct declarations:</p> +<ul> +<li>If struct declarations are shared with another package, put the declarations +in <code>types.hal</code> rather than embedding inside an interface.</li> +<li>Use a space after the struct type name before the open brace.</li> +<li>Align field names (optional). Example: +<pre class="prettyprint"> +struct MyStruct { + vec<uint8_t> data; + int32_t someInt; +} +</pre> +</li> +</ul> + +<h3 id=array-declarations>Array declarations</h3> +<p>Do not put spaces between the following:</p> +<ul> +<li>Element type and open square bracket.</li> +<li>Open square bracket and array size.</li> +<li>Array size and close square bracket.</li> +<li>Close square bracket and the next open square bracket, if more than one +dimension exists.</li> +</ul> + +<p>Examples:</p> +<pre class="prettyprint"> +// Good +int32_t[5] array; + +// Good +int32_t[5][6] multiDimArray; + +// Bad +int32_t [ 5 ] [ 6 ] array; +</pre> + +<h3 id=vectors>Vectors</h3> +<p>Do not put spaces between the following:</p> +<ul> +<li><code>vec</code> and open angle bracket.</li> +<li>Open angle bracket and element type (<em>Exception: element type is also a +<code>vec</code></em>).</li> +<li>Element type and close angle bracket (<em>Exception: element type is also a +<code>vec</code>)</em>.</li> +</ul> + +<p>Examples:</p> +<pre class="prettyprint"> +// Good +vec<int32_t> array; + +// Good +vec<vec<int32_t>> array; + +// Good +vec< vec<int32_t> > array; + +// Bad +vec < int32_t > array; + +// Bad +vec < vec < int32_t > > array; +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl/converting.html b/en/devices/architecture/hidl/converting.html new file mode 100644 index 00000000..d4dbacea --- /dev/null +++ b/en/devices/architecture/hidl/converting.html @@ -0,0 +1,87 @@ +<html devsite> + <head> + <title>Converting HAL Modules</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>You can update pre-existing HAL modules to HIDL HAL modules by converting the +header in <code>hardware/libhardware/include/hardware</code>.</p> + +<h2 id=c2hal>Using c2hal</h2> +<p>The +<code><a href="https://android.googlesource.com/platform/system/tools/hidl/+/master/c2hal/">c2hal</code></a> +tool handles most of the conversion work, reducing the number of required manual +changes. For example, to generate a HIDL <code>.hal</code> file for the NFC +HAL:</p> +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal">make c2hal</code> +c2hal -r android.hardware:hardware/interfaces -randroid.hidl:system/libhidl/transport -p android.hardware.nfc@1.0 hardware/libhardware/include/hardware/nfc.h +</pre> + +<p>These commands add files in <code>hardware/interfaces/nfc/1.0/</code>. Running +<code>hardware/interfaces/update-makefiles.sh</code> from the $ANDROID_BUILD_TOP +directory will also add the required makefile to the HAL. From here, you can +make manual changes to fully convert the HAL.</p> + +<h2 id=activities>c2hal activities</h2> +<p>When you run <code>c2hal</code>, everything in the header file is transferred +to <code>.hal</code> files.</p> + +<p><code>c2hal</code> identifies structs that contain function pointers in the +provided header file and converts each struct into a separate interface file. +For example, <code>alloc_device_t</code> is converted to the +<code>IAllocDevice</code> HAL module (in the file +<code>IAllocDevice.hal</code>).</p> + +<p>All other data types are copied over into a <code>types.hal</code> file. +Pound-defines are moved into enums, and items not a part of HIDL or not +convertible (such as static-function declarations) are copied into comments +marked with the text "<code>NOTE</code>".</p> + +<h2 id=manual>Manual activities</h2> +<p>The <code>c2hal</code> tool does not know what to do when it encounters +certain constructs. For example, HIDL has no concept of raw pointers; because of +this, when <code>c2hal</code> encounters a pointer in header files, it doesn't +know whether the pointer should be interpreted as an array or as a reference to +another object. Void pointers are also similarly opaque.</p> + +<p>Field such as <code>int reserved[7]</code> must be manually removed during +the transition to HIDL. Items such as the name of the return value should be +updated to something more meaningful; for example, converting the return +parameter of methods such as <code>write</code> in NFC from the autogenerated +<code>int32_t write_ret</code> to <code>Status status</code> (where +<code>Status</code> is a new enum containing possible NFC statuses).</p> + +<h2 id=implement>Implementing the HAL</h2> +<p>After you have created <code>.hal</code> files to represent your HAL, you +must generate the makefiles (Make or Soong) that create the language support in +C++ and Java (unless the HAL uses a feature unsupported in Java). The +<code>./hardware/interfaces/update-makefiles.sh</code> script can automatically +generate makefiles for HALs located in the <code>hardware/interfaces</code> +directory (for HALs in other locations, simply update the script).</p> + +<p>When the makefiles are up to date, you are ready to generate header files and +implement methods. For details on implementing the generated interface, see +<a href="/devices/architecture/hidl-cpp/index.html">HIDL C++</a> (for C++ +implementations) or <a href="/devices/architecture/hidl-java/index.html">HIDL +Java</a> (for Java implementations).</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/fmq.html b/en/devices/architecture/hidl/fmq.html new file mode 100644 index 00000000..c09fd1cc --- /dev/null +++ b/en/devices/architecture/hidl/fmq.html @@ -0,0 +1,424 @@ +<html devsite> + <head> + <title>Fast Message Queue (FMQ)</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>HIDL's remote procedure call (RPC) infrastructure uses Binder mechanisms, +meaning calls involve overhead, require kernel operations, and may trigger +scheduler action. However, for cases where data must be transferred between +processes with less overhead and no kernel involvement, the Fast Message Queue +(FMQ) system is used.</p> + +<p>FMQ creates message queues with the desired properties. An +<code>MQDescriptorSync</code> or <code>MQDescriptorUnsync</code> object can be +sent over a HIDL RPC call and used by the receiving process to access the +message queue.</p> + +<p class="caution">Fast Message Queues are supported only in C++.</p> + +<h2 id=flavors>MessageQueue types</h2> +<p>Android supports two queue types:</p> +<ul> +<li><em>Unsynchronized</em> queues are allowed to overflow, and can have many +readers; each reader must read data in time or lose it. +<li><em>Synchronized</em> queues are not allowed to overflow, and can have only +one reader.</li> +</ul> + +<p>Both queue types are not allowed to underflow (read from an empty queue will +fail) and can only have one writer.</p> + +<h3 id=unsync>Unsynchronized</h3> +<p>An unsynchronized queue has only one writer, but can have any number of +readers. There is one write position for the queue; however, each reader keeps +track of its own independent read position.</p> + +<p>Writes to the queue always succeed (are not checked for overflow) as long as +they are no larger than the configured queue capacity (writes larger than the +queue capacity fail immediately). As each reader may have a different read +position, rather than waiting for every reader to read every piece of data, data +is allowed to fall off the queue whenever new writes need the space.</p> + +<p>Reads are responsible for retrieving data before it falls off the end of +the queue. A read that attempts to read more data than is available either +fails immediately (if nonblocking) or waits for enough data to be available (if +blocking). A read that attempts to read more data than the queue capacity always +fails immediately.</p> + +<p>If a reader fails to keep up with the writer, so that the amount of data +written and not yet read by that reader is larger than the queue capacity, the +next read does not return data; instead, it resets the readers read +position to equal the latest write position then returns failure. If the +data available to read is checked after overflow but before the next read, it +shows more data available to read than the queue capacity, indicating +overflow has occurred. (If the queue overflows between checking available data +and attempting to read that data, the only indication of overflow is that the +read fails.)</p> + +<h3 id=sync>Synchronized</h3> +<p>A synchronized queue has one writer and one reader with a single write +position and a single read position. It is impossible to write more data than +the queue has space for or read more data than the queue currently holds. +Depending on whether the blocking or nonblocking write or read function is +called, attempts to exceed available space or data either return failure +immediately or block until the desired operation can be completed. Attempts to +read or write more data than the queue capacity will always fail immediately. +</p> + +<h2 id=setup>Setting up an FMQ</h2> +<p>A message queue requires multiple <code>MessageQueue</code> objects: one to +be written to, and one or more to be read from. There is no explicit +configuration of which object is used for writing or reading; it is up to the +user to ensure that no object is used for both reading and writing, that there +is at most one writer, and, for synchronized queues, that there is at most one +reader.</p> + +<h3 id=first-object>Creating the first MessageQueue object</h3> +<p>A message queue is created and configured with a single call:</p> + +<pre class="prettyprint"> +#include <fmq/MessageQueue.h> +using android::hardware::kSynchronizedReadWrite; +using android::hardware::kUnsynchronizedWrite; +using android::hardware::MQDescriptorSync; +using android::hardware::MQDescriptorUnsync; +using android::hardware::MessageQueue; +.... +// For a synchronized non-blocking FMQ +mFmqSynchronized = + new (std::nothrow) MessageQueue<uint16_t, kSynchronizedReadWrite> + (kNumElementsInQueue); +// For an unsynchronized FMQ that supports blocking +mFmqUnsynchronizedBlocking = + new (std::nothrow) MessageQueue<uint16_t, kUnsynchronizedWrite> + (kNumElementsInQueue, true /* enable blocking operations */); +</pre> + +<ul> +<li>The <code>MessageQueue<T, flavor>(numElements)</code> initializer +creates and initializes an object that supports the message queue functionality. +</li> +<li>The <code>MessageQueue<T, flavor>(numElements, +configureEventFlagWord)</code> initializer creates and initializes an object +that supports the message queue functionality with blocking.</li> +<li><code>flavor</code> can be either <code>kSynchronizedReadWrite</code> for a +synchronized queue or <code>kUnsynchronizedWrite</code> for an unsynchronized +queue.</li> +<li><code>uint16_t</code> (in this example) can be any +<a href="/devices/architecture/hidl-cpp/types.html">HIDL-defined type</a> that +does not involve nested buffers (no <code>string</code> or <code>vec</code> +types), handles, or interfaces.</li> +<li><code>kNumElementsInQueue</code> indicates the size of queue in number of +entries; it determines the size of shared memory buffer that will be allocated +for the queue.</li> +</ul> + +<h3 id=second-object>Creating the second MessageQueue object</h3> +<p>The second side of the message queue is created using an +<code>MQDescriptor</code> object obtained from the first side. The +<code>MQDescriptor</code> object is sent over a HIDL RPC call to the process +that will hold the second end of the message queue. The +<code>MQDescriptor</code> contains information about the queue:</p> + +<ul> +<li>Information to map the buffer and write pointer.</li> +<li>If the queue is synchronized, information to map the read pointer.</li> +<li>If the queue is blocking, information to map the event flag word.</li> +<li>The object type is made into a template with the HIDL-defined type of queue +elements and the flavor (synchronized or unsynchronized).</li> +</ul> + +<p>The <code>MQDescriptor</code> object can be used to construct a +<code>MessageQueue</code> object:</p> + +<pre class="prettyprint"> +MessageQueue<T, flavor>::MessageQueue(const MQDescriptor<T, flavor>& Desc, bool resetPointers) +</pre> + +<p>The <code>resetPointers</code> parameter indicates whether to reset the read +and write positions to 0 while creating this <code>MessageQueue</code> object. +In an unsynchronized queue, the read position (which is local to each +<code>MessageQueue</code> object in unsynchronized queues) is always set to 0 +during creation. Typically, the <code>MQDescriptor</code> is initialized during +creation of the first message queue object. For extra control over the shared +memory, you can set up the <code>MQDescriptor</code> manually +(<code>MQDescriptor</code> is defined in +<a href="https://android.googlesource.com/platform/system/libhidl/+/master/base/include/hidl/MQDescriptor.h" class="external"><code>system/libhidl/base/include/hidl/MQDescriptor.h</code></a>) +then create every <code>MessageQueue</code> object as described in this section. +</p> + +<h3 id=blocking>Blocking queues and event flags</h3> +<p>By default, queues do not support blocking reads/writes. There are two kinds +of blocking read/write calls:</p> +<ul> +<li><em>Short form</em>, with three parameters (data pointer, number of items, +timeout). Supports blocking on individual read/write operations on a single +queue. When using this form, the queue will handle the event flag and bitmasks +internally, and the first message queue object must be initialized with a +second parameter of <code>true</code>.</li> +<li><em>Long form</em>, with six parameters (includes event flag and bitmasks). +Supports using a shared <code>EventFlag</code> object between multiple queues +and allows specifying the notification bit masks to be used. In this case, the +event flag and bitmasks must be supplied to each read and write call.</li> +</ul> + +<p>For the long form, the <code>EventFlag</code> can be supplied explicitly in +each <code>blockingRead()</code> and <code>blockingWrite()</code> call. One of +the queues may be initialized with an internal event flag, which must then be +extracted from that queue's <code>MessageQueue</code> objects using +<code>getEventFlagWord()</code> and used to create <code>EventFlag</code> +objects in each process for use with other FMQs. Alternatively, the +<code>EventFlag</code> objects can be initialized with any suitable shared +memory.</p> + +<p>In general, each queue should use only one of non-blocking, short-form +blocking, or long-form blocking. It is not an error to mix them, but careful +programming is required to get the desired result.</p> + +<h2 id=using>Using the MessageQueue</h2> +<p>The public API of the <code>MessageQueue</code> object is:</p> + +<pre class="prettyprint"> +size_t availableToWrite() // Space available (number of elements). +size_t availableToRead() // Number of elements available. +size_t getQuantumSize() // Size of type T in bytes. +size_t getQuantumCount() // Number of items of type T that fit in the FMQ. +bool isValid() // Whether the FMQ is configured correctly. +const MQDescriptor<T, flavor>* getDesc() // Return info to send to other process. + +bool write(const T* data) // Write one T to FMQ; true if successful. +bool write(const T* data, size_t count) // Write count T's; no partial writes. + +bool read(T* data); // read one T from FMQ; true if successful. +bool read(T* data, size_t count); // Read count T's; no partial reads. + +bool writeBlocking(const T* data, size_t count, int64_t timeOutNanos = 0); +bool readBlocking(T* data, size_t count, int64_t timeOutNanos = 0); + +// Allows multiple queues to share a single event flag word +std::atomic<uint32_t>* getEventFlagWord(); + +bool writeBlocking(const T* data, size_t count, uint32_t readNotification, +uint32_t writeNotification, int64_t timeOutNanos = 0, +android::hardware::EventFlag* evFlag = nullptr); // Blocking write operation for count Ts. + +bool readBlocking(T* data, size_t count, uint32_t readNotification, +uint32_t writeNotification, int64_t timeOutNanos = 0, +android::hardware::EventFlag* evFlag = nullptr) // Blocking read operation for count Ts; + +//APIs to allow zero copy read/write operations +bool beginWrite(size_t nMessages, MemTransaction* memTx) const; +bool commitWrite(size_t nMessages); +bool beginRead(size_t nMessages, MemTransaction* memTx) const; +bool commitRead(size_t nMessages); +</pre> + +<p><code>availableToWrite()</code> and <code>availableToRead()</code> can be used +to determine how much data can be transferred in a single operation. In an +unsynchronized queue:</p> + +<ul> +<li><code>availableToWrite()</code> always returns the capacity of the queue. +<li>Each reader has its own read position and does its own calculation for +<code>availableToRead()</code>.</li> +<li>From the point of view of a slow reader, the queue is allowed to overflow; +this may result in <code>availableToRead()</code> returning a value larger than +the size of the queue. The first read after an overflow will fail and result in +the read position for that reader being set equal to the current write pointer, +whether or not the overflow was reported through +<code>availableToRead()</code>.</li> +</ul> + +<p>The <code>read()</code> and <code>write()</code> methods return +<code>true</code> if all requested data could be (and was) transferred to/from +the queue. These methods do not block; they either succeed (and return +<code>true</code>), or return failure (<code>false</code>) immediately.</p> + +<p>The <code>readBlocking()</code> and <code>writeBlocking()</code> methods wait +until the requested operation can be completed, or until they timeout (a +<code>timeOutNanos</code> value of 0 means never timeout).</p> + +<p>Blocking operations are implemented using an event flag word. By default, +each queue creates and uses its own flag word to support the short form of +<code>readBlocking()</code> and <code>writeBlocking()</code>. It is possible for +multiple queues to share a single word, so that a process can wait on writes or +reads to any of the queues. A pointer to a queue's event flag word can be +obtained by calling <code>getEventFlagWord()</code>, and that pointer (or any +pointer to a suitable shared memory location) can be used to create an +<code>EventFlag</code> object to pass into the long form of +<code>readBlocking()</code> and <code>writeBlocking()</code>for a different +queue. The <code>readNotification</code> and <code>writeNotification</code> +parameters tell which bits in the event flag should be used to signal reads and +writes on that queue. <code>readNotification</code> and +<code>writeNotification</code> are 32-bit bitmasks.</p> + +<p><code>readBlocking()</code> waits on the <code>writeNotification</code> bits; +if that parameter is 0, the call always fails. If the +<code>readNotification</code> value is 0, the call will not fail, but a +successful read will not set any notification bits. In a synchronized queue, +this would mean that the corresponding <code>writeBlocking()</code> call will +never wake up unless the bit is set elsewhere. In an unsynchronized queue, +<code>writeBlocking()</code> will not wait (it should still be used to set the +write notification bit), and it is appropriate for reads to not set any +notification bits. Similarly, <code>writeblocking()</code> will fail if +<code>readNotification</code> is 0, and a successful write sets the specified +<code>writeNotification</code> bits.</p> + +<p>To wait on multiple queues at once, use an <code>EventFlag</code> object's +<code>wait()</code> method to wait on a bitmask of notifications. The +<code>wait()</code> method returns a status word with the bits that caused the +wake up set. Using the information, the user can then check the corresponding +queue to see whether it has enough space or data for the desired write or read +operation and perform a nonblocking <code>read()</code>/<code>write()</code> +followed by a call to the <code>EventFlag</code>'s <code>wake()</code> method if +a notification is desired after the same. For a definition of the +<code>EventFlag</code> abstraction, refer to +<a href="https://android.googlesource.com/platform/system/libfmq/+/master/include/fmq/EventFlag.h" class="external"><code>system/libfmq/include/fmq/EventFlag.h</code></a>. +</p> + +<h2 id=zero>Zero copy operations</h2> +<p>The +<code>read</code>/<code>write</code>/<code>readBlocking</code>/<code>writeBlocking()</code> +APIs take a pointer to an input/output buffer as an argument and use +<code>memcpy()</code> calls internally to copy data between the same and the +FMQ ring buffer. To improve performance, Android O includes a set of APIs that +provide direct pointer access into the ring buffer, eliminating the need to use +<code>memcpy</code> calls.</p> + +<p>Use the following public APIs for zero copy FMQ operations:</p> + +<pre class="prettyprint"> +bool beginWrite(size_t nMessages, MemTransaction* memTx) const; +bool commitWrite(size_t nMessages); + +bool beginRead(size_t nMessages, MemTransaction* memTx) const; +bool commitRead(size_t nMessages); +</pre> + +<ul> +<li>The <code>beginWrite</code> method provides base pointers into the FMQ ring +buffer. After the data is written, commit it using <code>commitWrite()</code>. +The<code>beginRead</code>/<code>commitRead</code> methods act the same way.</li> +<li>The <code>beginRead</code>/<code>Write</code> methods take as input the +number of messages to be read/written and return a boolean indicating if the +read/write is possible. If the read or write is possible the <code>memTx</code> +struct is populated with base pointers that can be used for direct pointer +access into the ring buffer shared memory.</li> +<li>The <code>MemRegion</code> struct contains details about a block of memory, +i.e. a base pointer and length in terms of <code>T</code>(where the FMQ is +templatized to <code>T</code>).</li> +<li>The <code>MemTransaction</code> struct contains two <code>MemRegion</code> +structs, <code>first</code> and <code>second</code> as a read or write into +the ring buffer may require a wrap around to the beginning of the queue. This +would mean that two base pointers are needed to read/write data into the FMQ +ring buffer.</li> +</ul> + +<p>To get the base address and length from a <code>MemRegion</code> struct:</p> + +<pre class="prettyprint"> +T* getAddress(); // gets the base address +size_t getLength(); // gets the length of the memory region in terms of T +size_t getLengthInBytes(); // gets the length of the memory region in bytes +</pre> + +<p>To get references to the first and second <code>MemRegion</code>s within a +<code>MemTransaction</code> object:</p> + +<pre class="prettyprint"> +const MemRegion& getFirstRegion(); // get a reference to the first MemRegion +const MemRegion& getSecondRegion(); // get a reference to the second MemRegion +</pre> + +<p>Example write to the FMQ using zero copy APIs:</p> + +<pre class="prettyprint"> +MessageQueueSync::MemTransaction tx; +if (mQueue->beginRead(dataLen, &tx)) { +auto first = tx.getFirstRegion(); +auto second = tx.getSecondRegion(); + +foo(first.getAddress(), first.getLength()); // method that performs the data write +foo(second.getAddress(), second.getLength()); // method that performs the data write + +if(commitWrite(dataLen) == false) { +//report error +} +} else { +//report error +} +</pre> + +<p>The following helper methods are also part of <code>MemTransaction</code>: +</p> + +<ul> +<li><code>T* getSlot(size_t idx);</code> +<br>Returns a pointer to slot <code>idx</code> within the +<code>MemRegions</code> that are part of this <code>MemTransaction</code> +object. If the <code>MemTransaction</code> object is representing the memory +regions to read/write N items of type T, then the valid range of +<code>idx</code> is between 0 and N-1.</li> +<li><code>bool copyTo(const T* data, size_t startIdx, size_t nMessages = 1);</code> +<br>Write <code>nMessages</code>' items of type T into the memory regions +described by the object, starting from index <code>startIdx</code>. This method +uses <code>memcpy()</code> and is not to meant to be used for a zero copy +operation. If the <code>MemTransaction</code> object represents memory to +read/write N items of type T, then the valid range of <code>idx</code> is +between 0 and N-1.</li> +<li><code>bool copyFrom(T* data, size_t startIdx, size_t nMessages = 1);</code> +<br>Helper method to read <code>nMessages</code>' items of type T from the +memory regions described by the object starting from <code>startIdx</code>. This +method uses <code>memcpy()</code> and is not meant to be used for a zero copy +operation.</li> +</ul> + +<h2 id=sending>Sending the queue over HIDL</h2> +<p>On the creating side:</p> +<ol> +<li>Create message queue object as described above.</li> +<li>Verify the object is valid with <code>isValid()</code>.</li> +<li>If you will be waiting on multiple queues by passing an +<code>EventFlag</code> into the long form of +<code>readBlocking()</code>/<code>writeBlocking()</code>, you can extract the +event flag pointer (using <code>getEventFlagWord()</code>) from a +<code>MessageQueue</code> object that was initialized to create the flag, and +use that flag to create the necessary <code>EventFlag</code> object.</li> +<li>Use the <code>MessageQueue</code> <code>getDesc()</code> method to get a +descriptor object.</li> +<li>In the <code>.hal</code> file, give a method a parameter of type +<code>fmq_sync<T></code> or <code>fmq_unsync<T></code> where <code>T</code> is a +suitable HIDL-defined type. Use this to send the object returned by +<code>getDesc()</code> to the receiving process.</li> +</ol> + +<p>On the receiving side:</p> +<ol> +<li>Use the descriptor object to create a <code>MessageQueue</code> object. Be +sure to use the same queue flavor and data type, or the template will fail to +compile.</li> +<li>If you extracted an event flag, extract the flag from the corresponding +<code>MessageQueue</code> object in the receiving process.</li> +<li>Use the <code>MessageQueue</code> object to transfer data.</li> +</ol> + + </body> +</html> diff --git a/en/devices/architecture/hidl/hashing.html b/en/devices/architecture/hidl/hashing.html new file mode 100644 index 00000000..e083fd4c --- /dev/null +++ b/en/devices/architecture/hidl/hashing.html @@ -0,0 +1,165 @@ +<html devsite> + <head> + <title>Interface Hashing</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This document describes HIDL interface hashing, a mechanism to prevent +accidental interface changes and ensure interface changes are thoroughly vetted. +This mechanism is required because HIDL interfaces are versioned, which means +that after an interface is released it must not be changed except in an +Application Binary Interface (ABI) preserving manner (such as a comment +correction).</p> + +<h2 id=layout>Layout</h2> +<p>Every package root directory (i.e. <code>android.hardware</code> mapping to +<code>hardware/interfaces</code> or <code>vendor.foo</code> mapping to +<code>vendor/foo/hardware/interfaces</code>) must contain a +<code>current.txt</code> file that lists all released HIDL interface files.</p> + +<pre class="prettyprint"> +# current.txt files support comments starting with a ‘#' character +# this file, for instance, would be vendor/foo/hardware/interfaces/current.txt + +# Each line has a SHA-256 hash followed by the name of an interface. +# They have been shortened in this doc for brevity but they are +# 64 characters in length in an actual current.txt file. +d4ed2f0e...995f9ec4 vendor.awesome.foo@1.0::IFoo # comments can also go here + +# types.hal files are also noted in types.hal files +c84da9f5...f8ea2648 vendor.awesome.foo@1.0::types + +# Multiple hashes can be in the file for the same interface. This can be used +# to note how ABI sustaining changes were made to the interface. +# For instance, here is another hash for IFoo: + +# Fixes type where "FooCallback" was misspelled in comment on "FooStruct" +822998d7...74d63b8c vendor.awesome.foo@1.0::IFoo +</pre> + +<p class=note><strong>Note:</strong> To help keep track of which hashes come +from where, Google separates HIDL <code>current.txt</code> files into different +sections: The first section is <em>Released in Android O</em>; the next section +will be <em>Released in Android O MR1</em>. We strongly recommend using a +similar layout in your <code>current.txt</code> file.</p> + +<h2 id=hidl-gen>Hashing with hidl-gen</h2> +<p>You can add a hash to a <code>current.txt</code> file manually or by +using <code>hidl-gen</code>. The following code snippet provides examples of +commands you can use with <code>hidl-gen</code> to manage a +<code>current.txt</code> file (hashes have been shortened):</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal">hidl-gen -L hash -r vendor.awesome:vendor/awesome/hardware/interfaces -r android.hardware:hardware/interfaces -r android.hidl:system/libhidl/transport vendor.awesome.nfc@1.0::types</code> +9626fd18...f9d298a6 vendor.awesome.nfc@1.0::types +<code class="devsite-terminal">hidl-gen -L hash -r vendor.awesome:vendor/awesome/hardware/interfaces -r android.hardware:hardware/interfaces -r android.hidl:system/libhidl/transport vendor.awesome.nfc@1.0::INfc</code> +07ac2dc9...11e3cf57 vendor.awesome.nfc@1.0::INfc +<code class="devsite-terminal">hidl-gen -L hash -r vendor.awesome:vendor/awesome/hardware/interfaces -r android.hardware:hardware/interfaces -r android.hidl:system/libhidl/transport vendor.awesome.nfc@1.0</code> +9626fd18...f9d298a6 vendor.awesome.nfc@1.0::types +07ac2dc9...11e3cf57 vendor.awesome.nfc@1.0::INfc +f2fe5442...72655de6 vendor.awesome.nfc@1.0::INfcClientCallback +<code class="devsite-terminal">hidl-gen -L hash -r vendor.awesome:vendor/awesome/hardware/interfaces -r android.hardware:hardware/interfaces -r android.hidl:system/libhidl/transport vendor.awesome.nfc@1.0 >> vendor/awesome/hardware/interfaces/current.txt</code> +</pre> + +<p class="warning"><strong>Warning:</strong> Do not replace a hash for a +previously-released interface. When changing such an interface, add a new hash +to the end of the <code>current.txt</code> file. For details, refer to +<a href="#abi-stability">ABI stability</a>.</p> + +<p>Every interface definition library generated by <code>hidl-gen</code> +includes hashes, which can be retrieved by calling +<code>IBase::getHashChain</code>. When <code>hidl-gen</code> is compiling an +interface, it checks the <code>current.txt</code> file in the root directory of +the HAL package to see if the HAL has been changed:</p> + +<ul> +<li>If no hash for the HAL is found, the interface is considered unreleased (in +development) and compilation proceeds.</li> +<li>If hashes are found, they are checked against the current interface: +<ul> +<li>If the interface does match the hash, compilation proceeds.</li> +<li>If the interface does not match a hash, compilation is halted as this means a previously-released interface is being changed. +<ul> +<li>For an ABI-preserving change (see +<a href="#abi-stability">ABI stability</a>), the <code>current.txt</code> file +must be modified before compilation can proceed.</li> +<li>All other changes should be made in a minor or major version upgrade of the +interface.</li> +</ul></li></ul></li></ul> + +<h2 id=abi-stability>ABI stability</h2> +<aside class="key-point"><strong>Key Point:</strong> Please read and understand +this section carefully.</aside> + +<p>An Application Binary Interface (ABI) includes the binary +linkages/calling conventions/etc. If the ABI/API changes, the interface no +longer works with a generic <code>system.img</code> that was compiled with +official interfaces.</p> + +<p>Making sure that interfaces are versioned and ABI stable is +<strong>crucial</strong> for several reasons:</p> + +<ul> +<li>It ensures your implementation can pass the Vendor Test Suite (VTS), which +puts you on track to being able to do framework-only OTAs.</li> +<li>As an OEM, it enables you to provide a Board Support Package (BSP) that is +straightforward to use and compliant.</li> +<li>It helps you keep track of what interfaces can be released. Consider +<code>current.txt</code> a map of an interfaces directory that allows you to see +the history and state of all interfaces being provided in a package root.</li> +</ul> + +<p>When adding a new hash for an interface that already has an entry in +<code>current.txt</code>, make sure to add only the hashes that represent +interfaces which maintain ABI stability. Review the following types of changes: +</p> + +<table> +<tbody> + +<tr> +<th>Changes allowed</th> +<td> +<ul> +<li>Changing a comment (unless this changes the meaning of a method).</li> +<li>Changing the name of a parameter.</li> +<li>Changing the name of a return parameter.</li> +<li>Changing annotations.</li> +</ul> +</td> +</tr> + +<tr> +<th>Changes not allowed</th> +<td> +<ul> +<li>Reordering arguments, methods, etc…</li> +<li>Adding a method/struct field/etc… anywhere in the interface.</li> +<li>Anything that would break a C++ vtable.</li> +<li>etc..</li> +</ul> +</td> +</tr> + +</tbody> +</table> + + </body> +</html> diff --git a/en/devices/architecture/hidl/index.html b/en/devices/architecture/hidl/index.html new file mode 100644 index 00000000..20f4a607 --- /dev/null +++ b/en/devices/architecture/hidl/index.html @@ -0,0 +1,337 @@ +<html devsite> + <head> + <title>HIDL</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>HAL interface definition language or HIDL (pronounced "hide-l") is an +interface description language (IDL) to specify the interface between a HAL and +its users. It allows specifying types and method calls, collected into +interfaces and packages. More broadly, HIDL is a system for communicating +between codebases that may be compiled independently.</p> + +<p>HIDL is intended to be used for inter-process communication (IPC). +Communication between processes is referred to as +<a href="/devices/architecture/hidl/binder-ipc"><em>Binderized</em></a>. For +libraries that must be linked to a process, a <a href="#passthrough">passthough +mode</a> is also available (not supported in Java).</p> + +<p>HIDL specifies data structures and method signatures, organized in interfaces +(similar to a class) that are collected into packages. The syntax of HIDL will +look familiar to C++ and Java programmers, though with a different set of +keywords. HIDL also uses Java-style annotations.</p> + +<h2 id=design>HIDL design</h2> +<p>The goal of HIDL is that the framework can be replaced without having to +rebuild HALs. HALs will be built by vendors or SOC makers and put in a +<code>/vendor</code> partition on the device, enabling the framework, in its own +partition, to be replaced with an OTA without recompiling the HALs.</p> + +<p>HIDL design balances the following concerns:</p> +<ul> +<li><strong>Interoperability</strong>. Create reliably interoperable interfaces +between processes which may be compiled with various architectures, toolchains, +and build configurations. HIDL interfaces are versioned and cannot be changed +after they are published.</li> +<li><strong>Efficiency</strong>. HIDL tries to minimize the number of copy +operations. HIDL-defined data is delivered to C++ code in C++ standard layout +data structures that can be used without unpacking. HIDL also provides shared +memory interfaces and, as RPCs are inherently somewhat slow, HIDL supports two +ways to transfer data without using an RPC call: shared memory and a Fast +Message Queue (FMQ).</li> +<li><strong>Intuitive</strong>. HIDL avoids thorny issues of memory ownership by +using only <code>in</code> parameters for RPC (see +<a href="https://developer.android.com/guide/components/aidl.html" class="external">Android +Interface Definition Language (AIDL)</a>); values that cannot be efficiently +returned from methods are returned via callback functions. Neither passing data +into HIDL for transfer nor receiving data from HIDL changes the ownership of the +data—ownership always remains with the calling function. Data needs to +persist only for the duration of the called function and may be destroyed +immediately after the called function returns.</li> +</ul> + +<h2 id=passthrough>Using passthrough mode</h2> +<p>To update devices running earlier versions of Android to Android O, you can +wrap both conventional (and legacy) HALs in a new HIDL interface that serves the +HAL in binderized and same-process (passthrough) modes. This wrapping is +transparent to both the HAL and the Android framework.</p> + +<p>Passthrough mode is available only for C++ clients and implementations. +Devices running earlier versions of Android do not have HALs written in Java, so +Java HALs are inherently binderized.</p> + +<h3 id=header>Passthrough header files</h3> +<p>When a <code>.hal</code> file is compiled, <code>hidl-gen</code> produces an +extra passthrough header file <code>BsFoo.h</code> in addition to the headers +used for binder communication; this header defines functions to be +<code>dlopen</code>ed. As passthrough HALs run in the same process in which +they are called, in most cases passthrough methods are invoked by direct +function call (same thread). <code>oneway</code> methods run in their own thread +as they are not intended to wait for the HAL to process them (this means any HAL +that uses <code>oneway</code> methods in passthrough mode must be thread-safe). +</p> + +<p>Given an <code>IFoo.hal</code>, <code>BsFoo.h</code> wraps the HIDL-generated +methods to provide additional features (such as making <code>oneway</code> +transactions run in another thread). This file is similar to +<code>BpFoo.h</code>, however instead of passing on calls IPC using binder, the +desired functions are directly invoked. Future implementations of HALs +<strong>may provide</strong> multiple implementations, such as FooFast HAL and a +FooAccurate HAL. In such cases, a file for each additional implementation would +be created (e.g., <code>PTFooFast.cpp</code> and +<code>PTFooAccurate.cpp</code>).</p> + +<h3 id=binderized>Binderizing passthrough HALs</h3> +<p>You can binderize HAL implementations that support passthrough mode. Given a +HAL interface <code>a.b.c.d@M.N::IFoo</code>, two packages are created:</p> + +<ul> +<li><code>a.b.c.d@M.N::IFoo-impl</code>. Contains the implementation of the HAL +and exposes function <code>IFoo* HIDL_FETCH_IFoo(const char* name)</code>. On +legacy devices, this package is <code>dlopen</code>ed and the implementation is +instantiated using <code>HIDL_FETCH_IFoo</code>. You can generate the base code +using <code>hidl-gen</code> and <code>-Lc++-impl</code> and +<code>-Landroidbp-impl</code>.</li> +<li><code>a.b.c.d@M.N::IFoo-service</code>. Opens the passthrough HAL and +registers itself as a binderized service, enabling the same HAL implementation +to be used as both passthrough and binderized.</li> +</ul> + +<p>Given the type <code>IFoo</code>, you can call <code>sp<IFoo> +IFoo::getService(string name, bool getStub)</code> to get access to an instance +of <code>IFoo</code>. If <code>getStub</code> is true, <code>getService</code> +attempts to open the HAL only in passthrough mode. If <code>getStub</code> is +false, <code>getService</code> attempts to find a binderized service; if that +fails, it then tries to find the passthrough service. The <code>getStub</code> +parameter should never be used except in +<code>defaultPassthroughServiceImplementation</code>. (Devices launching with +Android O are fully binderized devices, so opening a service in passthrough mode +is disallowed.)</p> + +<h2 id=grammar>HIDL grammar</h2> +<p>By design, the HIDL language is similar to C (but does not use the C +preprocessor). All punctuation not described below (aside from the obvious use +of <code>=</code> and <code>|</code>) is part of the grammar.</p> + +<p class=note><strong>Note:</strong> For details on HIDL code style, see the +<a href="code-style.html">Code Style Guide</a>.</p> + +<ul> +<li><code>/** */</code> indicates a documentation comment.</li> +<li><code>/* */</code> indicates a multiline comment.</li> +<li><code>//</code> indicates a comment to end of line. Aside from +<code>//</code>, newlines are the same as any other whitespace. </li> +<li>In the example grammar below, text from <code>//</code> to the end of the +line is not part of the grammar but is instead a comment on the grammar.</li> +<li><code>[empty]</code> means that the term may be empty. </li> +<li><code>?</code> following a literal or term means it is optional.</li> +<li><code>...</code> indicates sequence containing zero or more items with +separating punctuation as indicated. There are no variadic arguments in HIDL.</li> +<li>Commas separate sequence elements.</li> +<li>Semicolons terminate each element, including the last element.</li> +<li>UPPERCASE is a nonterminal.</li> +<li><code><em>italics</code></em> is a token family such as +<code><em>integer</code></em> or <code><em>identifier</code></em> (standard C +parsing rules).</li> +<li><code><em>constexpr</em> </code>is a C style constant expression (such as +<code>1 + 1</code> and <code>1L << 3</code>).</li> +<li><code><em>import_name</code></em> is a package or interface name, qualified +as described in <a href="/devices/architecture/hidl/versioning.html"> HIDL +Versioning</a>.</li> +<li>Lowercase <code>words</code> are literal tokens.</li> +</ul> +<p>Example:</p> +<pre class="prettyprint"> +ROOT = + PACKAGE IMPORTS PREAMBLE { ITEM ITEM ... } // not for types.hal + PREAMBLE = interface identifier EXTENDS + | PACKAGE IMPORTS ITEM ITEM... // only for types.hal; no method definitions + +ITEM = + ANNOTATIONS? oneway? identifier(FIELD, FIELD ...) GENERATES?; + | struct identifier { SFIELD; SFIELD; ...}; // Note - no forward declarations + | union identifier { UFIELD; UFIELD; ...}; + | enum identifier: TYPE { ENUM_ENTRY, ENUM_ENTRY ... }; // TYPE = enum or scalar + | typedef TYPE identifier; + +VERSION = integer.integer; + +PACKAGE = package android.hardware.identifier[.identifier[...]]@VERSION; + +PREAMBLE = interface identifier EXTENDS + +EXTENDS = <empty> | extends import_name // must be interface, not package + +GENERATES = generates (FIELD, FIELD ...) + +// allows the Binder interface to be used as a type +// (similar to typedef'ing the final identifier) +IMPORTS = + [empty] + | IMPORTS import import_name; + +TYPE = + uint8_t | int8_t | uint16_t | int16_t | uint32_t | int32_t | uint64_t | int64_t | + float | double | bool | string +| identifier // must have been previously typedef'd + // or defined with struct, union, enum, or import +| memory +| pointer +| vec<TYPE> +| bitfield<TYPE> // TYPE is user-defined enum +| fmq_sync<TYPE> +| fmq_unsync<TYPE> +| TYPE[SIZE] + +FIELD = + TYPE identifier + +UFIELD = + TYPE identifier + | struct identifier { FIELD; FIELD; ...} identifier; + | union identifier { FIELD; FIELD; ...} identifier; + +SFIELD = + TYPE identifier + | struct identifier { FIELD; FIELD; ...}; + | union identifier { FIELD; FIELD; ...}; + | struct identifier { FIELD; FIELD; ...} identifier; + | union identifier { FIELD; FIELD; ...} identifier; + +SIZE = // Must be greater than zero + constexpr + +ANNOTATIONS = + [empty] + | ANNOTATIONS ANNOTATION + +ANNOTATION = + | @identifier + | @identifier(VALUE) + | @identifier(ANNO_ENTRY, ANNO_ENTRY ...) + +ANNO_ENTRY = + identifier=VALUE + +VALUE = + "any text including \" and other escapes" + | constexpr + | {VALUE, VALUE ...} // only in annotations + +ENUM_ENTRY = + identifier + | identifier = constexpr +</pre> + +<h2 id=terms>Terminology</h2> +<p>This section uses the following HIDL-related terms:</p> + +<table> +<tbody> + +<tr> +<th>binderized</th> +<td>Indicates HIDL is being used for remote procedure calls between processes, +implemented over a Binder-like mechanism. See also <em>passthrough</em>.</td> +</tr> + +<tr> +<th>callback, asynchronous</th> +<td>Interface served by a HAL user, passed to the HAL (via a HIDL method), and +called by the HAL to return data at any time.</td> +</tr> + +<tr> +<th>callback, synchronous</th> +<td>Returns data from a server's HIDL method implementation to the client. +Unused for methods that return void or a single primitive value.</td> +</tr> + +<tr> +<th>client</th> +<td>Process that calls methods of a particular interface. A HAL or framework +process may be a client of one interface and a server of another. See also +<em>passthrough</em>.</td> +</tr> + +<tr> +<th>extends</th> +<td>Indicates an interface that adds methods and/or types to another interface. +An interface can extend only one other interface. Can be used for a minor +version increment in the same package name or for a new package (e.g. a vendor +extension) to build on an older package.</td> +</tr> + +<tr> +<th>generates</th> +<td>Indicates an interface method that returns values to the client. To return +one non-primitive value, or more than one value, a synchronous callback function +is generated.</td> +</tr> + +<tr> +<th>interface</th> +<td>Collection of methods and types. Translated into a class in C++ or Java. All +methods in an interface are called in the same direction: a client process +invokes methods implemented by a server process.</td> +</tr> + +<tr> +<th>oneway</th> +<td>When applied to a HIDL method, indicates the method returns no values and +does not block.</td> +</tr> + +<tr> +<th>package</th> +<td>Collection of interfaces and data types sharing a version.</td> +</tr> + +<tr> +<th>passthrough</th> +<td>Mode of HIDL in which the server is a shared library, <code>dlopen</code>ed +by the client. In passthrough mode, client and server are the same process but +separate codebases. Used only to bring legacy codebases into the HIDL model. +See also <em>Binderized</em>.</td> +</tr> + +<tr> +<th>server</th> +<td>Process that implements methods of an interface. See also +<em>passthrough</em>.</td> +</tr> + +<tr> +<th>transport</th> +<td>HIDL infrastructure that moves data between the server and client.</td> +</tr> + +<tr> +<th>version</th> +<td>Version of a package. Consists of two integers, major and minor. Minor +version increments may add (but not change) types and methods.</td> +</tr> + +</tbody> +</table> + + </body> +</html> diff --git a/en/devices/architecture/hidl/interfaces.html b/en/devices/architecture/hidl/interfaces.html new file mode 100644 index 00000000..a044abde --- /dev/null +++ b/en/devices/architecture/hidl/interfaces.html @@ -0,0 +1,228 @@ +<html devsite> + <head> + <title>Interfaces & Packages</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>HIDL is built around interfaces, an abstract type used in object-oriented +languages to define behaviors. Each interface is part of a package.</p> + +<h2 id=packages>Packages</h2> + +<p>Package names can have sublevels such as <code>package.subpackage</code>. The +root directory for published HIDL packages is <code>hardware/interfaces</code> +or <code>vendor/vendorName</code> (e.g. <code>vendor/google</code> for Pixel +devices). The package name forms one or more subdirectories under the root +directory; all files defining a package are in the same directory. For example, +<code>package android.hardware.example.extension.light@2.0</code> could be found +under <code>hardware/interfaces/example/extension/light/2.0</code>.</p> + +<p>The following table lists package prefixes and locations:</p> +<table> +<tbody> + +<tr> +<th>Package Prefix</th> +<th>Location</th> +</tr> + +<tr> +<td><code>android.hardware.*</code></td> +<td><code>hardware/interfaces/*</code></td> +</tr> + +<tr> +<td><code>android.frameworks.*</code></td> +<td><code>frameworks/hardware/interfaces/*</code></td> +</tr> + +<tr> +<td><code>android.system.*</code></td> +<td><code>system/hardware/interfaces/*</code></td> +</tr> + +<tr> +<td><code>android.hidl.*</code></td> +<td><code>system/libhidl/transport/*</code></td> +</tr> + +</tbody> +</table> + +<p>The package directory contains files with extension <code>.hal</code>. Every +file must contain a <code>package</code> statement naming the package and +version the file is part of. The file <code>types.hal</code>, if present, does +not define an interface but instead defines data types accessible to every +interface in the package.</p> + +<h2 id=interface-def>Interface definition</h2> +<p>Aside from <code>types.hal</code>, every other <code>.hal</code> file defines +an interface. For example, an interface is typically defined as follows:</p> + +<pre class="prettyprint"> +interface IBar extends IFoo { // IFoo is another interface + // embedded types + struct MyStruct {/*...*/}; + + // interface methods + create(int32_t id) generates (MyStruct s); + close(); +}; +</pre> + +<p>An interface without an explicit <code>extends</code> declaration implicitly +extends from <code>android.hidl.base@1.0::IBase</code> (similar to +<code>java.lang.Object</code> in Java.) The IBase interface, implicitly +imported, declares several reserved methods that should not and cannot be +redeclared in user-defined interfaces or used otherwise. These methods +include:</p> + +<ul> +<li><code>ping</code></li> +<li><code>interfaceChain</code></li> +<li><code>interfaceDescriptor</code></li> +<li><code>notifySyspropsChanged</code></li> +<li><code>linkToDeath</code></li> +<li><code>unlinkToDeath</code></li +<li><code>setHALInstrumentation</code></li> +<li><code>getDebugInfo</code></li> +<li><code>debug</code></li> +<li><code>getHashChain</code></li> +</ul> + +<h2 id=import>Importing</h2> +<p>The <code>import</code> statement is HIDL mechanism to access package +interfaces and types in another package. An <code>import</code> statement +concerns itself with two entities:</p> + +<ul> +<li>The import<em>ing</em> entity, which can be either a package or an +interface; and</li> +<li>The import<em>ed</em> entity, which too can be either a package or an +interface.</li> +</ul> + +<p>The importing entity is determined by the location of the +<code>import</code> statement. When the statement is inside a package's +<code>types.hal</code>, what is being imported is visible by the entire package; +this is a <em>package-level</em> import. When the statement is inside an +interface file, the importing entity is the interface itself; this is an +<em>interface-level</em> import.</p> + +<p>The imported entity is determined by the value after the <code>import</code> +keyword. The value need not be a fully-qualified name; if a component is +omitted, it is automatically filled with information from the current package. +For fully-qualified values, the following import cases are supported:</p> + +<ul> +<li><strong>Whole-package imports</strong>. If the value is a package name and a +version (syntax described below), then the entire package is imported into the +importing entity.</li> +<li><strong>Partial imports</strong>. +<ul> +<li>If the value is an interface, the package's <code>types.hal</code> and that +interface are imported into the importing entity.</li> +<li>If the value is an UDT defined in <code>types.hal</code>, then only that UDT +is imported into the importing entity (other types in <code>types.hal</code> are +not imported).</li> +</ul> +<li><strong>Types-only imports</strong>. If the value uses the syntax of a +partial import described above, but with the keyword <code>types</code> instead +of an Interface name, only the UDTs in <code>types.hal</code> of the designated +package are imported.</li> +</ul> + +<p>The importing entity gets access to a combination of:</p> +<ul> +<li>The imported package's common UDTs defined in <code>types.hal</code>;</li> +<li>The imported package's interfaces (for a whole-package import) or specified +interface (for a partial import) for the purposes of invoking them, passing +handles to them and/or inheriting from them.</li> +</ul> + +<p>The import statement uses the fully-qualified-type-name syntax to provide the +name and version of the package or interface being imported:</p> + +<pre class="prettyprint"> +import android.hardware.nfc@1.0; // import a whole package +import android.hardware.example@1.0::IQuux; // import an interface and types.hal +import android.hardware.example@1.0::types; // import just types.hal +</pre> + +<h2 id=inheritance>Interface inheritance</h2> + +<p>An interface can be an extension of a previously-defined interface. +Extensions can be one of the following three types:</p> +<ul> +<li>Interface can add functionality to another one, incorporating its API +unchanged.</li> +<li>Package can add functionality to another one, incorporating its API +unchanged.</li> +<li>Interface can import types from a package or from a specific interface.</li> +</ul> + +<p>An interface can extend only one other interface (no multiple inheritance). +Each interface in a package with a non-zero minor version number must extend an +interface in the previous version of the package. For example, if an interface +<code>IBar</code> in version 4.0 of package <code>derivative</code> is based on +(extends) an interface <code>IFoo</code> in version 1.2 of package +<code>original</code>, and a version 1.3 of package <code>original</code> is +created, <code>IBar</code> version 4.1 cannot extend version 1.3 of +<code>IFoo</code>. Instead, <code>IBar</code> version 4.1 must extend +<code>IBar</code> version 4.0, which is tied to <code>IFoo</code> version 1.2. +<code>IBar</code> version 5.0 could extend <code>IFoo</code> version 1.3, if +desired.</p> + +<p>Interface extensions do not imply library dependence or cross-HAL inclusion +in the generated code—they simply import the data structure and method +definitions at the HIDL level. Every method in a HAL must be implemented in that +HAL.</p> + +<h2 id=vendor-ext>Vendor extensions</h2> +<p>In some cases, vendor extensions will be implemented as a subclass of the +base object that represents the core interface they extend. The same object will +be registered under the base HAL name and version, and under the extension's +(vendor) HAL name and version.</p> + + +<h2 id=version>Versioning</h2> +<p>Packages are versioned, and interfaces have the version of their package. +Versions are expressed in two integers, <em>major</em>.<em>minor</em>.</p> +<ul> +<li><strong>Major</strong> versions are not backwards compatible. Incrementing +the major version number resets the minor version to 0.</li> +<li><strong>Minor</strong> versions are backwards compatible. Incrementing the +minor number indicates the newer version is fully backward compatible with the +previous version. New data structures and methods can be added, but no existing +data structures or method signatures may be changed.</li> +</ul> + +<p>For broader compatibility with frameworks, multiple major versions of a HAL +can be present on a device simultaneously. While multiple minor versions can +also be present on a device, as minor versions are backwards compatible no +reason exists to support more than the latest minor version for each major +version.</p> + +<p>For more details on versioning and vendor extensions, see +<a href="/devices/architecture/hidl/versioning">HIDL Versioning</a>.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/network-stack.html b/en/devices/architecture/hidl/network-stack.html new file mode 100644 index 00000000..88377324 --- /dev/null +++ b/en/devices/architecture/hidl/network-stack.html @@ -0,0 +1,176 @@ +<html devsite> + <head> + <title>Network Stack Configuration Tools</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> +<p>The Android operating system contains standard Linux networking utilities +such as <code>ifconfig</code>, <code>ip</code>, and <code>ip6tables</code>. +These utilities reside on the system image and enable configuration of the +entire Linux networking stack. On devices running Android 7.x and earlier, +vendor code is allowed to call these binaries directly, which presents the +following problems:</p> + +<ul> +<li>Because network utilities are updated in the system image, they do not +provide a stable implementation.</li> +<li>The scope of the networking utilities is so broad it is difficult to evolve +the system image while guaranteeing predictable behaviour.</li> +</ul> + +<p>On devices running Android 8.0, the vendor partition can remain the same +while the system partition receives an update. To achieve this, Android 8.0 +provides the ability to define a stable, versioned interface while also using +SELinux restrictions to keep the interdependency of vendor and system image to a +known good set.</p> + +<p>Vendors can use the platform-provided network configuration utilities to +configure the Linux networking stack, but these utilities do not yet include a +HIDL interface wrapper. To define such an interface, Android 8.0 includes the +<code>netutils-wrapper-1.0</code> tool. +</p> + +<h2 id="netutils-wrapper">Netutils wrapper</h2> +<p>The <code>netutils</code> wrapper utility provides a subset of the Linux +network stack configuration that is not affected by system partition updates. +Android 8.0 contains version 1.0 of the wrappers, which allows you to pass the +same arguments as the wrapped utilities, installed in the system partition at +<code>/system/bin</code> as follows:</p> + +<pre class="prettyprint"> +u:object_r:system_file:s0 /system/bin/ip-wrapper-1.0 -> netutils-wrapper-1.0 +u:object_r:system_file:s0 /system/bin/ip6tables-wrapper-1.0 -> netutils-wrapper-1.0 +u:object_r:system_file:s0 /system/bin/iptables-wrapper-1.0 -> netutils-wrapper-1.0 +u:object_r:system_file:s0 /system/bin/ndc-wrapper-1.0 -> netutils-wrapper-1.0 +u:object_r:netutils_wrapper_exec:s0 /system/bin/netutils-wrapper-1.0 +u:object_r:system_file:s0 /system/bin/tc-wrapper-1.0 -> netutils-wrapper-1.0 +</pre> + +<p>Symlinks show the networking utilities wrapped by the <code>netutils</code> +wrapper, which include:</p> +<ul> +<li><code>ip</code></li> +<li><code>iptables</code></li> +<li><code>ip6tables</code></li> +<li><code>ndc</code></li> +<li><code>tc</code></li> +</ul> + +<p>To use these utilities in Android 8.0 and later, vendor implementations must +adhere to the following rules:</p> +<ul> +<li>Vendor processes must not execute +<code>/system/bin/netutils-wrapper-1.0</code> directly; attempts to do so will +result in error.</li> +<li>All utilities wrapped by <code>netutils-wrapper-1.0</code> must be launched +using their symlinks. For example, change the vendor code that did this before +(<code>/system/bin/ip <FOO> <BAR></code>) to +<code>/system/bin/ip-wrapper-1.0 <FOO> <BAR></code>.</li> +<li>Executing the wrappers without domain transition is prohibited in platform +SELinux policy. This rule must not be changed and is tested against in the +<a href="/compatibility/cts.html">Android Compatibility Test Suite (CTS)</a>. +</li> +<li>Executing the utilities directly (e.g., +<code>/system/bin/ip <FOO> <BAR></code>) from the vendor processes +is also prohibited in the platform SELinux policies. This rule must not be +changed and is tested against in CTS.</li> +<li>Any vendor domain (process) that needs to launch a wrapper must add the +following domain transition rule in the SELinux policy: +<code>domain_auto_trans(<var>VENDOR-DOMAIN-NAME</var>, netutils_wrapper_exec, +netutils_wrapper)</code>.</li> +</ul> + +<aside class="note"><strong>Note:</strong> For details on Android 8.0 SELinux, +see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for Android +8.0</a>.</aside> + +<h2 id="netutils-wrapper-filters">Netutils wrapper filters</h2> +<p>Wrapped utilities can be used to configure almost any aspect of the Linux +networking stack. However, to ensure it is possible to maintain a stable +interface and allow updates to the system partition, only certain combinations +of command line arguments are allowed; other commands will be rejected.</p> + +<h3 id="vendor-interfaces-and-chains">Vendor interfaces and chains</h3> +<p>The wrapper has a concept of <em>vendor interfaces</em>. These are interfaces +typically managed by vendor code, such as cellular data interfaces. Typically, +other types of interfaces (such as Wi-Fi) are managed by the HALs and the +framework. The wrapper recognizes vendor interfaces by name (using a regular +expression) and allows vendor code to perform many operations on them. +Currently, vendor interfaces are:</p> +<ul> +<li>Interfaces whose names end in "oem" followed by a number, such as +<code>oem0</code> or <code>r_oem1234</code>.</li> +<li>Interfaces used by current SOC and OEM implementations, such as +<code>rmnet_data[0-9]</code>.</li> +</ul> + +<p>Names of interfaces that are typically managed by the framework (such as +<code>wlan0</code>) are never vendor interfaces.</p> + +<p>The wrapper has a similar concept of <em>vendor chains</em>. These are used +in <code>iptables</code> commands and are also recognized by name. Currently, +vendor chains:</p> +<ul> +<li>Start with <code>oem_</code>.</li> +<li>Are used by current SOC and OEM implementations, e.g., chains starting in +<code>nm_</code> or <code>qcom_</code>.</li> +</ul> + +<h3 id="allowed-commands">Allowed commands</h3> +<p>Currently allowed commands are listed below. Restrictions are implemented via +a set of regular expressions on the executed command lines. For details, refer +to <code>system/netd/netutils_wrappers/NetUtilsWrapper-1.0.cpp</code>.</p> + +<h4 id="ip">ip</h4> +<p>The <code>ip</code> command is used to configure IP addresses, routing, IPsec +encryption, and a number of other network parameters. The wrapper allows the +following commands:</p> +<ul> +<li>Add and remove IP addresses from vendor-managed interfaces.</li> +<li>Configure IPsec encryption.</li> +</ul> + +<h4 id="iptables-ip6tables">iptables/ip6tables</h4> +<p>The <code>iptables</code> and <code>ip6tables</code> commands are used to +configure firewalling, packet mangling, NAT, and other per-packet processing. +The wrapper allows the following commands:</p> +<ul> +<li>Add and delete vendor chains.</li> +<li>Add and delete rules in any chain that refers to packets going into +(<code>-i</code>) or out of (<code>-o</code>) a vendor interface.</li> +<li>Jump to a vendor chain from any point in any other chain.</li> +</ul> + +<h4 id="ndc">ndc</h4> +<p><code>ndc</code> is used to communicate to the <code>netd</code> daemon that +performs most network configuration on Android. The wrapper allows the following +commands:</p> +<ul> +<li>Create and destroy OEM networks (<code>oemXX</code>).</li> +<li>Add vendor-managed interfaces to OEM networks.</li> +<li>Add routes to OEM networks.</li> +<li>Enable or disable IP forwarding globally and on vendor interfaces.</li> +</ul> + +<h4 id="tc">tc</h4> +<p>The <code>tc</code> command is used to configure traffic queueing and shaping +on vendor interfaces.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/services.html b/en/devices/architecture/hidl/services.html new file mode 100644 index 00000000..647555b7 --- /dev/null +++ b/en/devices/architecture/hidl/services.html @@ -0,0 +1,219 @@ +<html devsite> + <head> + <title>Services & Data Transfer</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>This section describes how to register and discover services and how to send +data to a service by calling methods defined in interfaces in <code>.hal</code> +files.</p> + +<h2 id=register>Registering services</h2> +<p>HIDL interface servers (objects implementing the interface) can be registered +as named services. The registered name need not be related to the interface or +package name. If no name is specified, the name "default" is used; this should +be used for HALs that do not need to register two implementations of the same +interface. For example, the C++ call for service registration defined in each +interface is:</p> + +<pre class="prettyprint"> +registerAsService(); +registerAsService("another_foo_service"); // if needed +</pre> + +<p>The version of a HIDL interface is included in the interface itself. It is +automatically associated with service registration and can be retrieved via a +method call (<code>android::hardware::IInterface::getInterfaceVersion()</code>) +on every HIDL interface. Server objects need not be registered and can be passed +via HIDL method parameters to another process that will make HIDL method calls +into the server.</p> + +<h2 id=discover>Discovering services</h2> +<p>Requests by client code are made for a given interface by name and by +version, calling <code>getService</code> on the desired HAL class:</p> + +<pre class="prettyprint"> +sp<V1_1::IFooService> service = V1_1::IFooService::getService(); +sp<V1_1::IFooService> alternateService = 1_1::IFooService::getService("another_foo_service"); +</pre> + +<p>Each version of a HIDL interface is treated as a separate interface. Thus, +<code>IFooService</code> version 1.1 and <code>IFooService</code> version 2.2 +can both be registered as "foo_service" and +<code>getService("foo_service")</code> on either interface gets the registered +service for that interface. This is why, in most cases, no name parameter needs +to be supplied for registration or discovery (meaning name "default").</p> + +<p>The Vendor Interface Object also plays a part in the transport method of the +returned interface. For an interface <code>IFoo</code> in package +<code>android.hardware.foo@1.0</code>, the returned interface by +<code>IFoo::getService</code> always use the transport method declared for +<code>android.hardware.foo</code> in the device manifest if the entry exists; +and if the transport method is not available, nullptr is returned.</p> + +<h2 id=death>Service death notifications</h2> +<p>Clients who want to be notified when a service dies can receive death +notifications delivered by the framework. To receive notifications, the client +must:</p> +<ol> +<li>Subclass the HIDL class/interface <code>hidl_death_recipient</code> (in C++ +code, not in HIDL).</li> +<li>Override its <code>serviceDied()</code> method.</li> +<li>Instantiate an object of the <code>hidl_death_recipient</code> subclass. +</li> +<li>Call the <code>linkToDeath()</code> method on the service to monitor, +passing in the <code>IDeathRecipient</code>'s interface object.</li> +</ol> + +<p>A pseudocode example (C++ and Java are similar):</p> + +<pre class="prettyprint"> +class IMyDeathReceiver : hidl_death_recipient { + virtual void serviceDied(uint64_t cookie, + wp<IBase>& service) override { + log("RIP service %d!", cookie); // Cookie should be 42 + } +}; +.... +IMyDeathReceiver deathReceiver = new IMyDeathReceiver(); +m_importantService->linkToDeath(deathReceiver, 42); +</pre> + +<p>The same death recipient may be registered on multiple different services. +</p> + +<h2 id=data-transwer>Data transfer</h2> +<p>Data may be sent to a service by calling methods defined in interfaces in +<code>.hal</code> files. There are two kinds of methods:</p> + +<ul> +<li><strong>Blocking</strong> methods wait until the server has produced a +result.</li> +<li><strong>Oneway</strong> methods send data in only one direction and do not +block. If the amount of data in-flight in RPC calls exceeds implementation +limits, the calls may either block or return an error indication (behavior is +not yet determined).</li> +</ul> + +<p>A method that does not return a value but is not declared as +<code>oneway</code> is still blocking.</p> + +<p>All methods declared in a HIDL interface are called in a single direction, +either from the HAL or into the HAL. The interface does not specify which +direction it will be called in. Architectures that need calls to originate from +the HAL should provide two (or more) interfaces in the HAL package and serve the +appropriate interface from each process. The words <em>client</em> and +<em>server</em> are used with respect to the calling direction of the interface +(i.e. the HAL can be a server of one interface and a client of another +interface).</p> + +<h3 id=callbacks>Callbacks</h3> +<p>The word <em>callback</em> refers to two different concepts, distinguished by +<em>synchronous callback</em> and <em>asynchronous callback</em>.</p> + +<p><em>Synchronous callbacks</em> are used in some HIDL methods that return +data. A HIDL method that returns more than one value (or returns one value of +non-primitive type) returns its results via a callback function. If only one +value is returned and it is a primitive type, a callback is not used and the +value is returned from the method. The server implements the HIDL methods and +the client implements the callbacks.</p> + +<p><em>Asynchronous callbacks</em> allow the server of a HIDL interface to +originate calls. This is done by passing an instance of a second interface +through the first interface. The client of the first interface must act as the +server of the second. The server of the first interface can call methods on the +second interface object. For example, a HAL implementation may send information +asynchronously back to the process that is using it by calling methods on an +interface object created and served by that process. Methods in interfaces used +for asynchronous callback may be blocking (and may return values to the caller) +or <code>oneway</code>. For an example, see "Asynchronous callbacks" in +<a href="/devices/architecture/hidl-cpp/interfaces.html">HIDL C++</a>.</p> + +<p>To simplify memory ownership, method calls and callbacks take only +<code>in</code> parameters and do not support <code>out</code> or +<code>inout</code> parameters.</p> + +<h3 id=limits>Per-transaction limits</h3> +<p>Per-transaction limits may be imposed on the amount of data sent in HIDL +methods and callbacks. The limits are yet to be determined but may be as small +as 4K. Calls exceeding these limits return failure immediately. Another +limitation is the resources available to the HIDL infrastructure to handle +multiple simultaneous transactions. Multiple transactions can be in-flight +simultaneously due to multiple threads or processes sending calls to a process +or multiple <code>oneway</code> calls that are not handled quickly by the +receiving process.</p> + +<p>In a well-designed interface, exceeding these resource limitations should not +happen; if it does, the call which exceeded them may either block until +resources become available or signal a transport error. Each occurrence of +exceeding per-transaction limits or overflowing HIDL implementation resources by +aggregate in-flight transactions is logged to facilitate debugging.</p> + +<h3 id=method-implement>Method implementations</h3> +<p>HIDL generates header files declaring the necessary types, methods, and +callbacks in the target language (C++ or Java). The prototype of HIDL-defined +methods and callbacks is the same for both client and server code. The HIDL +system provides <strong>proxy</strong> implementations of the methods on the +caller side that organize the data for IPC transport, and <strong>stub</strong> +code on the callee side that passes the data into developer implementations of +the methods.</p> + +<p>The caller of a function (HIDL method or callback) has ownership of the data +structures passed into the function, and retains ownership after the call; in +all cases the callee does not need to free or release the storage.</p> + +<ul> +<li>In C++, the data may be read-only (attempts to write to it may cause a +segmentation fault) and are valid for the duration of the call. The client can +deep-copy the data to propagate it beyond the call.</li> +<li>In Java, the code receives a local copy of the data (a normal Java object), +which it may keep and modify or allow to be garbage-collected.</li> +</ul> + +<h3 id=non-rpc>Non-RPC data transfer</h3> +<p>HIDL has two ways to transfer data without using an RPC call: shared +memory and a Fast Message Queue (FMQ), both supported only in C++.</p> + +<ul> +<li><strong>Shared memory</strong>. The built-in HIDL type <code>memory</code> +is used to pass an object representing shared memory that has been allocated. +Can be used in a receiving process to map the shared memory.</li> +<li><strong>Fast Message Queue (FMQ)</strong>. HIDL provides a templated message +queue type that implements no-wait message-passing. It does not use the kernel +or scheduler in passthrough or binderized mode (inter-device communication will +not have these properties). Typically, the HAL sets up its end of the queue, +creating an object that can be passed through RPC via a parameter of built-in +HIDL type <code>MQDescriptorSync</code> or <code>MQDescriptorUnsync</code>. This +object can be used by the receiving process to set up the other end of the queue. +<ul> +<li><em>Sync</em> queues are not allowed to overflow, and can only have one +reader.</li> +<li><em>Unsync</em> queues are allowed to overflow, and can have many readers, +each of which must read data in time or lose it.</li> +</ul> +Neither type is allowed to underflow (read from an empty queue will fail), and +each type can only have one writer.</li></ul> + +<p>For more details on FMQ, see +<a href="/devices/architecture/hidl/fmq.html">Fast Message Queue (FMQ)</a>.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/threading.html b/en/devices/architecture/hidl/threading.html new file mode 100644 index 00000000..059b61d4 --- /dev/null +++ b/en/devices/architecture/hidl/threading.html @@ -0,0 +1,156 @@ +<html devsite> + <head> + <title>Threading Models</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p>Methods marked as <code>oneway</code> do not block. For methods not marked as +<code>oneway</code>, a client's method call will block until the server has +completed execution or called a synchronous callback (whichever comes first). +Server method implementations may call at most one synchronous callback; extra +callback calls are discarded and logged as errors. If a method is supposed to +return values via callback and does not call its callback, this is logged as an +error and reported as a transport error to the client.</p> + +<h2 id=passthrough>Threads in passthrough mode</h2> +<p>In passthrough mode, most calls are synchronous. However, to preserve the +intended behavior that <code>oneway</code> calls do not block the client, a +thread is created for each process. For details, see the +<a href="/devices/architecture/hidl/index.html#passthrough">HIDL overview</a>. +</p> + +<h2 id=binderized>Threads in binderized HALs</h2> +<p>To serve incoming RPC calls (including asynchronous callbacks from HALs to +HAL users) and death notifications, a threadpool is associated with each process +that uses HIDL. If a single process implements multiple HIDL interfaces and/or +death notification handlers, its threadpool is shared between all of them. When +a process receives an incoming method call from a client, it picks a free thread +from the threadpool and executes the call on that thread. If no free thread is +available, it blocks until one is available.</p> + +<p>If the server has only one thread, then calls into the server are completed +in order. A server with more than one thread may complete calls out of order +even if the client has only one thread. As <code>oneway</code> calls do not +block the client, multiple <code>oneway</code> calls may be processed +simultaneously or out of order by a server with more than one thread, and +<code>oneway</code> calls may be processed concurrently with a subsequent +blocking call.</p> + +<h2 id=model>Server threading model</h2> +<p>Except for passthrough mode, server implementations of HIDL interfaces live +in a different process than the client and need one or more threads waiting for +incoming method calls. These threads are the server's threadpool; the server may +decide how many threads it wants running in its threadpool, and can use a +threadpool size of one to serialize all calls on its interfaces. If the server +has more than one thread in the threadpool, it can receive concurrent incoming +calls on any of its interfaces (in C++, this means that shared data must be +carefully locked).</p> + +<p>Oneway calls into the same interface are serialized. If a multi-threaded +client calls <code>method1</code> and <code>method2</code> on interface +<code>IFoo</code>, and <code>method3</code> on interface <code>IBar</code>, +<code>method1</code> and <code>method2</code> will always be serialized, but +<code>method3</code> may run in parallel with <code>method1</code> and +<code>method2</code>.</p> + +<p>A single client thread of execution can cause concurrent execution on a +server with multiple threads in two ways:</p> + +<ul> +<li><code>oneway</code> calls do not block. If a <code>oneway</code> call is +executed and then a non-<code>oneway</code> is called, the server may execute +the <code>oneway</code> call and the non-<code>oneway</code> call +simultaneously.</li> +<li>Server methods that pass data back with synchronous callbacks can unblock +the client as soon as the callback is called from the server.</li> +</ul> + +<p>For the second way, any code in the server function that executes after the +callback is called may execute concurrently, with the server handling subsequent +calls from the client. This includes code in the server function and automatic +destructors that execute at the end of the function. If the server has more than +one thread in its threadpool, concurrency issues arise even if calls are coming +in from only one single client thread. (If any HAL served by a process needs +multiple threads, all HALs will have multiple threads because the threadpool is +shared per-process.)</p> + +<p>As soon as the server calls the provided callback, the transport can call the +implemented callback on the client and unblock the client. The client proceeds +in parallel with whatever the server implementation does after it calls the +callback (which may include running destructors). Code in the server function +after the callback is no longer blocking the client (as long as the server +threadpool has enough threads to handle incoming calls), but may be executed +concurrently with future calls from the client (unless the server threadpool has +only one thread).</p> + +<p>In addition to synchronous callbacks, <code>oneway</code> calls from a +single-threadedclient may be handled concurrently by a server with multiple +threads in its threadpool, but only if those <code>oneway</code> calls are +executed on different interfaces. <code>oneway</code> calls on the same +interface are always serialized.</p> + +<p class=note><strong>Note:</strong> We strongly encourage server functions to +return as soon as they have called the callback function.</p> + +<p>For example (in C++):</p> + +<pre class="prettyprint"> +Return<void> someMethod(someMethod_cb _cb) { + // Do some processing, then call callback with return data + hidl_vec<uint32_t> vec = ... + _cb(vec); + // At this point, the client's callback will be called, + // and the client will resume execution. + ... + return Void(); // is basically a no-op +}; +</pre> + +<h2 id=client>Client threading model</h2> +<p>The threading model on the client differs between non-blocking calls +(functions that are marked with the <code>oneway</code> keyword) and blocking +calls (functions that do not have the <code>oneway</code> keyword specified).</p> + +<h3 id=block>Blocking calls</h3> +<p>For blocking calls, the client blocks until one of the following happens:</p> + +<ul> +<li>Transport error occurs; the <code>Return</code> object contains an error +state that can be retrieved with <code>Return::isOk()</code>.</li> +<li>Server implementation calls the callback (if there was one).</li> +<li>Server implementation returns a value (if there was no callback parameter). +</li> +</ul> + +<p>In case of success, the callback function the client passes as an argument is +always called by the server before the function itself returns. The callback is +executed on the same thread that the function call is made on, so implementers +must be careful with holding locks during function calls (and avoid them +altogether when possible). A function without a <code>generates</code> statement +or a <code>oneway</code> keyword is still blocking; the client blocks until the +server returns a <code>Return<void></code> object.</p> + +<h3 id=oneway>Oneway calls</h3> +<p>When a function is marked <code>oneway</code>, the client returns immediately +and does not wait for the server to complete its function call invocation.</p> + + </body> +</html> diff --git a/en/devices/architecture/hidl/types.html b/en/devices/architecture/hidl/types.html new file mode 100644 index 00000000..1f36262d --- /dev/null +++ b/en/devices/architecture/hidl/types.html @@ -0,0 +1,408 @@ +<html devsite> + <head> + <title>Data Types</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This section describes HIDL data types. For implementation details, see +<a href="/devices/architecture/hidl-cpp/index.html">HIDL C++</a> (for C++ +implementations) or <a href="/devices/architecture/hidl-java/index.html">HIDL +Java</a> (for Java implementations).</p> + +<p>Similarities to C++ include:</p> +<ul> +<li><code>structs</code> use C++ syntax; <code>unions</code> support C++ syntax +by default. Both must be named; anonymous structs and unions are not supported. +</li> +<li>Typedefs are allowed in HIDL (as they are in C++).</li> +<li>C++-style comments are allowed and are copied to the generated header file. +</li> +</ul> + +<p>Similarities to Java include:</p> +<ul> +<li>For each file, HIDL defines a Java-style namespace that must begin with +<code>android.hardware.</code>. The generated C++ namespace is +<code>::android::hardware::…</code>.</li> +<li>All definitions of the file are contained within a Java-style +<code>interface</code> wrapper.</li> +<li>HIDL array declarations follow the Java style, not the C++ style. Example: +<pre class="prettyprint"> +struct Point { + int32_t x; + int32_t y; +}; +Point[3] triangle; // sized array +</pre> +</li> +<li>Comments are similiar to the javadoc format.</li> +</ul> + +<h2 id=represent>Data representation</h2> +<p>A <code>struct</code> or <code>union</code> composed of +<a href="http://en.cppreference.com/w/cpp/language/data_members#Standard_layout">Standard-Layout</a> +(a subset of the requirement of plain-old-data types) has a consistent memory +layout in generated C++ code, enforced with explicit alignment attributes on +<code>struct</code> and <code>union</code> members.</p> + +<p>Primitive HIDL types, as well as <code>enum</code> and <code>bitfield</code> +types (which always derive from primitive types), map to standard C++ types +such as <code>std::uint32_t</code> from +<a href="http://en.cppreference.com/w/cpp/types/integer">cstdint</a>.</p> + +<p>As Java does not support unsigned types, unsigned HIDL types are mapped to +the corresponding signed Java type. <em>Structs</em> map to Java classes; +<em>arrays</em> map to Java arrays; <em>unions</em> are not currently supported +in Java. <em>Strings</em> are stored internally as UTF8. Since Java supports +only UTF16 strings, string values sent to or from a Java implementation are +translated, and may not be identical on re-translation as the character sets do +not always map smoothly.</p> + +<p>Data received over IPC in C++ is marked <code>const</code> and is in +read-only memory that persists only for the duration of the function call. Data +received over IPC in Java has already been copied into Java objects, so it can +be retained without additional copying (and may be modified).</p> + +<h2 id=annotations>Annotations</h2> +<p>Java-style annotations may be added to type declarations. Annotations are +parsed by the Vendor Test Suite (VTS) backend of the HIDL compiler but none of +such parsed annotations are actually understood by the HIDL compiler. Instead, +parsed VTS annotations are handled by the VTS Compiler (VTSC).</p> + +<p>Annotations use Java syntax: <code>@annotation</code> or +<code>@annotation(value)</code> or <code>@annotation(id=value, id=value…)</code> +where value may be either a constant expression, a string, or a list of values +inside <code>{}</code>, just as in Java. Multiple annotations of the same name +can be attached to the same item.</p> + +<h2 id=forward>Forward declarations</h2> +<p>In HIDL, structs may not be forward-declared, making user-defined, +self-referential data types impossible (e.g., you cannot describe a linked list +or a tree in HIDL). Most existing (pre-Android O) HALs have limited use of +forward declarations, which can be removed by rearranging data structure +declarations.</p> + +<p>This restriction allows data structures to be copied by-value with a simple +deep-copy, rather than keeping track of pointer values that may occur multiple +times in a self-referential data structure. If the same data is passed twice, +such as with two method parameters or <code>vec<T></code>'s that point to +the same data, two separate copies are made and delivered.</p> + +<h2 id=nested>Nested declarations</h2> +<p>HIDL supports nested declarations to as many levels as desired (with one +exception noted below). For example:</p> + +<pre class="prettyprint"> +interface IFoo { + uint32_t[3][4][5][6] multidimArray; + + vec<vec<vec<int8_t>>> multidimVector; + + vec<bool[4]> arrayVec; + + struct foo { + struct bar { + uint32_t val; + }; + bar b; + } + struct baz { + foo f; + foo.bar fb; // HIDL uses dots to access nested type names + } + … +</pre> + +<p>The exception is that interface types can only be embedded in +<code>vec<T></code> and only one level deep (no +<code>vec<vec<IFoo>></code>).</p> + +<h2 id=raw-pointer>Raw pointer syntax</h2> +<p>The HIDL language does not use <strong>*</strong> and does not support the +full flexibility of C/C++ raw pointers. For details on how HIDL encapsulates +pointers and arrays/vectors, see <a href="#vec">vec<T> template</a>.</p> + +<h2 id=interfaces>Interfaces</h2> +<p>The <code>interface</code> keyword has two usages.</p> + +<ul> +<li>It opens the definition of an interface in a .hal file.</li> +<li>It can be used as a special type in struct/union fields, method parameters, +and returns. It is viewed as a general interface and synonym to +<code>android.hidl.base@1.0::IBase</code>.</li> +</ul> + +<p>For example, <code>IServiceManager</code> has the following method:</p> +<pre class="prettyprint"> +get(string fqName, string name) generates (interface service); +</pre> + +<p>The method promises to lookup <em>some interface</em> by name. It is also +identical to replace interface with <code>android.hidl.base@1.0::IBase</code>. +</p> + +<p>Interfaces can be only passed in two ways: as top-level parameters, or as +members of a <code>vec<IMyInterface></code>. They cannot be members of +nested vecs, structs, arrays, or unions.</p> + +<h2 id=mqdescriptor>MQDescriptorSync & MQDescriptorUnsync</h2> +<p>The <code>MQDescriptorSync</code> and <code>MQDescriptorUnsync</code> types +pass a synchronized or unsynchronized Fast Message Queue (FMQ) descriptors +across a HIDL interface. For details, see +<a href="/devices/architecture/hidl-cpp/index.html">HIDL C++</a> (FMQs are not +supported in Java).</p> + +<h2 id=memory>memory type</h2> +<p>The <code>memory</code> type is used to represent unmapped shared memory in +HIDL. It is only supported in C++. A value of this type can be used on the +receiving end to initialize an <code>IMemory</code> object, mapping the memory +and making it usable. For details, see +<a href="/devices/architecture/hidl-cpp/index.html">HIDL C++</a>.</p> + +<p class=warning><strong>Warning:</strong> Structured data placed in shared +memory MUST be a type whose format will never change for the lifetime of the +interface version passing the <code>memory</code>. Otherwise, HALs may suffer +fatal compatibility problems.</p> + +<h2 id=pointer>pointer type</h2> +<p>The <code>pointer</code> type is for HIDL internal use only.</p> + +<h2 id=bitfield>bitfield<T> type template</h2> +<p><code>bitfield<T></code> in which <code>T</code> is a +<a href="#enum">user-defined enum</a> suggests the value is a bitwise-OR of the +enum values defined in <code>T</code>. In generated code, +<code>bitfield<T></code> appears as the underlying type of T. For +example:</p> + +<pre class="prettyprint"> +enum Flag : uint8_t { + HAS_FOO = 1 << 0, + HAS_BAR = 1 << 1, + HAS_BAZ = 1 << 2 +}; +typedef bitfield<Flag> Flags; +setFlags(Flags flags) generates (bool success); +</pre> + +<p>The compiler handles the type Flags the same as <code>uint8_t</code>.</p> + +<p>Why not use +<code>(u)int8_t</code>/<code>(u)int16_t</code>/<code>(u)int32_t</code>/<code>(u)int64_t</code>? +Using <code>bitfield</code> provides additional HAL information to the reader, +who now knows that <code>setFlags</code> takes a bitwise-OR value of Flag (i.e. +knows that calling <code>setFlags</code> with 16 is invalid). Without +<code>bitfield</code>, this information is conveyed only via documentation. In +addition, VTS can actually check if the value of flags is a bitwise-OR of Flag. +</p> + +<h2 id=handle-primitive>handle primitive type</h2> + +<p class=warning><strong>WARNING:</strong> Addresses of any kind (even physical +device addresses) must never be part of a native handle. Passing this +information between processes is dangerous and makes them susceptible to attack. +Any values passed between processes must be validated before they are used to +look up allocated memory within a process. Otherwise, bad handles may cause bad +memory access or memory corruption.</p> + +<p>HIDL semantics are copy-by-value, which implies that parameters are copied. +Any large pieces of data, or data that needs to be shared between processes +(such as a sync fence), are handled by passing around file descriptors pointing +to persistent objects: <code>ashmem</code> for shared memory, actual files, or +anything else that can hide behind a file descriptor. The binder driver +duplicates the file descriptor into the other process.</p> + +<h3 id=handle_t>native_handle_t</h3> +<p>Android supports <code>native_handle_t</code>, a general handle concept +defined in <code>libcutils</code>.</p> + +<pre class="prettyprint"> +typedef struct native_handle +{ + int version; /* sizeof(native_handle_t) */ + int numFds; /* number of file-descriptors at &data[0] */ + int numInts; /* number of ints at &data[numFds] */ + int data[0]; /* numFds + numInts ints */ +} native_handle_t; +</pre> + +<p>A native handle is a collection of ints and file descriptors that gets passed +around by value. A single file descriptor can be stored in a native handle with +no ints and a single file descriptor. Passing handles using native handles +encapsulated with the <code>handle</code> primitive type ensures that native +handles are directly included in HIDL.</p> + +<p>As a <code>native_handle_t</code> has variable size, it cannot be included +directly in a struct. A handle field generates a pointer to a separately +allocated <code>native_handle_t</code>.</p> + +<p>In earlier versions of Android, native handles were created using the same +functions present in +<a href="https://android.googlesource.com/platform/system/core/+/master/libcutils/native_handle.c">libcutils</a>. +In Android O, these functions are now copied to the +<code>android::hardware::hidl</code> namespace or moved to the NDK. HIDL +autogenerated code serializes and deserializes these functions automatically, +without involvement from user-written code.</p> + +<h3 id=ownership>Handle and file descriptor ownership</h3> +<p>When you call a HIDL interface method that passes (or returns) a +<code>hidl_handle</code> object (either top-level or part of a compound type), +the ownership of the file descriptors contained in it is as follows:</p> + +<ul> +<li>When passing a <code>hidl_handle</code> object as an argument, the caller +retains ownership of the file descriptors contained in the +<code>native_handle_t</code> it wraps, and must close them when it is done with +them. Likewise, when returning a <code>hidl_handle</code> object (by passing it +into a <code>_cb</code> function), the process returning it retains ownership of +the file descriptors contained in the <code>native_handle_t</code> it wraps, and +must close them when it is done with them.</li> +<li>When receiving a <code>hidl_handle</code> object, the +<strong>transport</strong> owns the file descriptors inside the +<code>native_handle_t</code> it wraps; the receiver can use them as-is during +the transaction callback, but must clone the native handle if it wants to keep +using its file descriptors beyond the callback. The transport will automatically +<code>close()</code> the file descriptors when the transaction is done.</li> +</ul> + +<p>HIDL does not support handles in Java (as Java doesn't support handles at +all).</p> + +<h2 id=sized-arrays>Sized arrays</h2> +<p>For sized arrays in HIDL structs, their elements can be of any type a struct +can contain:</p> + +<pre class="prettyprint"> +struct foo { +uint32_t[3] x; // array is contained in foo +}; +</pre> + +<h2 id=strings>Strings</h2> +<p>Strings appear differently in C++ and Java, but the underlying transport +storage type is a C++ structure. For details, see +<a href="/devices/architecture/hidl-cpp/types.html">HIDL C++ Data Types</a> or +<a href="/devices/architecture/hidl-java/types.html">HIDL Java Data Types</a>. +</p> + +<p class=note><strong>Note:</strong> Passing a string to or from Java through a +HIDL interface (including Java to Java) will cause character set conversions +that may not exactly preserve the original encoding.</p> + +<h2 id=vec>vec<T> type template</h2> +<p>The <code>vec<T></code> template represents a variable-sized buffer +containing instances of <code>T</code>. <code>T</code> can be any HIDL-provided +or user-defined type except handle. (A <code>vec<></code> of +<code>vec<T></code> will point to an array of the +<code>vec<T></code> structs, not an array of the inner T buffers.)</p> + +<p><code>T</code> can be one of the following:</p> +<ul> +<li>Primitive types (e.g. uint32_t)</li> +<li>Strings</li> +<li>User-defined enums</li> +<li>User-defined structs</li> +<li>Interfaces, or the <code>interface</code> keyword +(<code>vec<IFoo></code>, <code>vec<interface></code> is supported +only as a top-level parameter)</li> +<li>Handles</li> +<li>bitfield<U></li> +<li>vec<U>, where U is in this list except interface (e.g. +<code>vec<vec<IFoo>></code> is not supported)</li> +<li>U[] (sized array of U), where U is in this list except interface</li> +</ul> + +<h2 id=user>User-defined types</h2> +<p>This section describes user-defined types.</p> + +<h3 id=enum>Enum</h3> +<p>HIDL does not support anonymous enums. Otherside, enums in HIDL are similar +to C++11:</p> +<pre class="prettyprint"> +enum name : type { enumerator , enumerator = constexpr , … } +</pre> + +<p>Enums are defined in terms of one of the primitive types in HIDL, or as an +extension of other enums. For example:</p> +<pre class="prettyprint"> +enum Color : uint32_t { RED = 0, GREEN, BLUE = 2 } // GREEN == 1 +</pre> + +<p>Values of enums are referred to with the colon syntax (not dot syntax as +nested types). The syntax is <code>Type:VALUE_NAME</code>. No need to specify +type if the value is referred in the same enum type or child types. Example:</p> +<pre class="prettyprint"> +enum Grayscale : uint32_t { BLACK = 0, WHITE = BLACK + 1 }; +enum Color : Grayscale { RED = WHITE + 1 }; +enum Unrelated : uint32_t { FOO = Color:RED + 1 }; +</pre> + +<h3 id=struct>Struct</h3> +<p>HIDL does not support anonymous structs. Otherwise, structs in HIDL are very +similar to C.</p> + +<p>HIDL does not support variable-length data structures contained wholly within +a struct. This includes the indefinite-length array that is sometimes used as +the last field of a struct in C/C++ (sometimes seen with a size of +<code>[0]</code>). HIDL <code>vec<T></code> represents dynamically-sized +arrays with the data stored in a separate buffer; such instances are represented +with an instance of the <code>vec<T></code> in the <code>struct</code>. +</p> + +<p>Similarly, <code>string</code> can be contained in a <code>struct</code> +(associated buffers are separate). In the generated C++, instances of the HIDL +handle type are represented via a pointer to the actual native handle as +instances of the underlying data type are variable-length.</p> + +<h3 id=union>Union</h3> +<p>HIDL does not support anonymous unions. Otherwise, unions are similar to C. +</p> + +<p>Unions cannot contain fix-up types (pointers, file descriptors, binder +objects, etc.). They do not need special fields or associated types and are +simply copied via <code>memcpy()</code> or equivalent. An union may not directly +contain (or contain via other data structures) anything that requires setting +binder offsets (i.e., handle or binder-interface references). For example:</p> + +<pre class="prettyprint"> +union UnionType { +uint32_t a; +// vec<uint32_t> r; // Error: can't contain a vec<T> +uint8_t b;1 +}; +fun8(UnionType info); // Legal +</pre> + +<p>Unions can also be declared inside of structs. For example:</p> +<pre class="prettyprint"> +struct MyStruct { + union MyUnion { + uint32_t a; + uint8_t b; + }; // declares type but not member + + union MyUnion2 { + uint32_t a; + uint8_t b; + } data; // declares type but not member + } +</pre> + + </body> +</html> diff --git a/en/devices/architecture/hidl/versioning.html b/en/devices/architecture/hidl/versioning.html new file mode 100644 index 00000000..aed6f9fe --- /dev/null +++ b/en/devices/architecture/hidl/versioning.html @@ -0,0 +1,655 @@ +<html devsite> + <head> + <title>Versioning</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>HIDL requires every interface written in HIDL be versioned. After a HAL +interface is published, it is frozen and any further changes must be made to a +new version of that interface. While a given published interface may not be +modified, it can be extended by another interface.</p> + +<h2 id=code-structure>HIDL code structure</h2> + +<p> +<a href="/reference/hidl/index.html">HIDL code is organized</a> in user-defined +types, interfaces, and packages:</p> + +<ul> +<li><strong>User-defined types (UDTs)</strong>. HIDL provides access to a set of +primitive data types that can be used to compose more complex types via +structures, unions, and enumerations</a>. UDTs are passed to methods of +interfaces, and can be defined at the level of a package (common to all +interfaces) or locally to an interface.</li> +<li><strong>Interfaces</strong>. As a basic building block of HIDL, an interface +consists of UDT and method declarations. Interfaces can also inherit from +another interface.</li> +<li><strong>Packages</strong>. Organizes related HIDL interfaces and the data +types on which they operate. A package is identified by a name and a version and +includes the following: +<ul> + <li>Data-type definition file called <code>types.hal</code>.</li> + <li>Zero or more interfaces, each in their own <code>.hal</code> file.</li> +</ul> +</li> +</ul> + +<p>The data-type definition file <code>types.hal</code> contains only UDTs (all +package-level UDTs are kept in a single file). Representations in the target +language are available to all interfaces in the package.</p> + +<h2 id=philosophy>Versioning philosophy</h2> +<p>A HIDL package (such as <code>android.hardware.nfc</code>), after being +published for a given version (such as <code>1.0</code>), is immutable; it +cannot be changed. Modifications to the interfaces in the package or any +changes to its UDTs can take place only in <em>another</em> package.</p> + +<p>In HIDL, versioning applies at the package level, not at the interface level, +and all interfaces and UDTs in a package share the same version. Package +versions follow <a href="http://semver.org/" class="external">semantic +versioning</a> without the patch level and build-metadata components. Within a +given package, a <strong>minor version</strong> bump implies the new version of +the package is backwards-compatible with the old package and a <strong>major +version</strong> bump implies the new version of the package is not +backwards-compatible with the old package.</p> + +<p>Conceptually, a package can relate to another package in one of several ways: +</p> + +<ul> +<li><strong>Not at all</strong>.</li> +<li><strong>Package-level backwards-compatible extensibility</strong>. This +occurs for new minor-version uprevs (next incremented revision) of a package; +the new package has the same name and major version as the old package, but a +higher minor version. Functionally, the new package is a superset of the old +package, meaning: +<ul> + <li>Top-level interfaces of the parent package are present in the new package, + though the interfaces may have new methods, new interface-local UDTs (the + interface-level extension described below), and new UDTs in + <code>types.hal</code>.</li> + <li>New interfaces may also be added to the new package.</li> + <li>All data types of the parent package are present in the new package and + can be handled by the (possibly reimplemented) methods from the old package. + </li> + <li>New data types may also be added for use by either new methods of uprev'ed + existing interfaces, or by new interfaces.</li> +</ul> +</li> + +<li><strong>Interface-level backwards-compatible extensibility</strong>. The new +package can also extend the original package by consisting of logically separate +interfaces that simply provide additional functionality, and not the core one. +For this purpose, the following may be desirable: +<ul> + <li>Interfaces in the new package need recourse to the data types of the old + package.</li> + <li>Interfaces in new package may extend interfaces of one or more old + packages.</li> +</ul> +</li> +<li><strong>Extend the original backwards-incompatibility</strong>. This is a +major-version uprev of the package and there need not be any correlation between +the two. To the extent that there is, it can be expressed with a combination of +types from the older version of the package, and inheritance of a subset of +old-package interfaces.</li> +</ul> + +<h2 id=code-layout>HIDL code layout</h2> +<p>HIDL includes core and vendor packages.</p> + +<p>Core HIDL interfaces are those specified by Google. The packages they belong +to start with <code>android.hardware.</code> and are named by subsystem, +potentially with nested levels of naming. For example, the NFC package is named +<code>android.hardware.nfc</code> and the camera package is +<code>android.hardware.camera</code>. In general, a core package has the name +<code>android.hardware.</code>[<code>name1</code>].[<code>name2</code>]…. +HIDL packages have a version in addition to their name. For example, the package +<code>android.hardware.camera</code> may be at version <code>3.4</code>; this is +important, as the version of a package affects its placement in the source tree. +</p> + +<p>All core packages are placed under <code>hardware/interfaces/</code> in the +build system. The package +<code>android.hardware.</code>[<code>name1</code>].[<code>name2</code>]… +at version <code>$m.$n</code> is under +<code>hardware/interfaces/name1/name2/</code>…<code>/$m.$n/</code>; package +<code>android.hardware.camera</code> version <code>3.4</code> is in directory +<code>hardware/interfaces/camera/3.4/.</code> A hard-coded mapping exists +between the package prefix <code>android.hardware.</code> and the path +<code>hardware/interfaces/</code>.</p> + +<p>Non-core (vendor) packages are those produced by the SoC vendor or ODM. The +prefix for non-core packages is <code>vendor.$(VENDOR).hardware.</code> where +<code>$(VENDOR)</code>refers to an SoC vendor or OEM/ODM. This maps to the path +<code>vendor/$(VENDOR)/interfaces</code> in the tree (this mapping is also +hard-coded).</p> + +<h2 id=fqn>Fully-qualified user-defined-type names</h2> +<p>In HIDL, every UDT has a fully-qualified name that consists of the UDT name, +the package name where the UDT is defined, and the package version. The +fully-qualified name is used only when instances of the type are declared and +not where the type itself is defined. For example, assume package +<code>android.hardware.nfc,</code> version <code>1.0</code> defines a struct +named <code>NfcData</code>. At the site of the declaration (whether in +<code>types.hal</code> or within an interface's declaration), the declaration +simply states:</p> + +<pre class="prettyprint"> +struct NfcData { + vec<uint8_t> data; +}; +</pre> + +<p>When declaring an instance of this type (whether within a data structure or +as a method parameter), use the fully-qualified type name:</p> + +<pre class="prettyprint">android.hardware.nfc@1.0::NfcData</pre> + +<p>The general syntax is +<code><var>PACKAGE</var>@<var>VERSION</var>::<var>UDT</var></code>, where:</p> + +<ul> +<li><code><var>PACKAGE</var></code> is the dot-separated name of a HIDL package +(e.g., <code>android.hardware.nfc</code>).</li> +<li><code><var>VERSION</var></code> is the dot-separated major.minor-version +format of the package (e.g., <code>1.0</code>).</li> +<li><code><var>UDT</var></code> is the the dot-separated name of a HIDL UDT. +Since HIDL supports nested UDTs and HIDL interfaces can contain UDTs (a type of +nested declaration), dots are used to access the names.</li> +</ul> + +<p>For example, if the following nested declaration was defined in the common +types file in package <code>android.hardware.example</code> version +<code>1.0</code>:</p> + +<pre class="prettyprint"> +// types.hal +package android.hardware.example@1.0; +struct Foo { + struct Bar { + // … + }; + Bar cheers; +}; +</pre> + +<p>The fully-qualified name for <code>Bar</code> is +<code>android.hardware.example@1.0::Foo.Bar</code>. If, in addition to being in +the above package, the nested declaration were in an interface called +<code>IQuux</code>:</p> + +<pre class="prettyprint"> +// IQuux.hal +package android.hardware.example@1.0; +interface IQuux { + struct Foo { + struct Bar { + // … + }; + Bar cheers; + }; + doSomething(Foo f) generates (Foo.Bar fb); +}; +</pre> + +<p>The fully-qualified name for <code>Bar</code> is +<code>android.hardware.example@1.0::IQuux.Foo.Bar</code>.</p> + +<p>In both cases, <code>Bar</code> can be referred to as <code>Bar</code> only +within the scope of the declaration of <code>Foo</code>. At the package or +interface level, you must refer to <code>Bar</code> via <code>Foo</code>: +<code>Foo.Bar</code>, as in the declaration of method <code>doSomething</code> +above. Alternatively, you could declare the method more verbosely as:</p> + +<pre class="prettyprint"> +// IQuux.hal +doSomething(android.hardware.example@1.0::IQuux.Foo f) generates (android.hardware.example@1.0::IQuux.Foo.Bar fb); +</pre> + +<h2 id=enumeration>Fully-qualified enumeration values</h2> +<p>If a UDT is an enum type, then each value of the enum type has a +fully-qualified name that starts with the fully-qualified name of the enum type, +followed by a colon, then followed by the name of the enum value. For example, +assume package <code>android.hardware.nfc,</code> version <code>1.0</code> +defines an enum type <code>NfcStatus</code>:</p> + +<pre class="prettyprint"> +enum NfcStatus { + STATUS_OK, + STATUS_FAILED +}; +</pre> + +<p>When referring to <code>STATUS_OK</code>, the fully qualified name is:</p> + +<pre class="prettyprint">android.hardware.nfc@1.0::NfcStatus:STATUS_OK</pre> + +<p>The general syntax is +<code><var>PACKAGE</var>@<var>VERSION</var>::<var>UDT</var>:<var>VALUE</var></code>, +where: + +<ul> +<li><code><var>PACKAGE</var>@<var>VERSION</var>::<var>UDT</var></code> is the +exact same fully qualified name for the enum type.</li> +<li><code><var>VALUE</var></code> is the value's name.</li> +</ul> + +<h2 id=auto-interference>Auto-inference rules</h2> +<p>A fully-qualified UDT name does not need to be specified. A UDT name can +safely omit the following:</p> +<ul> +<li>The package, e.g. <code>@1.0::IFoo.Type</code></li> +<li>Both package and version, e.g. <code>IFoo.Type</code></li> +</ul> + +<aside class="caution"><strong>Caution:</strong> UDT names missing a version but +specifying a package present are not allowed.</aside> + +<p>HIDL attempts to complete the name using auto-interference rules (lower rule +number means higher priority).</p> + +<h3 id=rule1>Rule 1</h3> +<p>If no package and version is provided, a local name lookup is attempted. +Example:</p> + +<pre class="prettyprint"> +interface Nfc { + typedef string NfcErrorMessage; + send(NfcData d) generates (@1.0::NfcStatus s, NfcErrorMessage m); +}; +</pre> + +<p><code>NfcErrorMessage</code> is looked up locally, and the <code>typedef</code> +above it is found. <code>NfcData</code> is also looked up locally, but as it is +not defined locally, rule 2 and 3 are used. <code>@1.0::NfcStatus</code> +provides a version, so rule 1 does not apply.</p> + +<h3 id=rule2>Rule 2</h3> +<p>If rule 1 fails and a component of the fully-qualified name is missing +(package, version, or package and version), the component is autofilled with +information from the current package. The HIDL compiler then looks in the +current file (and all imports) to find the autofilled fully-qualified name. +Using the example above, assume the declaration of <code>ExtendedNfcData</code> +was made in the same package (<code>android.hardware.nfc</code>) at the same +version (<code>1.0</code>) as <code>NfcData</code>, as follows:</p> + +<pre class="prettyprint"> +struct ExtendedNfcData { + NfcData base; + // … additional members +}; +</pre> + +<p>The HIDL compiler fills out the package name and version name from the +current package to produce the fully-qualified UDT name +<code>android.hardware.nfc@1.0::NfcData</code>. As the name exists in the +current package (assuming it is imported properly), it is used for the +declaration.</p> + +<p>A name in the current package is imported only if one of the following is +true:</p> +<ul> +<li>It is imported explicitly with an <code>import</code> statement.</li> +<li>It is defined in <code>types.hal</code> in the current package</li> +</ul> + +<p>The same process is followed if <code>NfcData</code> was qualified by only +the version number:</p> + +<pre class="prettyprint"> +struct ExtendedNfcData { + // autofill the current package name (android.hardware.nfc) + @1.0::NfcData base; + // … additional members +}; +</pre> + +<h3 id=rule3>Rule 3</h3> +<p>If rule 2 fails to produce a match (the UDT is not defined in the current +package), the HIDL compiler scans for a match within all imported packages. +Using the above example, assume <code>ExtendedNfcData</code> is declared in +version <code>1.1</code> of package <code>android.hardware.nfc</code>, +<code>1.1</code> imports <code>1.0</code> as it should (see +<a href="#package-ext">Package-Level Extensions</a>), and the definition +specifies only the UDT name:</p> + +<pre class="prettyprint"> +struct ExtendedNfcData { + NfcData base; + // … additional members +}; +</pre> + +<p>The compiler looks for any UDT named <code>NfcData</code> and finds one in +<code>android.hardware.nfc</code> at version <code>1.0</code>, resulting in a +fully-qualified UDT of <code>android.hardware.nfc@1.0::NfcData</code>. If more +than one match is found for a given partially-qualified UDT, the HIDL compiler +throws an error.</p> + +<h3 id=rule-example>Example</h3> +<p>Using rule 2, an imported type defined in the current package is favored over +an imported type from another package:</p> + +<pre class="prettyprint"> +// hardware/interfaces/foo/1.0/types.hal +package android.hardware.foo@1.0; +struct S {}; + +// hardware/interfaces/foo/1.0/IFooCallback.hal +package android.hardware.foo@1.0; +interface IFooCallback {}; + +// hardware/interfaces/bar/1.0/types.hal +package android.hardware.bar@1.0; +typedef string S; + +// hardware/interfaces/bar/1.0/IFooCallback.hal +package android.hardware.bar@1.0; +interface IFooCallback {}; + +// hardware/interfaces/bar/1.0/IBar.hal +package android.hardware.bar@1.0; +import android.hardware.foo@1.0; +interface IBar { + baz1(S s); // android.hardware.bar@1.0::S + baz2(IFooCallback s); // android.hardware.foo@1.0::IFooCallback +}; +</pre> + +<ul> +<li><strong><code>S</code></strong> is interpolated as +<code>android.hardware.bar@1.0::S</code>, and is found in +<code>bar/1.0/types.hal</code> (because <code>types.hal</code> is automatically +imported).</li> +<li><strong><code>IFooCallback</code></strong> is interpolated as +<code>android.hardware.bar@1.0::IFooCallback</code> using rule 2, but it +cannot be found because <code>bar/1.0/IFooCallback.hal</code> is not imported +automatically (as <code>types.hal</code> is). Thus, rule 3 resolves it to +<code>android.hardware.foo@1.0::IFooCallback</code> instead, which is imported +via <code>import android.hardware.foo@1.0;</code>).</li> +</ul> + +<h2 id=types>types.hal</h2> +<p>Every HIDL package contains a <code>types.hal</code> file with UDTs share +among all interfaces participating in that package. HIDL types are always +public; regardless of whether a UDT is declared in <code>types.hal</code> or +within an interface declaration, these types are accessible outside of the scope +where they are defined. <code>types.hal</code> is not meant to describe the +public API of a package, but rather to host UDTs used by all interfaces within +the package. Due to the nature of HIDL, all UDTs are a part of the interface. +</p> + +<p><code>types.hal</code> consists of UDTs and <code>import</code> statements. +Because <code>types.hal</code> is made available to every interface of the +package (it is an implicit import), these <code>import</code> statements are +package-level by definition. UDTs in <code>types.hal</code> may also incorporate +UDTs and interfaces thus imported.</p> + +<p>For example, for an <code>IFoo.hal</code>:</p> + +<pre class="prettyprint"> +package android.hardware.foo@1.0; +// whole package import +import android.hardware.bar@1.0; +// types only import +import android.hardware.baz@1.0::types; +// partial imports +import android.hardware.qux@1.0::IQux.Quux; +// partial imports +import android.hardware.quuz@1.0::Quuz; +</pre> + +<p>The following are imported:</p> +<ul> +<li><code>android.hidl.base@1.0::IBase</code> (implicitly)</li> +<li><code>android.hardware.foo@1.0::types</code> (implicitly)</li> +<li>Everything in <code>android.hardware.bar@1.0</code> (including all +interfaces and its <code>types.hal</code>)</li> +<li><code>types.hal</code> from <code>android.hardware.baz@1.0::types</code> +(interfaces in <code>android.hardware.baz@1.0</code> are not imported)</li> +<li><code>IQux.hal</code> and <code>types.hal</code> from +<code>android.hardware.qux@1.0</code></li> +<li><code>Quuz</code> from <code>android.hardware.quuz@1.0</code> (assuming +<code>Quuz</code> is defined in <code>types.hal</code>, the entire +<code>types.hal</code> file is parsed, but types other than <code>Quuz</code> +are not imported).</li> +</ul> + +<h2 id=interface-version>Interface-level versioning</h2> +<p>Each interface within a package resides in its own file. The package the +interface belongs to is declared at the top of the interface using the +<code>package</code> statement. Following the package declaration, zero or more +interface-level imports (partial or whole-package) may be listed. For example: +</p> + +<pre class="prettyprint">package android.hardware.nfc@1.0;</pre> + +<p>In HIDL, interfaces can inherit from other interfaces using the +<code>extends</code> keyword. For an interface to extend another interface, it +must have access to it via an <code>import</code> statement. The name of the +interface being extended (the base interface) follows the rules for type-name +qualification explained above. An interface may inherit only from one interface; +HIDL does not support multiple inheritance.</p> + +<p>The uprev versioning examples below use the following package:</p> + +<pre class="prettyprint"> +// types.hal +package android.hardware.example@1.0 +struct Foo { + struct Bar { + vec<uint32_t> val; + }; +}; + +// IQuux.hal +package android.hardware.example@1.0 +interface IQuux { + fromFooToBar(Foo f) generates (Foo.Bar b); +} +</pre> + +<h3 id=rules>Uprev rules</h3> +<p>To define a package <code>package@major.minor</code>, either A or all of B +must be true:</p> + +<table> +<tr> +<th width="10%">Rule A</th> +<td>"Is a start minor version": All previous minor versions, +<code>package@major.0</code>, <code>package@major.1</code>, …, +<code>package@major.(minor-1)</code> must not be defined. +</td> +</tr> +</table> + +<strong>OR</strong> + +<table> +<tr> +<th width="10%">Rule B</th> +<td><p>All of the following is true:</p> + +<ol> +<li>"Previous minor version is valid": <code>package@major.(minor-1)</code> +must be defined and follow the same rule A (none of +<code>package@major.0</code> through <code>package@major.(major-2)</code> +are defined) or rule B (if it is an uprev from <code>@major.(major-2)</code>); +<br><br> +AND +<br><br> +</li> +<li>"Inherit at least one interface with the same name": There exists an +interface <code>package@major.minor::IFoo</code> that extends +<code>package@major.(minor-1)::IFoo</code>; +<br><br> +AND +<br><br> +</li> +<li>"No inherited interface with a different name": There must not exist +<code>package@major.minor::IBar</code> that extends +<code>package@major.(minor-1)::IBaz</code>, where <code>IBar</code> and +<code>IBaz</code> are two different names. If there is an interface with the +same name, <code>package@major.minor::IBar</code> must extend +<code>package@major.(minor-k)::IBar</code> such that no IBar exists with a +smaller k.</li> +</ol> +</td> +</tr> +</table> + +<p>Because of rule A:</p> +<ul> +<li>The package can start with any minor version number (for example, +<code>android.hardware.biometrics.fingerprint</code> starts at +<code>@2.1</code>.)</li> +<li>The requirement "<code>android.hardware.foo@1.0</code> is not defined" means +the directory <code>hardware/interfaces/foo/1.0</code> should not even exist. +</li> +</ul> + +<p>However, rule A does not affect a package with the same package name but a +different <em>major</em> version (for example, +<code>android.hardware.camera.device</code> has both <code>@1.0</code> and +<code>@3.2</code> defined; <code>@3.2</code> doesn't need to interact with +<code>@1.0</code>.) Hence, <code>@3.2::IExtFoo</code> can extend +<code>@1.0::IFoo</code>.</p> + +<p>Provided the package name is different, +<code>package@major.minor::IBar</code> may extend from an interface with a +different name (for example, <code>android.hardware.bar@1.0::IBar</code> can +extend <code>android.hardware.baz@2.2::IBaz</code>). If an interface does not +explicitly declare a super type with the <code>extend</code> keyword, it will +extend <code>android.hidl.base@1.0::IBase</code> (except <code>IBase</code> +itself).</p> + +<p>B.2 and B.3 must be followed at the same time. For example, even if +<code>android.hardware.foo@1.1::IFoo</code> extends +<code>android.hardware.foo@1.0::IFoo</code> to pass rule B.2, if an +<code>android.hardware.foo@1.1::IExtBar</code> extends +<code>android.hardware.foo@1.0::IBar</code>, this is still not a valid uprev. +</p> + +<h3 id=uprev>Upreving interfaces</h3> +<p>To uprev <code>android.hardware.example@1.0</code> (defined above) to +<code>@1.1</code>:</p> + +<pre class="prettyprint"> +// types.hal +package android.hardware.example@1.1; +<strong>import android.hardware.example@1.0;</strong> + +// IQuux.hal +package android.hardware.example@1.1 +interface IQuux <strong>extends @1.0::IQuux</strong> { + <strong>fromBarToFoo(Foo.Bar b) generates (Foo f);</strong> +} +</pre> + +<p>This is a package-level <code>import</code> of version <code>1.0</code> of +<code>android.hardware.example</code> in <code>types.hal</code>. While no new +UDTs are added in version <code>1.1</code> of the package, references to UDTs in +version <code>1.0</code> are still needed, hence the package-level import +in <code>types.hal</code>. (The same effect could have been achieved with an +interface-level import in <code>IQuux.hal</code>.)</p> + +<p>In <code>extends @1.0::IQuux</code> in the declaration of +<code>IQuux</code>, we specified the version of <code>IQuux</code> that is being +inherited (disambiguation is required because <code>IQuux</code> is used to +declare an interface and to inherit from an interface). As declarations are +simply names that inherit all package and version attributes at the site of the +declaration, the disambiguation must be in the name of the base interface; we +could have used the fully-qualified UDT as well, but that would have been +redundant.</p> + +<p>The new interface <code>IQuux</code> does not re-declare method +<code>fromFooToBar()</code> it inherits from <code>@1.0::IQuux</code>; it simply +lists the new method it adds <code>fromBarToFoo()</code>. In HIDL, inherited +methods may <strong>not</strong> be declared again in the child interfaces, so +for <code>IQuux</code> it would not be an option to declare +<code>fromFooToBar()</code> explicitly.</p> + +<aside class="key-point"><strong>Key Point:</strong> In HIDL, every inherited +method from a base class must be explicitly implemented in the inheriting class. +If a method implementation needs to fall back to that of the base, the fallback +must be in the implementation.</aside> + +<h3 id=conventions>Uprev conventions</h3> +<p>Sometimes interface names must rename the extending interface. We recommend +that enum extensions, structs, and unions have the same name as what they extend +unless they are sufficiently different to warrant a new name. Examples:</p> + +<pre class="prettyprint"> +// in parent hal file +enum Brightness : uint32_t { NONE, WHITE }; + +// in child hal file extending the existing set with additional similar values +enum Brightness : @1.0::Brightness { AUTOMATIC }; + +// extending the existing set with values that require a new, more descriptive name: +enum Color : @1.0::Brightness { HW_GREEN, RAINBOW }; +</pre> + +<p>Unless a method warrants a new name, it should be named similarly to what it +is extending. For example, the method <code>foo_1_1</code> in +<code>@1.1::IFoo</code> may replace the functionality of the <code>foo</code> +method in <code>@1.0::IFoo</code>.</p> + +<h2 id=package-ext>Package-level versioning</h2> +<p>HIDL versioning occurs at the package level; after a package is published, it +is immutable (its set of interfaces and UDTs cannot be changed). Packages can +relate to each other in several ways, all of which are expressible via a +combination of interface-level inheritance and building of UDTs by composition. +</p> + +<p>However, one type of relationship is strictly-defined and must be enforced: +<em>Package-level backwards-compatible inheritance</em>. In this scenario, the +<em>parent</em> package is the package being inherited from and the +<em>child</em> package is the one extending the parent. Package-level +backwards-compatible inheritance rules are as follows:</p> + +<ol> +<li>All interfaces of the parent package are inherited from by interfaces in the +child package.</li> +<li>All data types of the parent package are present in the new package and can +be handled by the (possibly reimplemented) methods from the old package.</li> +<li>New interfaces may also be added the new package (no restrictions about +relationships to other interfaces in other packages).</li> +<li>New data types may also be added for use by either new methods of uprev'ed +existing interfaces, or by new interfaces.</li> +</ol> + +<p>These rules can be implemented using HIDL interface-level inheritance and UDT +composition, but require meta-level knowledge to know these relationships +constitute a backwards-compatible package extension. This knowledge is inferred +as follows:</p> + +<aside class="key-point"><strong>Key Point:</strong> For package +<code>package</code> at version <code>major.minor</code>, if a +<code>package</code> exists at <code>major.(minor-1)</code>, then +<code>package@major.minor</code> is a minor uprev, and must follow the rules for +backwards-compatibility.</aside> + +<p>If a package meets this requirement, <code>hidl-gen</code> enforces +backwards-compatibility rules.</p> + + </body> +</html> diff --git a/en/devices/architecture/images/treble_configstore_design.png b/en/devices/architecture/images/treble_configstore_design.png Binary files differnew file mode 100644 index 00000000..c51f7ef2 --- /dev/null +++ b/en/devices/architecture/images/treble_configstore_design.png diff --git a/en/devices/architecture/images/treble_cpp_compiler_generated_files.png b/en/devices/architecture/images/treble_cpp_compiler_generated_files.png Binary files differnew file mode 100644 index 00000000..7d9de250 --- /dev/null +++ b/en/devices/architecture/images/treble_cpp_compiler_generated_files.png diff --git a/en/devices/architecture/images/treble_cpp_legacy_hal_progression.png b/en/devices/architecture/images/treble_cpp_legacy_hal_progression.png Binary files differnew file mode 100644 index 00000000..bf2bb65e --- /dev/null +++ b/en/devices/architecture/images/treble_cpp_legacy_hal_progression.png diff --git a/en/devices/architecture/images/treble_dto_appending.png b/en/devices/architecture/images/treble_dto_appending.png Binary files differnew file mode 100644 index 00000000..7c9fbfc4 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_appending.png diff --git a/en/devices/architecture/images/treble_dto_bootloader.png b/en/devices/architecture/images/treble_dto_bootloader.png Binary files differnew file mode 100644 index 00000000..eab12e26 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_bootloader.png diff --git a/en/devices/architecture/images/treble_dto_dtbo.png b/en/devices/architecture/images/treble_dto_dtbo.png Binary files differnew file mode 100644 index 00000000..7e4b4a88 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_ab.png b/en/devices/architecture/images/treble_dto_dtbo_ab.png Binary files differnew file mode 100644 index 00000000..64263d7c --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_ab.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_ab_1.png b/en/devices/architecture/images/treble_dto_dtbo_ab_1.png Binary files differnew file mode 100644 index 00000000..606b81b5 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_ab_1.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_ab_2.png b/en/devices/architecture/images/treble_dto_dtbo_ab_2.png Binary files differnew file mode 100644 index 00000000..307ff61c --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_ab_2.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_multiple.png b/en/devices/architecture/images/treble_dto_dtbo_multiple.png Binary files differnew file mode 100644 index 00000000..6fd63b99 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_multiple.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_partition_1.png b/en/devices/architecture/images/treble_dto_dtbo_partition_1.png Binary files differnew file mode 100644 index 00000000..58b5a20e --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_partition_1.png diff --git a/en/devices/architecture/images/treble_dto_dtbo_partition_2.png b/en/devices/architecture/images/treble_dto_dtbo_partition_2.png Binary files differnew file mode 100644 index 00000000..5b84c909 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_dtbo_partition_2.png diff --git a/en/devices/architecture/images/treble_dto_multiple_dt.png b/en/devices/architecture/images/treble_dto_multiple_dt.png Binary files differnew file mode 100644 index 00000000..038c2c97 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_multiple_dt.png diff --git a/en/devices/architecture/images/treble_dto_multiple_dt_runtime.png b/en/devices/architecture/images/treble_dto_multiple_dt_runtime.png Binary files differnew file mode 100644 index 00000000..0d47b4fb --- /dev/null +++ b/en/devices/architecture/images/treble_dto_multiple_dt_runtime.png diff --git a/en/devices/architecture/images/treble_dto_overriding.png b/en/devices/architecture/images/treble_dto_overriding.png Binary files differnew file mode 100644 index 00000000..d838cec4 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_overriding.png diff --git a/en/devices/architecture/images/treble_dto_partition_1.png b/en/devices/architecture/images/treble_dto_partition_1.png Binary files differnew file mode 100644 index 00000000..3585fa6a --- /dev/null +++ b/en/devices/architecture/images/treble_dto_partition_1.png diff --git a/en/devices/architecture/images/treble_dto_partition_2.png b/en/devices/architecture/images/treble_dto_partition_2.png Binary files differnew file mode 100644 index 00000000..fa53dde9 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_partition_2.png diff --git a/en/devices/architecture/images/treble_dto_partition_layout.png b/en/devices/architecture/images/treble_dto_partition_layout.png Binary files differnew file mode 100644 index 00000000..5519ab31 --- /dev/null +++ b/en/devices/architecture/images/treble_dto_partition_layout.png diff --git a/en/devices/architecture/images/treble_dto_simulate.png b/en/devices/architecture/images/treble_dto_simulate.png Binary files differnew file mode 100644 index 00000000..c61e05da --- /dev/null +++ b/en/devices/architecture/images/treble_dto_simulate.png diff --git a/en/devices/architecture/images/treble_kernel_current.png b/en/devices/architecture/images/treble_kernel_current.png Binary files differnew file mode 100644 index 00000000..5870b8b8 --- /dev/null +++ b/en/devices/architecture/images/treble_kernel_current.png diff --git a/en/devices/architecture/images/treble_kernel_treble.png b/en/devices/architecture/images/treble_kernel_treble.png Binary files differnew file mode 100644 index 00000000..3be5bb03 --- /dev/null +++ b/en/devices/architecture/images/treble_kernel_treble.png diff --git a/en/devices/architecture/images/treble_rs_bcc_plugin_new.png b/en/devices/architecture/images/treble_rs_bcc_plugin_new.png Binary files differnew file mode 100644 index 00000000..dc58bfdf --- /dev/null +++ b/en/devices/architecture/images/treble_rs_bcc_plugin_new.png diff --git a/en/devices/architecture/images/treble_rs_bcc_plugin_old.png b/en/devices/architecture/images/treble_rs_bcc_plugin_old.png Binary files differnew file mode 100644 index 00000000..b621ba39 --- /dev/null +++ b/en/devices/architecture/images/treble_rs_bcc_plugin_old.png diff --git a/en/devices/architecture/images/treble_rs_cpu_fallback.png b/en/devices/architecture/images/treble_rs_cpu_fallback.png Binary files differnew file mode 100644 index 00000000..b9fd07aa --- /dev/null +++ b/en/devices/architecture/images/treble_rs_cpu_fallback.png diff --git a/en/devices/architecture/images/treble_rs_gpu_fallback.png b/en/devices/architecture/images/treble_rs_gpu_fallback.png Binary files differnew file mode 100644 index 00000000..4fbcbedd --- /dev/null +++ b/en/devices/architecture/images/treble_rs_gpu_fallback.png diff --git a/en/devices/architecture/images/treble_rs_linking.png b/en/devices/architecture/images/treble_rs_linking.png Binary files differnew file mode 100644 index 00000000..984adeaa --- /dev/null +++ b/en/devices/architecture/images/treble_rs_linking.png diff --git a/en/devices/architecture/images/treble_rs_namespace.png b/en/devices/architecture/images/treble_rs_namespace.png Binary files differnew file mode 100644 index 00000000..f3568121 --- /dev/null +++ b/en/devices/architecture/images/treble_rs_namespace.png diff --git a/en/devices/architecture/images/treble_rs_vendor_driver.png b/en/devices/architecture/images/treble_rs_vendor_driver.png Binary files differnew file mode 100644 index 00000000..26861aa4 --- /dev/null +++ b/en/devices/architecture/images/treble_rs_vendor_driver.png diff --git a/en/devices/architecture/images/treble_vintf_avb_o_p.png b/en/devices/architecture/images/treble_vintf_avb_o_p.png Binary files differnew file mode 100644 index 00000000..a92bfbf4 --- /dev/null +++ b/en/devices/architecture/images/treble_vintf_avb_o_p.png diff --git a/en/devices/architecture/images/treble_vintf_avb_p.png b/en/devices/architecture/images/treble_vintf_avb_p.png Binary files differnew file mode 100644 index 00000000..82e6986d --- /dev/null +++ b/en/devices/architecture/images/treble_vintf_avb_p.png diff --git a/en/devices/architecture/images/treble_vintf_mm.png b/en/devices/architecture/images/treble_vintf_mm.png Binary files differnew file mode 100644 index 00000000..95339e1d --- /dev/null +++ b/en/devices/architecture/images/treble_vintf_mm.png diff --git a/en/devices/architecture/images/treble_vndk_design.png b/en/devices/architecture/images/treble_vndk_design.png Binary files differnew file mode 100644 index 00000000..596aea27 --- /dev/null +++ b/en/devices/architecture/images/treble_vndk_design.png diff --git a/en/devices/architecture/kernel/hardening.html b/en/devices/architecture/kernel/hardening.html new file mode 100644 index 00000000..28b6d87e --- /dev/null +++ b/en/devices/architecture/kernel/hardening.html @@ -0,0 +1,100 @@ +<html devsite> + <head> + <title>Kernel Hardening</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> +<p> +Android 8.0 added kernel hardening features to help mitigate kernel +vulnerabilities and find bugs in kernel drivers. The features are in <a +href="https://android.googlesource.com/kernel/common/">kernel/common</a> in +branches android-3.18, android-4.4, and android-4.9. +</p> +<h2 id="implementation">Implementation</h2> +<p> +To acquire these features, device manufacturers and SOCs should merge all +hardening patches from <code>kernel/common</code> to their kernel tree and +enable the following kernel configuration options: +</p> +<ul> + <li>Hardened usercopy: <code>CONFIG_HARDENED_USERCOPY=y</code></li> + <li>PAN emulation - arm64: <code>CONFIG_ARM64_SW_TTBR0_PAN=y</code></li> + <li>PAN emulation - arm: <code>CONFIG_CPU_SW_DOMAIN_PAN=y</code></li> + <li>KASLR - 4.4 and later kernels: + <code>CONFIG_RANDOMIZE_BASE=y</code></li> +</ul> +<p> +KASLR also requires bootloader support for passing hardware entropy through +either the device tree node <code>/chosen/kaslr-seed</code> or by implementing +<code>EFI_RNG_PROTOCOL</code>. +</p> +<p> +Also ensure existing hardening features are enabled: +</p> +<ul> + <li>Stack buffer overflow mitigation: + <code>CONFIG_CC_STACKPROTECTOR_STRONG=y</code></li> + <li>Internal memory protection: <code>CONFIG_DEBUG_RODATA=y</code> or + <code>CONFIG_STRICT_KERNEL_RWX=y</code></li> + <li>Restrict user-space access from kernel - x86 (enabled by default): + <code>CONFIG_X86_SMAP=y</code></li> +</ul> +<h2 id="testing">Testing</h2> +<p> +To test your implementation, add <code>CONFIG_LKDTM=y</code> to the kernel +configuration and confirm that each of the following commands lead to a kernel +panic: +</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="# ">echo ACCESS_USERSPACE > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_USERSPACE > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_RO > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_RO_AFTER_INIT > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo WRITE_KERN > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_STACK > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_RODATA > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_KMALLOC > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo EXEC_VMALLOC > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo CORRUPT_STACK > /sys/kernel/debug/provoke-crash/DIRECT</code> +</pre> +<p> +<strong>For android-4.9:</strong> +</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="# ">echo USERCOPY_HEAP_SIZE_TO > /sys/kernel/debug/provoke-crash/DIRECT</code> +<code class="devsite-terminal" data-terminal-prefix="# ">echo USERCOPY_HEAP_SIZE_FROM > /sys/kernel/debug/provoke-crash/DIRECT</code> +</pre> + +<h2 id="common-issues">Common issues</h2> +<p> +These changes are likely to expose bugs in kernel drivers, which need to be +fixed either by the device manufacturer or the owner of the kernel driver. +</p> +<ul> + <li>Hardened usercopy exposes incorrect bounds checking when copying data + to/from user space. These should be fixed like any other memory corruption bugs.</li> + <li>PAN emulation exposes direct user space access from the kernel, which is not + allowed. Drivers attempting to access user space memory need to be changed to + use the standard <code>copy_to_user()</code>/<code>copy_from_user()</code> + functions instead.</li> +</ul> +</body> +</html> diff --git a/en/devices/architecture/kernel/index.html b/en/devices/architecture/kernel/index.html new file mode 100644 index 00000000..456662f7 --- /dev/null +++ b/en/devices/architecture/kernel/index.html @@ -0,0 +1,80 @@ +<html devsite> + <head> + <title>Kernel</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>The Linux kernel is an extremely important part of the software on nearly +every Android device. This section describes Linux kernel development and +release models (below), +<a href="/devices/architecture/kernel/releases.html">stable and long-term +supported (LTS) kernels</a> (including why all Android devices should use stable +releases instead of cherry picking patches), kernel +<a href="/devices/architecture/kernel/config.html">configuration</a> and +<a href="/devices/architecture/kernel/hardening.html">hardening</a>, +requirements for +<a href="/devices/architecture/kernel/reqs-interfaces.html">interfaces</a> and +the +<a href="/devices/architecture/kernel/modular-kernels.html">modular kernels</a> +(introduced in Android O), kernel +<a href="/devices/architecture/kernel/lldb-debug.html">debugging</a> and +<a href="/devices/architecture/kernel/network_tests.html">network testing</a>, +and <a href="/devices/architecture/kernel/squashfs.html">SquashFS</a>.</p> + +<h2 id="linux-kernel-development">Linux kernel development</h2> +<p> +The Linux kernel is the largest collaborative software project ever. In 2016, +over 4,000 different developers from over 450 different companies contributed to +the project and there were 6 releases, each containing between 12,000 and 16,000 +different changes. At the end of 2016 the size of the Linux kernel was just over +56 thousand files, consisting of 22 million lines of code, build scripts, and +documentation (kernel release 4.9). (For full Linux development statistics, +refer to +<a href="https://kernelnewbies.org/DevelopmentStatistics" class="external">https://kernelnewbies.org/DevelopmentStatistics</a>.) +</p> +<p> +While the Linux kernel contains code for all the different chip architectures +and hardware drivers it supports, an individual system runs only a fraction of +the codebase. An average laptop uses around 2 million lines of kernel code from +5 thousand files to function properly, while the Pixel phone uses 3.2 million +lines of kernel code from 6 thousand files (due to the increased complexity of +an SoC). +</p> + +<h2 id="linux-kernel-releases">Linux kernel releases</h2> +<p>The Linux kernel uses a release model that differs substantially from +standard AOSP releases. With the release of the 2.6 kernel in December of 2003, +the kernel developer community switched from the previous model of having a +separate development and stable kernel branch, and moved to a <em>stable +only</em> branch model. In this model, a new release occurred every 2 to 3 +months, and that release was declared <em>stable</em> and recommended for all +users to run. This change in development model was due to the very long +release cycle prior to the 2.6 kernel (almost 3 years), and the struggle to +maintain two different branches of the codebase at the same time.</p> + +<p>The numbering of the kernel releases began at 2.6.x, where x was an +incrementing number that changed on every release (the value of the number has +no meaning, other than it is newer than the previous kernel release). The kernel +version since then has now moved to 4.x accounting for 2 major version changes. +These version numbers are chosen by the maintainer(s) only to avoid confusion +among users caused by higher minor release numbers.</p> + + </body> +</html> diff --git a/en/devices/architecture/kernel/lldb-debug.html b/en/devices/architecture/kernel/lldb-debug.html new file mode 100644 index 00000000..0c019fd5 --- /dev/null +++ b/en/devices/architecture/kernel/lldb-debug.html @@ -0,0 +1,110 @@ + <html devsite> + <head> + <title>Kernel Enhancements to LLDB/C++ Debugging</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p> +The Android 8.0 release includes kernel enhancements that help developers create +better applications by improving their debugging experience. +</p> +<p> +The arm64 Android kernels support setting watchpoints on memory addresses that +are not 4/8-byte aligned, and reporting all accesses to those addresses. +</p> +<h2 id="implementation">Implementation</h2> +<p> +The feature runs on any ARM 64-bit device. Adding the relevant support for +32-bit hardware/kernels is optional. All necessary kernel modifications have +already been done. +</p> +<p> +This feature is included in the current 3.10, 3.18, 4.4, and 4.9 kernel branches. +To add it to a kernel that does not already include it, cherry pick the +necessary CLs into your kernel build. Choose the patchset according to the +version your kernel is based on, as the patches required some adjustments due to +the kernel codebase evolving over time: +</p> + +<ul> + <li>branch android-3.10: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/bb42e87236d9defb511819991df21d464c1ef2eb">https://android.googlesource.com/kernel/common/+/bb42e87236d9defb511819991df21d464c1ef2eb</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/e8e0018306e1b9426dd70b44f057d1555c212ffb">https://android.googlesource.com/kernel/common/+/e8e0018306e1b9426dd70b44f057d1555c212ffb</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/fc0e9d5d77d9368ab15a06f7e61fff2ae2bc254d">https://android.googlesource.com/kernel/common/+/fc0e9d5d77d9368ab15a06f7e61fff2ae2bc254d</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/482ff2ca45737ee4bab3cf7221f6df1c9e14ea2c">https://android.googlesource.com/kernel/common/+/482ff2ca45737ee4bab3cf7221f6df1c9e14ea2c</a> + </ul> + + <li>branch android-3.18: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/b14849ae202487251b91e9f89af7eda484a80489">https://android.googlesource.com/kernel/common/+/b14849ae202487251b91e9f89af7eda484a80489</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/58a9c1be1966ca89facba6343a5d8781154196b7">https://android.googlesource.com/kernel/common/+/58a9c1be1966ca89facba6343a5d8781154196b7</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/d34d016794f345c5f1dccf4060b1132439098699">https://android.googlesource.com/kernel/common/+/d34d016794f345c5f1dccf4060b1132439098699</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/c82e0fdadfba89cdb50c96af622c2c5ad6cb55eb">https://android.googlesource.com/kernel/common/+/c82e0fdadfba89cdb50c96af622c2c5ad6cb55eb</a> + </ul> + + <li>branch android-4.4: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/91495bc622e9356dd63a7c77b98a21f0e2f5d2b2">https://android.googlesource.com/kernel/common/+/91495bc622e9356dd63a7c77b98a21f0e2f5d2b2</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/7409857a0717fa78dc936ea08099880be893156c">https://android.googlesource.com/kernel/common/+/7409857a0717fa78dc936ea08099880be893156c</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/b48318f371e8a8b98238deac868bc7af8ed8ba4b">https://android.googlesource.com/kernel/common/+/b48318f371e8a8b98238deac868bc7af8ed8ba4b</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/edc166a8714b012a3dd207e437c772ae2a264eca">https://android.googlesource.com/kernel/common/+/edc166a8714b012a3dd207e437c772ae2a264eca</a> + </ul> + + <li>branch android-4.9: + <ul> + <li><a + href="https://android.googlesource.com/kernel/common/+/85c450ac7ac0a69aca4c6eb845e419ae092d8c86">https://android.googlesource.com/kernel/common/+/85c450ac7ac0a69aca4c6eb845e419ae092d8c86</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/67de4de5628a47fa794920745dc7869c97b558d3">https://android.googlesource.com/kernel/common/+/67de4de5628a47fa794920745dc7869c97b558d3</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/32649b0a393303a97fab6003c5757ff7b100331a">https://android.googlesource.com/kernel/common/+/32649b0a393303a97fab6003c5757ff7b100331a</a> + <li><a + href="https://android.googlesource.com/kernel/common/+/d32793b6be517dffa29329a92ffcc688e74bc690">https://android.googlesource.com/kernel/common/+/d32793b6be517dffa29329a92ffcc688e74bc690</a></li> + </ul> +</ul> + +<p> +This feature implements an extension of the <code>ptrace</code> API to support +writing a debugger. +</p> +<h2 id="validation">Validation</h2> +<p> +There is a kernel selftest in the upstream repository, a CTS test, and there +are CTS tests added exercising the new kernel API: +</p> +<p> +<a +href="https://android.googlesource.com/platform/bionic/+/master/tests/sys_ptrace_test.cpp">https://android.googlesource.com/platform/bionic/+/master/tests/sys_ptrace_test.cpp</a> +</p> +</body> +</html> diff --git a/en/devices/architecture/kernel/modular-kernels.html b/en/devices/architecture/kernel/modular-kernels.html new file mode 100644 index 00000000..98ac0528 --- /dev/null +++ b/en/devices/architecture/kernel/modular-kernels.html @@ -0,0 +1,751 @@ +<html devsite> + <head> + <title>Modular Kernel Requirements</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>In Android O, the device kernel splits into System-on-Chip (SoC), device, and +board-specific deliverables. This sets up the kernel and Android such that +Original Device Manufacturers (ODMs) and Original Equipment Manufacturers (OEMs) +can work in isolated board–specific trees for board–specific features, drivers, +etc., enabling them to override common kernel configuration, add new drivers in +the form of kernel modules, etc.</p> + +<p>This page provides details on requirements for:</p> + +<ul> +<li>Platform support for independent SoC and OEM/ODM kernel development. Android +O recommends all board–specific code to be built and shipped as kernel modules +in devices. As a result: + <ul> + <li>All platforms should support either + <a href="https://www.devicetree.org/" class="external">Device Tree</a> or + <a href="http://www.uefi.org/acpi/specs" class="external">Advanced + Configuration and Power Interface (ACPI)</a> to describe all non-discoverable + devices.</li> + <li>For device tree-based platforms, board–specific device nodes should be + added to the kernel device tree as + <a href="/devices/architecture/dto/index.html">overlays</a>.</li> + </ul> +</li> +<li>Application binary interface (ABI)/application programming interface (API) +tests in <a href="/devices/tech/vts/index.html">Vendor Test Suite (VTS)</a> to +ensure a given kernel can run the Android Open Source Project (AOSP) +framework.</li> +<li>Minimum kernel version per Android release and support for generating +<a href="/devices/architecture/vintf/index.html">Android Vendor Interface +(VINTF) kernel objects</a>.</li> +</ul> + +<h2 id="loadable-kernel-modules">Loadable kernel modules</h2> +<p>All SoC kernels should support loadable kernel modules. As a starting point, +the following kernel-config options (or their kernel-version equivalent) have +been added to +<a href="https://android.googlesource.com/kernel/common/+/android-4.4/android/configs/android-base.cfg" class="external">android-base.cfg</a> +in all common kernels and must be enabled in all device kernels:</p> + +<pre class="prettyprint"> +CONFIG_MODULES=y +CONFIG_MODULE_UNLOAD=y +CONFIG_MODVERSIONS=y +</pre> + +<p>All kernel modules are subject to module load/unload testing to ensure the +correctness of the driver/module.</p> + +<aside class="note"><strong>Note:</strong> +<code>CONFIG_MODULE_SRCVERSION_ALL</code> is optional and will not be tested +against.</aside> + +<h3 id="module-signing">Module signing</h3> +<p>Optionally, ODMs can enable module signing in their own kernel configuration +by enabling following kernel config options:</p> + +<pre class="prettyprint"> +CONFIG_MODULE_SIG=y +CONFIG_MODULE_SIG_FORCE=y +</pre> + +<p>On devices required to support verified boot, Android requires the kernel +modules to be in the partitions that have dm-verity enabled. Module signing is +not mandatory and will not be tested against; however, if desired, an ODM can +enable module signing as long as they have the key signing and other +infrastructure required to ensure independent kernel and filesystem OTA updates +in the future.</p> + +<h3 id="file-locations">File locations</h3> +<p>While Android 7.x and earlier do not mandate against kernel modules (and +include support for <code>insmod</code> and <code>rmmod</code>), Android O +recommends the use of kernel modules in the ecosystem. The following table shows +potential board–specific peripheral support required across three Android boot +modes:</p> + +<table> +<tr> +<th>Boot Mode</th> +<th>Storage</th> +<th>Display</th> +<th>Keypad</th> +<th>Battery</th> +<th>PMIC</th> +<th>Touchscreen</th> +<th>NFC, Wi-Fi,<br>Bluetooth</th> +<th>Sensors</th> +<th>Camera</th> +</tr> +<tr> +<td>Recovery</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td>> +<td><span class="compare-no"></span></td> +</tr> +<tr> +<td>Charger</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +<td><span class="compare-no"></span></td> +</tr> +<tr> +<td>Android</td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +<td><span class="compare-yes"></span></td> +</tr> +</table> + +<p>In addition to availability in Android boot modes, kernel modules may also be +categorized by who owns them (the SoC vendor or the ODM). If kernel modules are +being used, requirements for their placement in filesystem are as follows:</p> + +<ul> +<li>All kernels should have built-in support for booting and mounting partitions. +</li> +<li>Kernel modules should be loaded from a read-only partition.</li> +<li>For devices required to have verified boot, kernel modules should be loaded +from verified partitions.</li> +<li>Kernel modules should not be located in <code>/system</code>.</li> +<li>Kernel modules from the SoC vendor that are required for full Android or +Charger modes should be located in <code>/vendor/lib/modules</code>.</li> +<li>If an ODM partition exists, kernel modules from the ODM that are required +for full Android or Charger modes should be located in +<code>/odm/lib/modules</code>. Otherwise, these modules should be located in +<code>/vendor/lib/modules</code>.</li> +<li>Kernel modules from both the SoC vendor and ODM that are required for +Recovery mode should be located in the recovery <code>ramfs</code> at +<code>/lib/modules</code>.</li> +<li>If a kernel module is required for both Recovery mode and full Android or +Charger modes, it should exist both in the recovery <code>rootfs</code> and +either the <code>/vendor</code> or <code>/odm<strong> </strong></code>partitions +(as described above).</li> +<li>Kernel modules used in Recovery mode should not depend on modules located +only in <code>/vendor</code> or <code>/odm</code>, as those partitions are not +mounted in Recovery mode.</li> +<li>SoC vendor kernel modules should not depend on ODM kernel modules.</li> +</ul> + +<p>In Android 7.x and earlier, <code>/vendor</code> and <code>/odm</code> +partitions are <strong>not</strong> mounted early. In Android O, to make module +loading from these partitions possible, provisions have been made to mount +partitions early for both +<a href="https://source.android.com/devices/tech/ota/ab_updates">non-A/B and A/B +devices</a>. This also ensures the partitions are mounted in both Android and +Charger modes.</p> + +<h3 id="android-build-system-support">Android build system support</h3> +<p>In <code>BoardConfig.mk</code>, the Android build defines a +<code>BOARD_VENDOR_KERNEL_MODULES</code> variable that provides a full list of +the kernel modules intended for the vendor image. The modules listed in this +variable are copied into the vendor image at <code>/lib/modules/</code>, and, +after being mounted in Android, appear in <code>/vendor/lib/modules</code> (in +accordance with the above requirements). Example configuration of the vendor +kernel modules:</p> + +<pre class="prettyprint"> +vendor_lkm_dir := device/$(vendor)/lkm-4.x +BOARD_VENDOR_KERNEL_MODULES := \ + $(vendor_lkm_dir)/vendor_module_a.ko \ + $(vendor_lkm_dir)/vendor_module_b.ko \ + $(vendor_lkm_dir)/vendor_module_c.ko +</pre> + +<p>… where a vendor kernel module pre-built repository is mapped into the +Android build at the location listed above.</p> + +<p>The recovery image is likely to contain a subset of the vendor modules. The +Android build defines the variable <code>BOARD_RECOVERY_KERNEL_MODULES</code> +for these modules. Example:</p> + +<pre class="prettyprint"> +vendor_lkm_dir := device/$(vendor)/lkm-4.x +BOARD_RECOVERY_KERNEL_MODULES := \ + $(vendor_lkm_dir)/vendor_module_a.ko \ + $(vendor_lkm_dir)/vendor_module_b.ko +</pre> + +<p>The Android build takes care of running <code>depmod</code> to generate the +required <code>modules.dep</code> files in <code>/vendor/lib/modules</code> and +<code>/lib/modules</code> (<code>recovery ramfs</code>).</p> + +<h3 id="module-loading-&-versioning">Module loading & versioning</h3> +<p>We recommend loading all kernel modules in one pass from +<code>init.rc*</code> by invoking <code>modprobe -a</code>. This avoids the +overhead of repeatedly initializing the C runtime environment for the +<code>modprobe</code> binary. The <code>early-init</code> event can be modified +to invoke <code>modprobe</code>:</p> + +<pre class="prettyprint"> +on early-init + exec u:r:modprobe:s0 -- /vendor/bin/modprobe -a -d \ + /vendor/lib/modules module_a module_b module_c ... +</pre> + +<p>The kernel image may be updated separately from the vendor image, meaning +that kernel modules may be used with kernels other than the one they were +originally compiled against. To allow for this, and to protect against ABI +breakages, module versioning is used. Module versioning is enabled by +<code>CONFIG_MODVERSIONS=y</code> (one of the required kernel configuration +options mentioned above) and is documented in the kernel tree at +<code>Documentation/kbuild/modules.txt</code>.</p> + +<h2 id="mounting-partitions-early-first-stage-mount">Mounting partitions early +(first stage mount)</h2> +<font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: green; color: white"> REQUIRED </font> +<p>All Treble-enabled devices must enable first stage mount to make sure +<code>init</code> can load SELinux policy fragments that are spread across +<code>system</code> and <code>vendor</code> partitions (this also enables +loading of kernel modules as soon as possible after kernel boot).</p> + +<aside class="note"><strong>Note:</strong> For details on SELinux in Android +8.0, see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for +Android 8.0</a>.</aside> + +<p>Android must have access to the filesystem(s) on which the modules reside. To +enable, Android O supports mounting <code>/system</code>, <code>/vendor</code>, +or <code>/odm</code> as early as <code>init</code>'s first stage (i.e before +selinux is initialized). Device makers can use +<a href="/devices/architecture/dto/index.html">device tree overlays</a> to +specify <code>fstab</code> entries for early mounted partitions.</p> + +<aside class="key-point"><strong>Summary of AOSP early mount changes:</strong> +<ul> +<li> +<a href="https://android-review.googlesource.com/#/q/topic:pre-early-mount" class="external">Pre-early +mount v1</a> & +<a href="https://android-review.googlesource.com/#/q/status:merged+project:platform/system/core+branch:master+topic:pre-early-mount-2" class="external">v2</a></li> +<li><a href="https://android-review.googlesource.com/#/c/339471/" class="external">Miscellaneous +partition lookup removal</a></li> +<li> +<a href="https://android-review.googlesource.com/#/q/branch:master+topic:early-mount-support" class="external">Early +mount support</a></li> +<li> +<a href="https://android-review.googlesource.com/#/q/branch:master+topic:early-mount-support" class="external">Early +mount with VBoot 2.0 (AVB)</a></li> +</ul> +</aside> + +<h3 id="early-mounting-partitions-vboot-1-0">Early mounting partitions, VBoot +1.0</h3> +<p>Requirements to early mount partitions with vboot 1.0 include:</p> +<ol> +<li>Device node paths must use their <em>by-name</em> symlinks in +<code>fstab</code> and device tree entries. For example, instead of specifying +partitions using <code>/dev/block/mmcblk0pX</code>, ensure partitions are named +and the device node is <code>/dev/block/…./by-name/{system,vendor,odm}</code>. +</li> +<li>Paths given for <code>PRODUCT_{SYSTEM,VENDOR}_VERITY_PARTITION</code> and +<code>CUSTOM_IMAGE_VERITY_BLOCK_DEVICE</code> in the device configuration for +the product (i.e. in +<code>device/<em>oem</em>/<em>project</em>/device.mk</code>) must match the +corresponding block device nodes specified <em>by-name</em> in the +<code>fstab</code>/device tree entries. Example: +<pre class="prettyprint"> +PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/…./by-name/system +PRODUCT_VENDOR_VERITY_PARTITION := /dev/block/…./by-name/vendor +CUSTOM_IMAGE_VERITY_BLOCK_DEVICE := /dev/block/…./by-name/odm +</pre> +</li> +<li>Entries provided via device tree overlays must not repeat in the +<code>fstab</code> file fragments. For example, when specifying an entry to +mount <code>/vendor</code> in the device tree, the <code>fstab</code> file must +not repeat that entry.</li> +<li>Only <code>/system</code>, <code>/odm</code>, or <code>/vendor</code> can be +mounted early. Android does not include support to mount any other partitions in +<code>init</code> first stage.</li> +<li>Partitions requiring <code>verifyatboot</code> <strong>must not</strong> be +early mounted (doing so is unsupported).</li> +<li>The verity mode/state for verified partitions must be specified in kernel +cmdline using <code>androidboot.veritymode</code> option (existing +requirement).</li> +</ol> + +<h3 id="early-mounting-device-tree-vboot-1-0">Early mounting device tree, VBoot +1.0</h3> +<p>In Android O, <code>init</code> parses the device tree and creates +<code>fstab</code> entries to mount the partition early during its first stage. +An fstab entry takes the form:</p> + +<pre class="prettyprint">src mnt_point type mnt_flags fs_mgr_flags</pre> + +<p>Device tree properties are defined to mimic that format:</p> + +<ul> +<li><code>fstab</code>entries must be under <code>/firmware/android/fstab</code> +in the device tree and must have compatible string set to +<code>android,fstab</code>. </li> +<li>Each node under <code>/firmware/android/fstab</code> is treated as a single +early mount <code>fstab</code> entry. A node must have the following properties +defined: + <ul> + <li><code>dev</code>. Must point to the device node representing the + partition <em>by-name</em>.</li> + <li><code>type</code>. Must be the filesystem type (as in the + <code>fstab</code> files).</li> + <li><code>mnt_flags</code>. Must be the comma-separated list of mount + flags (as in <code>fstab</code> files).</li> + <li><code>fsmgr_flags</code>. Must be the list of Android <code>fs_mgr + flags</code> (as in <code>fstab</code> files). + <ul> + <li>A/B partitions must have <code>slotselect fs_mgr</code>option.</li> + <li>dm-verity enabled partitions must have <code>verify fs_mgr</code> option. + </li> + </ul> + </li> + </ul> +</li> +</ul> + +<h4><strong>Example:</strong> /system and /vendor on N6P</h4> +<p>The following example shows device tree early mount for <code>system</code> +and <code>vendor</code> partitions on Nexus 6P:</p> + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + system { + compatible = "android,system"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/system"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait,verify"; + }; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait"; + }; + }; + }; + }; +}; +</pre> + +<h4><strong>Example:</strong> /vendor on Pixel</h4> +<p>The following example shows device tree early mount for +<code>/vendor</code> on Pixel (remember to add <code>slotselect</code> for +partitions subject to A/B):</p> + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc/624000.ufshc/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,discard"; + fsmgr_flags = "wait,slotselect,verify"; + }; + }; + }; + }; +}; +</pre> + +<h3 id="early-mounting-partitions-vboot-2-0">Early mounting partitions, VBoot +2.0</h3> +<p>VBoot 2.0 is +<a href="https://android.googlesource.com/platform/external/avb/" class="external">Android +Verified Boot (AVB).</a> The requirements to early mount partitions with VBoot +2.0 are:</p> +<ol> +<li>The device node paths must use their <em>by-name</em> symlinks in +<code>fstab</code> and device tree entries. For example, instead of specifying +partitions using <code>/dev/block/mmcblk0pX</code>, ensure the partitions are +named and the device node is +<code>/dev/block/…./by-name/{system,vendor,odm}</code>.</li> +<li>Build system variables (such as +<code>PRODUCT_{SYSTEM,VENDOR}_VERITY_PARTITION</code> and +<code>CUSTOM_IMAGE_VERITY_BLOCK_DEVICE</code>) used for VBoot 1.0 are NOT +required for VBoot 2.0. Instead, new build variables introduced in VBoot 2.0 +(including <code>BOARD_AVB_ENABLE := true</code>) should be defined; for a full +configuration, refer to +<a href="https://android.googlesource.com/platform/external/avb/#Build-System-Integration" class="external">Build-System-Integration +for AVB</a>.</li> +<li>Entries provided via device tree overlays must not repeat in the +<code>fstab</code> file fragments. For example, if you specify an entry to mount +<code>/vendor</code> in the device tree, the <code>fstab</code> file must not +repeat that entry.</li> +<li>Only <code>/system</code>, <code>/odm</code>, or <code>/vendor</code> can be +mounted early. Android does not include support to mount any other partitions in +<code>init</code> first stage.</li> +<li>VBoot 2.0 does not support <code>verifyatboot</code>, regardless of whether +early mount is enabled or not.</li> +<li>The verity mode/state for verified partitions must be specified in kernel +cmdline using <code>androidboot.veritymode</code> option (existing requirement). +Make sure to include the following fixes for AVB: + <ul> + <li> + <a href="https://android-review.googlesource.com/#/q/topic:libavb-api-rev-for-verity-modes+(status:open+OR+status:merged)" class="external">https://android-review.googlesource.com/#/q/topic:libavb-api-rev-for-verity-modes+(status:open+OR+status:merged)</a></li> + <li> + <a href="https://android-review.googlesource.com/#/c/394215/" class="external">https://android-review.googlesource.com/#/c/394215/</a></li> + </ul> +</li> +</ol> + +<h3 id="early-mounting-device-tree-vboot-2-0">Early mounting device tree, VBoot +2.0</h3> +<p>The configuration in device tree for VBoot 2.0 is the same as that in +<a href="#early-mounting-device-tree-vboot-1-0">VBoot 1.0</a>, with the +following exceptions:</p> +<ul> +<li>The <code>fsmgr_flag</code> is switched from <code>verify</code> to +<code>avb</code>.</li> +<li>All partitions with AVB metadata must be in the vbmeta entry in device tree, +even when the partition isn't mounting early (e.g., <code>/boot</code>).</li> +</ul> + +<h4><strong>Example:</strong> /system and /vendor on N5X</h4> +<p>The following example shows device tree early mount for <code>system</code> +and <code>vendor</code> partitions on Nexus 5X. Note that:</p> +<ul> +<li><code>/system</code> is mounted with AVB and <code>/vendor</code> is mounted +without integrity verification.</li> +<li>As the Nexus 5X has no <code>/vbmeta</code> partition, so the top-level +vbmeta resides at the end of <code>/boot</code> partition (for details, refer to +the +<a href="https://android-review.googlesource.com/#/c/344907/" class="external">AOSP +changelist</a>). + +<pre class="prettyprint"> +/ { + firmware { + android { + compatible = "android,firmware"; + vbmeta { + compatible = "android,vbmeta"; + parts = "boot,system,vendor"; + }; + fstab { + compatible = "android,fstab"; + system { + compatible = "android,system"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/system"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait,avb"; + }; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc.0/f9824900.sdhci/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,inode_readahead_blks=8"; + fsmgr_flags = "wait"; + }; + }; + }; + }; +}; +</pre> +</li> +</ul> + +<h4><strong>Example:</strong> /vendor on Pixel</h4> +<p>The following example shows mounting <code>/vendor</code> early on a Pixel. +Note that:</p> + +<ul> +<li>More partitions are specified in the vbmeta entry because those partitions +are +<a href="https://android.googlesource.com/platform/external/avb/#The-VBMeta-struct">protected +by AVB.</a></li> +<li>All AVB partitions must be included, even if only <code>/vendor</code> is +early mounted.</li> +<li>Remember to add <strong><code>slotselect</code></strong> for partitions +subject to A/B. +<pre class="prettyprint"> +/ { + vbmeta { + compatible = "android,vbmeta"; + parts = "vbmeta,boot,system,vendor,dtbo"; + }; + firmware { + android { + compatible = "android,firmware"; + fstab { + compatible = "android,fstab"; + vendor { + compatible = "android,vendor"; + dev = "/dev/block/platform/soc/624000.ufshc/by-name/vendor"; + type = "ext4"; + mnt_flags = "ro,barrier=1,discard"; + fsmgr_flags = "wait,slotselect,avb"; + }; + }; + }; + }; +}; +</pre> +</li> +</ul> + +<h2 id="device-tree-overlay-support-bootloader">Device tree overlay support +(Bootloader)</h2> +<p> +<a href="https://lkml.org/lkml/2012/11/5/615">Device Tree Overlay</a> (DTO) was +designed to extend the existing +<a href="https://events.linuxfoundation.org/sites/events/files/slides/petazzoni-device-tree-dummies.pdf" class="external">flattened +device-tree (FDT)</a> implementation so that the initial device-tree data in +kernel can be modified by userspace at runtime by loading additional overlay +FDTs that amend the original data. Android does not require runtime updates of +DT blobs from user space, but instead recommends that vendors add the device +tree patching in the bootloader with the help of +<code>libfdt</code>/<code>libufdt</code>.</p> + +<p>In Android 7.x and earlier, Android did not require device tree support and +did not provide recommendations regarding how vendors pass DT blobs to the +kernel or where they store them. Android O recommends such support to keep the +board–specific and SoC-only parts of the kernel separate.</p> + +<h3 id="partitioning-requirements">Partitioning requirements</h3> +<p>Most Android devices today append the DT blob to the kernel at build time, +which the bootloader knows how to read from. As Android has no specific +requirements for how to build/store DT blobs (which is considered as part of the +SoC kernel), the DT blob can be appended to the kernel or stored in a separate +partition. The only assumption is that the bootloader already knows how and +where to load the DT blob from.</p> + +<p>Requirements for Device Tree Overlay support (if used):</p> +<ul> +<li>Device should have new device tree blob for overlay (DTBO) partion per +kernel image for board–specific DT overlay (for details on the partition format, +see <a href="/devices/architecture/dto/partitions.html">DTB/DTBO Partitions</a>. +The assumption is that bootloader already knows where and how to load the +SoC–specific DTB.</li> +<li>Overlay DT partition should be +<a href="https://source.android.com/devices/tech/ota/ab_updates.html">A/B-ed</a> +for A/B devices. For these devices, the recovery kernel is the same as Android +kernel, but the partition must be A/B-ed as it can be updated via OTA.</li> +<li>Partition size is board–specific. + <ul> + <li>The DT overlay partition size depends on the device and the amount of + changes needed on top of the main SoC kernel DT blob.</li> + <li>The size of the DTBO partition is a function of number of changes needed to + make the SoC kernel. Choose a size with room to grow for future updates + (typically, 8MB partition size is more than enough).</li> + </ul> +</li> +</ul> + +<h3 id="bootloader-requirements">Bootloader requirements</h3> +<p>Requirements for bootloader include the following:</p> +<ul> +<li>Bootloader should know how and where (considering the boot slot for A/B +devices) to load the SoC–specific DT blob from in a vendor-specific way. This is +typically extracted from the end of the kernel image as blobs are appended to +the kernel.</li> +<li>Bootloader should know how and where (considering the boot slot for A/B +devices) to load the overlay DT blob from in a vendor-specific way.</li> +<li>Bootloader must patch the main DT blob with the overlay before passing the +combined device tree to the kernel.</li> +</ul> + +<p>For more details about adding support for DTO in bootloader, see +<a href="/devices/architecture/dto/index.html">Device Tree Overlays</a>.</p> + +<h2 id="core-kernel-requirements">Core kernel requirements</h2> +<p>Android O mandates a minimum kernel version and kernel configuration and +checks them both in VTS as well as during an OTA. Android device kernels must +enable the kernel <code>.config</code> support along with the option to read the +kernel configuration at runtime through <code>procfs</code>.</p> + +<h3 id="kernel-config-support">Kernel .config support</h3> +<p>All device kernels must enable the entirety of +<a href="https://android.googlesource.com/kernel/common/+/android-4.4/android/configs/android-base.cfg" class="external">android-base.cfg</a>, +which must include the following kernel–config options (or their kernel–version +equivalent):</p> + +<pre class="prettyprint"> +CONFIG_IKCONFIG=y +CONFIG_IKCONFIG_PROC=y +</pre> + +<h3 id="kernel-version">Kernel version</h3> +<p>Kernel version requirements:</p> +<ul> +<li>All SoCs productized in 2017 must launch with kernel 4.4 or newer.</li> +<li>All other SoCs launching new Android devices running Android O must use +kernel 3.18 or newer.</li> +<li>Regardless of launch date, all SoCs with device launches on Android O remain +subject to kernel changes required to enable Treble.</li> +<li>Older Android devices released prior to Android O but that will be upgraded +to Android O can continue to use their original base kernel version if +desired.</li> +</ul> + +<h3 id="device-tree-support">Device tree support</h3> +<p>Device tree support in the kernel must be enabled and bootloaders must pass +the hardware description in the form of device tree to the kernel (unless the +platform supports ACPI). The device tree must also be available for Android to +read and be able to pass vendor/odm specific parameters to Android. +<code>CONFIG_OF</code> (along with all other device/subsystem specific +<code>CONFIG_OF_*</code> kernel config options) are mandatory.</p> + +<p><code>CONFIG_PROC_DEVICETREE</code> is required on kernels prior to 3.15 so +Android can access vendor/odm specific configuration very early during boot. On +kernels 3.15 and later, the functionality of this option is merged into +<code>CONFIG_OF</code>.</p> + +<pre class="prettyprint"> +CONFIG_OF=y +CONFIG_PROC_DEVICETREE=y (kernels prior to 3.15) +</pre> + +<p>For an example of using device tree to early mount +<code>vendor</code>/<code>odm</code> partitions, refer to the +<a href="https://android-review.googlesource.com/#/c/337310/" class="external">AOSP +changelist</a>.</p> + +<h3 id="debugfs">DebugFS</h3> +<p>The implementation of the vendor interface should not rely on +<code>debugfs</code>. It may be enabled, but VTS testing may be done with +<code>debugfs</code> unmounted.</p> + +<h2 id="beyond-android-o">Beyond Android O</h2> +<p>Android O recommends any board–specific kernel functionality to be in the +form of loadable kernel modules and device–tree overlays. The rest of the kernel +is treated monolithically with respect to Android (whether or not is it is +actually a monolithic kernel, or parts of it are compiled as kernel modules). +</p> + +<p>This monolithic kernel is an SoC kernel that can boot on the SoC vendor's +reference hardware but nothing beyond that. Today, SoC kernels are treated +similar to the common kernel; they are also heavily replicated in board–specific +repos. This distribution model causes them to be fixed differently for the same +bug in each branch, delaying future updates to the kernel due to cherry–picking +at different times or fixing the same bug differently. To counter this, the SoC +kernels must be a separate deliverable, with everyone who uses the SoC +contributing to the same SoC kernel.</p> + +<p>Figure 1 (below) illustrates a common example of how SoC kernels get +fragmented over time, across Android releases, and across ODMs.</p> + +<img src="../images/treble_kernel_current.png"> +<figcaption><strong>Figure 1.</strong> Device kernel replication.</figcaption> + +<p>Figure 1 shows the following:</p> +<ol> +<li>It takes a significant amount of effort and time for everyone to cross-merge +across board–specific branches/tags.</li> +<li>While waiting for the cross-merge, Android device manufacturers patch their +own kernel for bugs/security fixes.</li> +<li>Divergence from the ancestor make future upgrades/merges really +difficult.</li> +</ol> +<p>The proposed model for a common SoC kernel addresses problems created by +upmerging changes (SoC-specific bug fixes, LTS upgrades, security fixes, etc.). +Figure 2 (below) illustrates how the workflow will change in an ideal, +unified–per–SoC–kernel scenario:</p> + +<img src="../images/treble_kernel_treble.png"> +<figcaption><strong>Figure 2.</strong> Android O and higher device +kernels.</figcaption> + +<p>This is intended to solve the problem of fragmented kernel repos by +recommending and working with device manufacturers to stay up to date with the +common SoC kernel. Android O provides all possible options to ODMs to help them +avoid maintaining their own SoC kernels and instead rely on the common SoC +kernel for LTS upgrades/bug fixes/security patches/etc.</p> + +<p>As a start, we want to facilitate all ODMs/vendors using a single kernel +source for an SoC. In the future, we want to move towards a single binary +distribution of kernel per-SoC.</p> + +<h3 id="upstreaming">Upstreaming</h3> +<p>To make updating to newer kernel versions much easier and almost automatic, +and to provide a more secure and reliable platform for ODMs to build a product +with, it is strongly recommended that SoC vendors work to upstream their kernel +changes and get them accepted into the main kernel.org repository. While doing +so requires additional, up front efforts in time and engineering resources, it +is well documented to save both time and money in the long run. It has also been +documented that merged code is of a much higher quality with fewer bugs and +security issues (and usually smaller) than code that has not been reviewed by +the community.</p> + +<p>If full support for the SoC is merged upstream, the community can make needed +API changes as the internal kernel API evolves over time, automatically +extending the longevity of the platform. The kernel can also be automatically +tested for any regressions in development and stable releases by adding the +hardware platform to one of the many community-managed kernel test platforms +(such as +<code><a href="https://kernelci.org" class="external">kernelci.org</a></code>). +</p> + +<p>For help working with the Linux kernel community to upstream your code, refer +to the following resources:</p> +<ul> +<li><code>Documentation/process</code> +(<code>Documentation/development-process</code> in 4.9 and earlier)</li> +<li><code>Documentation/CodingStyle</code></li> +<li><code>Documentation/SubmittingPatches</code></li> +</ul> + +<p>The community uses a minimal review process to accept stand-alone drivers and +filesystems into the staging portion of the kernel, where the community works to +improve code quality.</p> + + </body> +</html> diff --git a/en/devices/architecture/kernel/releases.html b/en/devices/architecture/kernel/releases.html new file mode 100644 index 00000000..612922d9 --- /dev/null +++ b/en/devices/architecture/kernel/releases.html @@ -0,0 +1,180 @@ +<html devsite> + <head> + <title>Stable Kernel Releases & Updates</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p> +The Linux kernel stable release model started in 2005, when it was determined +that the existing kernel development model (a new release every 2-3 months) was +not meeting the needs of most users. Users wanted bugfixes made during those 2-3 +months, and Linux distributions found it difficult to keep kernels up to date +without feedback from the kernel community. In general, attempts to keep +individual kernels secure and with the latest bugfixes was a large and confusing +effort by lots of different individuals. +</p> +<p> +Stable kernel releases are based directly on Linus Torvalds' releases, and are +released every week or so, depending on various external factors (time of year, +available patches, maintainer workload, etc.). The numbering of the stable +releases starts with the number of the kernel release, and an additional number +is added to the end of it. For example, the 4.4 kernel is released by Linus, and +then the stable kernel releases based on this kernel are numbered 4.4.1, 4.4.2, +4.4.3, and so on. This sequence is usually shortened with the number 4.4.y when +referring to a stable kernel release tree. Each stable kernel release tree is +maintained by a single kernel developer, who is responsible for picking the +needed patches for the release and managing the review/release process. +</p> +<p> +Stable kernels are maintained for the length of the current development cycle. +After Linus releases a new kernel, the previous stable kernel release tree is +stopped and users must move to the newer released kernel. +</p> + +<h2 id="long-term-stable-kernels">Long-term stable kernels</h2> +<p> +After a year of this new stable release process, it was determined that many +different users of Linux wanted a kernel to be supported for longer than just a +few months. In response, the Long Term Supported (LTS) kernel release was +created, with the first LTS kernel (2.6.16) released in 2006. Since then, a new +LTS kernel has been selected once a year and kernel community maintains that +kernel for a minimum of 2 years. +</p> +<p>At the time of this writing, the LTS kernels are the 4.4.y and 4.9.y +releases, and a new kernel is released weekly. Due to the needs of some users +and distributions, a few additional older kernels are maintained by kernel +developers at a slower release cycle. Information about all long-term stable +kernels, who is in charge of them, and how long they will be maintained, can be +found on the +<a href="https://www.kernel.org/category/releases.html" class="external">kernel.org +releases</a> page.</p> +<p> +LTS kernel releases average 6-8 patches accepted per day, while the normal +stable kernel releases contain 10-15 patches per day. The number of patches +fluctuates per release given the current time of the corresponding development +kernel release, and other external variables. The older a LTS kernel is, the +less patches are applicable to it as many recent bugfixes are not relevant to +older kernels. However, the older a kernel is, the harder it is to backport the +changes that are needed to be applied, due to the changes in the codebase. So +while there might be a lower number of overall patches being applied, the effort +involved in maintaining a LTS kernel is greater than maintaining the normal +stable kernel. +</p> + +<h2 id="stable-kernel-patch-rules">Stable kernel patch rules</h2> +<p>The rules for what can be added to a stable kernel release have remained +almost identical since its introduction and are summarized below:</p> +<ul> +<li>Must be obviously correct and tested. +<li>Must not be bigger than 100 lines. +<li>Must fix only one thing. +<li>Must fix something that has been reported to be an issue. +<li>Can be a new device id or quirk for hardware, but not add major new +functionality. +<li>Must already be merged into Linus Torvalds' tree.</li> +</ul> +<aside class="note"><strong>Note:</strong> For a full list of the rules for +patches to be accepted into a stable kernel release, refer to the +<code><a href="https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html" class="external">Documentation/process/stable_kernel_rules.rst</a></code> +kernel file.</aside> + +<p>The last rule, "Must already be merged into Linus Torvalds' tree", prevents +the kernel community from losing fixes. The community never wants a fix to go +into a stable kernel release that is not already in Linus Torvalds' tree, so +that anyone who upgrades should never see a regression. This prevents many +problems that other projects who maintain a stable and development branch can +have.</p> + +<h2 id="kernel-updates">Kernel updates</h2> +<p>The Linux kernel community has promised its userbase that no upgrade will +ever break anything that is currently working in a previous release. That +promise still holds true today. Regressions do happen, but those are the highest +priority bugs and are either quickly fixed, or the change that caused the +regression is quickly reverted from the Linux kernel tree.</p> + +<p>This promise holds true for both the incremental stable kernel updates, as +well as the larger major updates that happen every three months. However, the +kernel community can only make this promise for the code that is merged into the +Linux kernel tree. Any code that is merged into a device's kernel that is not in +the <a href="https://www.kernel.org/">kernel.org</a> releases is unknown and +interactions with it can never be planned for, or even considered.</p> + +<p>Devices based on Linux that have large patch sets can have major issues when +updating to newer kernels, because of the large number of changes between each +release (10-14 thousand changes per release). SoC patchsets are especially known +to have issues with updating to newer kernels due to their large size and heavy +modification of architecture specific, and sometimes core, kernel code. As a +result, most SoC vendors are starting to standardize on using the LTS releases +for their devices, enabling those devices to receive bug and security updates +directly from the Linux kernel community.</p> + +<h2 id="security">Security</h2> +<p>When doing kernel releases, the Linux kernel community almost never declares +specific changes as <em>security fixes</em>. This is due to the basic problem of +the difficulty in determining if a bugfix is a security fix or not at the time +of creation. Also, many bugfixes are only determined to be security related +after much time has passed, so the kernel community strongly recommends always +taking all bugfixes that are released.</p> + +<aside class="note"><strong>Note:</strong> For details on Linus Torvalds' +statement on security fixes, refer to the relevant +<a href="http://marc.info/?t=121507404600023&r=4&w=2" class="external">email +thread</a>.</aside> + +<p> +When security problems are reported to the kernel community, they are fixed as +soon as possible and pushed out publically to the development tree and the +stable releases. As described above, the changes are almost never described as +a "security fix", but rather look like any other bugfix for the kernel. This is +done to allow affected parties the ability to update their systems before the +reporter of the problem announces it. +</p> + +<p>For details on reporting security bugs to the kernel community to get +them resolved and fixed as soon as possible, refer to +<a href="https://www.kernel.org/doc/html/latest/admin-guide/security-bugs.html" class="external">Security +bugs</a> in <em>The Linux kernel user's and administrator's guide</em> at +<a href="https://www.kernel.org">www.kernel.org</a>.</p> + +<p> +Because security bugs are not announced to the public by the kernel team, CVE +numbers for Linux kernel-related issues are usually released weeks, months, and +sometimes years after the fix was merged into the stable and development +branches. +</p> +<h3 id="keeping-a-secure-system">Keeping a secure system</h3> +<p>When deploying a device that uses Linux, it is strongly recommended that all +LTS kernel updates be taken by the manufacturer and pushed out to their users +after proper testing shows the update works well. This has several advantages: +</p> +<ul> +<li>Releases have been reviewed by the kernel developers as a whole, not in +individual parts.</li> +<li>It is hard, if not impossible, to determine which patches fix "security" +issues and which do not. Almost every LTS release contains at least one known +security fix, and many yet "unknown".</li> +<li>If testing shows a problem, the kernel developer community will react +quickly to resolve the issue.</li> +<li>Attempts to filter out only the changes you run will result in a kernel +tree that is impossible to merge correctly with future upstream releases.</li> +</ul> + + </body> +</html> diff --git a/en/devices/architecture/kernel/reqs-interfaces.html b/en/devices/architecture/kernel/reqs-interfaces.html new file mode 100644 index 00000000..82349f7b --- /dev/null +++ b/en/devices/architecture/kernel/reqs-interfaces.html @@ -0,0 +1,273 @@ +<html devsite> + <head> + <title>Interface Requirements</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This page describes a subset of the Linux kernel interfaces on which Android +relies to function properly. The presence and correctness of these interfaces is +tested as part of the <a href="/devices/tech/vts/index.html">Vendor Test Suite +(VTS)</a>. This subset will grow over time to contain a larger portion of +Android kernel interfaces.</p> + +<h2 id="system-calls">System calls</h2> +<p>System calls are expected to provide the same signatures and semantics as in +the upstream Linux kernel.</p> + +<p>ARM64 system calls required by bionic per +<code>bionic/libc/SYSCALLS.txt</code>:</p> + +<table> +<tr> +<td class="devsite-click-to-copy">accept4, acct, adjtimex, bind, brk, capget, capset, chdir, chroot, clock_adjtime, clock_getres, clock_gettime, clock_nanosleep, clock_settime, close, connect, delete_module, dup3, dup, epoll_create1, epoll_ctl, epoll_pwait, eventfd2, execve, exit, exit_group, faccessat, fadvise64, fallocate, fchdir, fchmodat, fchmod, fchownat, fchown, fcntl, fdatasync, fgetxattr, flistxattr, flock, fremovexattr, fsetxattr, fstat, newfstatat, fstatfs, fsync, ftruncate, getcpu, getcwd, getdents64, getegid, geteuid, getgid, getgroups, getitimer, getpeername, getpgid, getpid, getppid, getpriority, getresgid, getresuid, getrlimit, getrusage, getsid, getsockname, getsockopt, gettimeofday, getuid, getxattr, init_module, inotify_add_watch, inotify_init1, inotify_rm_watch, ioctl, kill, syslog, lgetxattr, linkat, listen, listxattr, llistxattr, lremovexattr, lseek, lsetxattr, madvise, mincore, mkdirat, mknodat, mlockall, mlock, mmap, mount, mprotect, mremap, msync, munlockall, munlock, munmap, nanosleep, openat, personality, pipe2, ppoll, prctl, pread64, preadv, prlimit64, process_vm_readv, process_vm_writev, pselect6, ptrace, pwrite64, pwritev, quotactl, readahead, readlinkat, read, readv, reboot, recvfrom, recvmmsg, recvmsg, removexattr, renameat, rt_sigaction, rt_sigpending, rt_sigprocmask, rt_sigqueueinfo, rt_sigsuspend, rt_sigtimedwait, sched_getaffinity, sched_getparam, sched_get_priority_max, sched_get_priority_min, sched_getscheduler, sched_rr_get_interval, sched_setaffinity, sched_setparam, sched_setscheduler, sched_yield, sendfile, sendmmsg, sendmsg, sendto, setdomainname, setfsgid, setfsuid, setgid, setgroups, sethostname, setitimer, setns, setpgid, setpriority, setregid, setresgid, setresuid, setreuid, setrlimit, setsid, setsockopt, set_tid_address, settimeofday, setuid, setxattr, shutdown, sigaltstack, signalfd4, socketpair, socket, splice, statfs, swapoff, swapon, symlinkat, sync_file_range, sync, sysinfo, tee, tgkill, timer_create, timer_delete, timerfd_create, timerfd_gettime, timerfd_settime, timer_getoverrun, timer_gettime, timer_settime, times, truncate, umask, umount2, uname, unlinkat, unshare, utimensat, vmsplice, wait4, waitid, write, writev</td> +</tr></table> + +<p>ARM32 system calls required by bionic per +<code>bionic/libc/SYSCALLS.txt</code>:</p> + +<table> +<tr> +<td class="devsite-click-to-copy">accept4, acct, adjtimex, arm_fadvise64_64, bind, brk, cacheflush, capget, capset, chdir, chroot, clock_adjtime, clock_getres, clock_gettime, clock_nanosleep, clock_settime, close, connect, delete_module, dup3, dup, epoll_create1, epoll_ctl, epoll_pwait, eventfd2, execve, exit, exit_group, faccessat, fallocate, fchdir, fchmodat, fchmod, fchownat, fchown32, fcntl64, fdatasync, fgetxattr, flistxattr, flock, fremovexattr, fsetxattr, fstat64, fstatat64, fstatfs64, fsync, ftruncate64, getcpu, getcwd, getdents64, getegid32, geteuid32, getgid32, getgroups32, getitimer, getpeername, getpgid, getpid, getppid, getpriority, getresgid32, getresuid32, ugetrlimit, getrusage, getsid, getsockname, getsockopt, gettimeofday, getuid32, getxattr, init_module, inotify_add_watch, inotify_init1, inotify_rm_watch, ioctl, kill, syslog, lgetxattr, linkat, listen, listxattr, llistxattr, _llseek, lremovexattr, lseek, lsetxattr, madvise, mincore, mkdirat, mknodat, mlockall, mlock, mmap2, mount, mprotect, mremap, msync, munlockall, munlock, munmap, nanosleep, openat, personality, pipe2, ppoll, prctl, pread64, preadv, prlimit64, process_vm_readv, process_vm_writev, pselect6, ptrace, pwrite64, pwritev, quotactl, readahead, readlinkat, read, readv, reboot, recvfrom, recvmmsg, recvmsg, removexattr, renameat, rt_sigaction, rt_sigpending, rt_sigprocmask, rt_sigqueueinfo, rt_sigsuspend, rt_sigtimedwait, sched_getaffinity, sched_getparam, sched_get_priority_max, sched_get_priority_min, sched_getscheduler, sched_rr_get_interval, sched_setaffinity, sched_setparam, sched_setscheduler, sched_yield, sendfile64, sendfile, sendmmsg, sendmsg, sendto, setdomainname, setfsgid, setfsuid, setgid32, setgroups32, sethostname, setitimer, setns, setpgid, setpriority, setregid32, setresgid32, setresuid32, setreuid32, setrlimit, setsid, setsockopt, set_tid_address, settimeofday, set_tls, setuid32, setxattr, shutdown, sigaction, sigaltstack, signalfd4, socketpair, socket, splice, statfs64, swapoff, swapon, symlinkat, sync_file_range2, sync, sysinfo, tee, tgkill, timer_create, timer_delete, timerfd_create, timerfd_gettime, timerfd_settime, timer_getoverrun, timer_gettime, timer_settime, times, truncate64, truncate, umask, umount2, uname, unlinkat, unshare, utimensat, vmsplice, wait4, waitid, write, writev</td> +</tr></table> + +<p>The system calls listed below are made by bypassing bionic:</p> + +<table> + <tr> + <th style="width:20%">All Architectures</th> + <td>gettid, futex, clone, rt_sigreturn, rt_tgsigqueueinfo, restart_syscall, +getrandom, perf_event_open, syncfs, tkill, seccomp</td> + </tr> + <tr> + <th>arm</th> + <td>vfork, sigreturn, pipe, access, stat64, lstat64, open, getdents, eventfd, +epoll_wait, readlink, epoll_create, creat, unlink</td> + </tr> + <tr> + <th>arm64</th> + <td>pivot_root, ioprio_get, ioprio_set</td> + </tr> +</table> + +<aside class="note"><strong>Note:</strong> x86 and x86_64 system calls will be +added in a future release.</aside> + +<h3 id="prctl">prctl</h3> +<p>In addition to the upstream <code>prctl</code> operations for supported +kernel versions, Android relies on additional <code>prctl</code> operations, the +implementation of which can be found in the android-common kernel.</p> + +<pre class="prettyprint"> +PR_SET_TIMERSLACK_PID +PR_SET_VMA +</pre> + +<h2 id="filesystems">Filesystems</h2> +<p>The Linux kernel exports interfaces via several filesystems. Android expects +these interfaces to communicate the same information, in the same format, and +provide the same semantics as in the upstream Linux kernel. For interfaces that +do not exist upstream, the appropriate behavior is dictated by the corresponding +branch of the Android common kernel.</p> + +<h3 id="procfs">procfs</h3> +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/proc/cmdline</code></td> + <td>Read-only file containing command line arguments passed to the kernel. + </td> + </tr> + <tr> + <td><code>/proc/config.gz</code></td> + <td>Read-only file containing kernel build configuration.</td> + </tr> + <tr> + <td><code>/proc/cpuinfo</code></td> + <td>Read-only file containing architecture-specific CPU details.</td> + </tr> + <tr> + <td><code>/proc/kmsg</code></td> + <td>Read-only file showing kernel messages in real time.</td> + </tr> + <tr> + <td><code>/proc/meminfo</code></td> + <td>Read-only file showing memory subsystem details.</td> + </tr> + <tr> + <td><code>/proc/modules</code></td> + <td>Read-only file containing information about loaded kernel modules.</td> + </tr> + <tr> + <td><code>/proc/mounts</code></td> + <td>Symlink to <code>/proc/self/mounts</code>, which is a read-only file + listing information about the mounted filesystems.</td> + </tr> + <tr> + <td><code>/proc/net/xt_qtaguid/ctrl</code></td> + <td>Read-write file providing information about tagged sockets.</td> + </tr> + <tr> + <td><code>/proc/self/maps</code></td> + <td>Read-only file containing the currently mapped memory regions and + permissions.</td> + </tr> + <tr> + <td><code>/proc/stat</code></td> + <td>Read-only file containing various kernel and system statistics.</td> + </tr> + <tr> + <td><code>/proc/sys/kernel/kptr_restrict</code></td> + <td>Read-write file that determines whether kernel pointers are printed in + <code>proc</code> files and other interfaces.</td> + </tr> + <tr> + <td><code>/proc/sys/kernel/randomize_va_space</code></td> + <td>Read-write file that determines the address layout randomization policy + for the system.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_min_addr</code></td> + <td>Read-write file that determines the minimum address than can be + <code>mmap</code>'d.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_rnd_bits</code></td> + <td>Read-write file that specifies the amount of randomness in + <code>mmap</code>'d addresses.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/mmap_rnd_compat_bits</code></td> + <td>Read-write file that specifies the amount of randomness in + <code>mmap</code>'d addresses.</td> + </tr> + <tr> + <td><code>/proc/sys/vm/overcommit_memory</code></td> + <td>Read-write file that determines the kernel virtual memory accounting + mode.</td> + </tr> + <tr> + <td><code>/proc/uid_cputime/remove_uid_range</code></td> + <td>Write-only file that, when written, removes UIDs from being shown in + <code>/proc/uid_cputime/show_uid_stat</code>.</td> + </tr> + <tr> + <td><code>/proc/uid_cputime/show_uid_stat</code></td> + <td>Read-only file containing the time a UID's processes spent in user and + kernel space.</td> + </tr> + <tr> + <td><code>/proc/version</code></td> + <td>Read-only file containing a string describing the kernel version.</td> + </tr> + <tr> + <td><code>/proc/vmallocinfo</code></td> + <td>Read-only file containing <code>vmalloc</code>'d ranges.</td> + </tr> + <tr> + <td><code>/proc/zoneinfo</code></td> + <td>Read-only file containing information about memory zones.</td> + </tr> +</table> + +<h3 id="dev">dev</h3> + +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/dev/ashmem</code></td> + <td>Anonymous shared memory device file.</td> + </tr> + <tr> + <td><code>/dev/binder</code></td> + <td>Binder device file.</td> + </tr> + <tr> + <td><code>/dev/hwbinder</code></td> + <td>Hardware binder device file.</td> + </tr> + <tr> + <td><code>/dev/tun</code></td> + <td>Universal TUN/TAP device file.</td> + </tr> + <tr> + <td><code>/dev/xt_qtaguid</code></td> + <td>QTAGUID netfilter device file.</td> + </tr> +</table> + +<h3 id="sysfs">sysfs</h3> +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/sys/devices/system/cpu/online</code></td> + <td>Read-only file showing ranges of CPUs that are currently online.</td> + </tr> + <tr> + <td><code>/sys/kernel/wakeup_reasons/last_resume_reason</code></td> + <td>Read-only file showing a textual description of why the system exited the + last instance of suspend.</td> + </tr> + <tr> + <td><code>/sys/devices/system/cpu/kernel_max</code></td> + <td>Read-only file showing the maximum CPU index supported by the kernel. + </td> + </tr> +</table> + +<h3 id="selinuxfs">selinuxfs</h3> +<p>The framework mounts <code>selinuxfs</code> at <code>/sys/fs/selinux</code>. +</p> + +<table> + <tr> + <th>Path</th> + <th>Description</th> + </tr> + <tr> + <td><code>/sys/fs/selinux/checkreqprot</code></td> + <td>Read/write file containing a binary flag that determines how selinux + protections are checked on <code>mmap</code> and <code>mprotect</code> calls. + </td> + </tr> + <tr> + <td><code>/sys/fs/selinux/null</code></td> + <td>Read/write null device for use by selinux.</td> + </tr> + <tr> + <td><code>/sys/fs/selinux/policy</code></td> + <td>Read-only file containing the selinux policy in binary form.</td> + </tr> +</table> + +<aside class="note"><strong>Note:</strong> For details on SELinux in Android +8.0, see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for +Android 8.0</a>.</aside> + + </body> +</html> diff --git a/en/devices/architecture/kernel/squashfs.html b/en/devices/architecture/kernel/squashfs.html new file mode 100644 index 00000000..93b97ad0 --- /dev/null +++ b/en/devices/architecture/kernel/squashfs.html @@ -0,0 +1,114 @@ +<html devsite> + <head> + <title>Optimizing SquashFS at the Kernel Level</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + <p> + SquashFS is a compressed read-only filesystem for Linux. The file system is + read-only by design and thus suitable for use on the system partition. Many + Android devices may benefit from using this file system for their system + partition, for example, the following: + </p><ul> + <li>Devices with a low capacity storage such as Android Watch. + <li>Devices with a slow flash storage (compression reduces the number of block + I/Os).</li></ul> + <p> + Unfortunately the performance of SquashFS lags behind ext4. + </p> + <h2 id="optimizations">Optimizations</h2> + <p> + The following optimizations have been implemented to improve the performance of + SquashFS. + </p> + <h3 id="reduce-the-memory-usage-and-memcpy">Reduce the memory usage and + memcpy</h3> + <p> + When reading a block (default 128K), SquashFS tries to grab all the pages + covering this block. + </p> + <p> + If a single page is up-to-date or locked, it falls back to allocating a full + block, submitting a read request, and then copying its content to the pages. + </p> + <p> + This approach is very ineffective; after some time the page cache is likely to + contain pages that are up-to-date even if the adjacent pages are not. + </p> + <p> + The code is now able to handle blocks with holes (=missing pages). This + improves performance in the following ways: + </p><ul> + <li>Reduces the number of <code>memcpy</code> calls + <li>Decreases memory allocations</li></ul> + <h3 id="asynchronous-reads">Asynchronous reads</h3> + <p> + SquashFS still uses the deprecated <code>ll_rw_block()</code> function. There + are two problems with this approach: + </p><ul> + <li>As the name implies, the function waits for the reads to complete before + returning. This is redundant since <code>.readpage()</code> already waits on the + page's lock. Moreover, we need an asynchronous mechanism to efficiently + implement <code>.readpages()</code>. + <li>Merging the read requests entirely depends on the I/O scheduler. + <code>ll_rw_block()</code> simply creates one request per buffer. SquashFS has + more information than the I/O scheduler about what should be merged. Moreover, + merging the request means that we rely less on the I/O scheduler.</li></ul> + <p> + For these reasons, the <code>ll_rw_block()</code> function has been replaced + with <code>submit_bio()</code>. + </p> + <h3 id="readpages-prefetching">Readpages (prefetching)</h3> + <p> + SquashFS does not implement <code>.readpages()</code>, so the kernel repeatedly + calls <code>.readpage()</code>. + </p> + <p> + Now that our read requests are asynchronous, the kernel can truly prefetch pages + using its asynchronous read-ahead mechanism. + </p> + <h3 id="optimize-reading-uncompressed-blocks">Optimize reading uncompressed + blocks</h3> + <p> + Modern systems such as Android contain a lot of files that are already + compressed. As a consequence, the image contains a lot of blocks that can't be + compressed. + </p> + <p> + SquashFS handles compressed and uncompressed blocks using the same logic: when + asked to read a single page, it actually reads a full block (default 128k). + While this is necessary for compressed blocks, it is just a waste of resources + for uncompressed blocks. + </p> + <p> + Instead of reading a full block, SquashFS now just reads what is advised by the + readahead algorithm. + </p> + <p> + This greatly improves the performance of random reads. + </p> + <h2 id="code">Code</h2> + <p> + SquashFS code optimizations are available in AOSP: + </p><ul> + <li><a + href="https://android-review.googlesource.com/#/q/topic:squashfs">https://android-review.googlesource.com/#/q/topic:squashfs</a></li></ul> + </body> +</html> diff --git a/en/devices/architecture/treble.html b/en/devices/architecture/treble.html index 343858df..ca3c4086 100644 --- a/en/devices/architecture/treble.html +++ b/en/devices/architecture/treble.html @@ -22,11 +22,13 @@ --> -<p>The upcoming Android O release includes Project Treble, a major re-architect +<p>The Android 8.0 release includes Project Treble, a major re-architect of the Android OS framework designed to make it easier, faster, and less costly -for manufacturers to update devices to a new version of Android.</p> +for manufacturers to update devices to a new version of Android. Treble is for +all new devices launching with Android 8.0 and beyond (the new architecture is +already running on the Developer Preview for Pixel phones).</p> -<h2 id=about-treble>Android updates</h2> +<h2 id=about-treble>About Android updates</h2> <p>Project Treble separates the vendor implementation (device-specific, lower-level software written by silicon manufacturers) from the Android OS framework via a new vendor interface.</p> @@ -51,23 +53,40 @@ additional work required from the silicon manufacturers:</p> environment</p> <h2 id=testing-treble>Testing Treble</h2> -<p>To ensure forward compatibility of the vendor implementation, the new vendor -interface will be validated by the Vendor Test Suite (VTS), which is analogous -to the <a href="/compatibility/cts/">Compatibility Test Suite (CTS)</a>. VTS is -already launched and can be used to automate HAL and OS kernel testing even in -pre-Treble environments; for details, see -<a href="/devices/tech/test_infra/tradefed/fundamentals/vts">Systems Testing -With VTS</a>.</p> - -<h2 id=launching-treble>Coming soon</h2> -<p>Project Treble is coming to all new devices launching with Android O and -beyond, and the new architecture is already running on the Developer Preview of -O for Pixel phones. When Android O launches, we'll have full details here (on -<a href="https://source.android.com/">source.android.com</a>). In the interim, -you can find more details on Treble over at the -<a href="https://android-developers.googleblog.com/2017/05/here-comes-treble-modular-base-for.html">Android -Developers Blog</a>.</p> - +<p>To ensure forward compatibility of vendor implementations, the new vendor +interface is validated by the <a href="/devices/tech/vts/index.html">Vendor Test +Suite (VTS)</a>, which is analogous to the +<a href="/compatibility/cts/">Compatibility Test Suite (CTS)</a>. You can use +VTS to automate HAL and OS kernel testing in both pre-Treble and Treble +environments.</p> + +<h2 id=treble-resources>Treble resources</h2> +<p>For details on the new Treble architecture, see the following sections:</p> +<ul> +<li><a href="/devices/architecture/hal-types.html">HAL Types</a>. Describes +binderized, passthrough, Same-Process (SP), and legacy HALs.</li> +<li><a href="/devices/architecture/hidl/index.html">HIDL (General)</a>. +Contains general information about the HAL interface definition language (HIDL, +pronounced "hide-l"), which is an interface description language (IDL) to +specify the interface between a HAL and its users.</li> +<li><a href="/devices/architecture/hidl-cpp/index.html">HIDL (C++)</a>. Contains +details for creating C++ implementations of HIDL interfaces.</li> +<li><a href="/devices/architecture/hidl-java/index.html">HIDL (Java)</a>. +Contains details about the Java frontend for HIDL interfaces.</li> +<li><a href="/devices/architecture/configstore/index.html">ConfigStore HAL</a>. +Describes the ConfigStore HAL, which provides a set of APIs for accessing +read-only configuration items used to configure the Android framework.</li> +<li><a href="/devices/architecture/dto/index.html">Device Tree Overlays</a>. +Provides details on using device tree overlays (DTOs) in Android.</li> +<li><a href="/devices/architecture/vndk/index.html">Vendor Native Development +Kit (VNDK)</a>. Describes the VNDK, which is a set of libraries exclusively for +vendors to implement their HALs.</li> +<li><a href="/devices/architecture/vintf/index.html">Vendor Interface Object +(VINTF)</a>. VINTF objects aggregate relevant information about a device and +make that information available through a queryable API.</li> +<li><a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for Android +8.0</a>. Details SELinux changes and customizations.</li> +</ul> </body> </html> diff --git a/en/devices/architecture/vintf/comp-matrices.html b/en/devices/architecture/vintf/comp-matrices.html new file mode 100644 index 00000000..d3512e70 --- /dev/null +++ b/en/devices/architecture/vintf/comp-matrices.html @@ -0,0 +1,263 @@ +<html devsite> + <head> + <title>Compatibility Matrices</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This section describes the framework and device compatibility matrices and +the <a href="#compatiblity-matrix-schema">compatibility matrix schema</a>. For +match rules, see <a href="/devices/architecture/vintf/match-rules.html">Matching +Rules</a>.</p> + +<h2 id="framework-compatibility-matrix">Framework compatibility matrix</h2> +<p>The framework compatibility matrix describes the requirements of the +framework on the device it runs on. The matrix file is associated with the +Android framework image (on <code>system.img</code>). It is expected the +requirements of the framework's compatibility matrix will be satisfied by the +device manifest (requirements enforced at launch and OTA time).</p> + +<p>Example framework compatibility matrix file:</p> + +<pre class="prettyprint"> +<?xml version="1.0" encoding="UTF-8"?> +<!-- Comments, Legal notices, etc. here --> +<compatibility-matrix version="1.0" type="framework"> + <hal> + <name>android.hardware.camera</name> + <version>1.0</version> + <version>3.1-4</version> + <interface> + <name>ICameraProvider</name> + <instance>default</instance> + </interface> + </hal> + <hal> + <name>android.hardware.nfc</name> + <version>1.0</version> + <interface> + <name>INfc</name> + <instance>default</instance> + </interface> + </hal> + <hal optional="true"> + <name>android.hardware.graphics.composer</name> + <version>2.1</version> + </hal> + <hal format="native"> + <name>GL</name> + <version>1.1</version> + <version>3.0</version> + </hal> + <hal format="native"> + <name>EGL</name> + <version>1.1</version> + </hal> + <kernel version="3.18.51"> + <config> + <key>CONFIG_A</key> + <value type="string"></value> + </config> + <config> + <key>CONFIG_B</key> + <value type="tristate">y</value> + </config> + </kernel> + <kernel version="4.1.22"> + <config> + <key>CONFIG_A</key> + <value type="string">foo</value> + </config> + <config> + <key>CONFIG_B2</key> + <value type="int">1024</value> + </config> + </kernel> + <sepolicy> + <kernel-sepolicy-version>30</kernel-sepolicy-version> + <sepolicy-version>25.0</sepolicy-version> + <sepolicy-version>26.0-3</sepolicy-version> + </sepolicy> + <avb> + <vbmeta-version>2.1</vbmeta-version> + </avb> + <xmlfile format="dtd"> + <name>media_profile</name> + <version>1.0</version> + <path>/system/etc/media_profile_V1_0.dtd</path> + </xmlfile> +</compatiblity-matrix> +</pre> + +<h2 id="device-compatibility-matrix">Device compatibility matrix</h2> +<p>The device compatibility matrix describes a set of requirements the device +expects from the framework (requirements enforced at launch and OTA time). +</p> +<p>Example device compatibility matrix file:</p> + +<pre class="prettyprint"> +<?xml version="1.0" encoding="UTF-8"?> +<!-- Comments, Legal notices, etc. here --> +<compatibility-matrix version="1.0" type="device"> + <hal> + <name>android.hidl.manager</name> + <version>1.0</version> + <interface> + <name>IServiceManager</name> + <instance>default</instance> + </interface> + </hal> + <hal> + <name>android.hidl.memory</name> + <version>1.0</version> + <interface> + <name>IMemory</name> + <instance>ashmem</instance> + </interface> + </hal> + <hal> + <name>android.hidl.allocator</name> + <version>1.0</version> + <interface> + <name>IAllocator</name> + <instance>ashmem</instance> + </interface> + </hal> + <hal> + <name>android.framework.sensor</name> + <version>1.0</version> + <interface> + <name>ISensorManager</name> + <instance>default</instance> + </interface> + </hal> + <xmlfile format="dtd" optional="false"> + <name>sample_xml</name> + <version>1.0</version> + </xmlfile> +</compatibility-matrix> +</pre> + +<h2 id="compatiblity-matrix-schema">Compatiblity matrix schema</h2> +<dl> +<dt><code>?xml</code></dt> +<dd>Optional. It only provides information to the XML parser.</dd> +<dt><code>compatibility-matrix.version</code></dt> +<dd>Required. Version of this compatiblity matrix. Describes the elements +expected in the manifest. Unrelated to XML version.</dd> +<dt><code>compatibility-matrix.type</code></dt> +<dd>Required. Type of this compatibility matrix: + <ul> + <li><code>"device"</code>: Device compatibility matrix.</li> + <li><code>"framework"</code>: Framework compatibility matrix.</li> + </ul> +</dd> +<dt><code>compatibility-matrix.hal</code></dt> +<dd>Optional and can repeat. Lists a single HAL (HIDL or native) that is +required by owner of the compatibility matrix (framework or device) to be +present. HAL entries are distinguished by a <code><name></code> element; +there can be several HAL entries with the same name (implies "and" condition). +</dd> +<dt><code>compatibility-matrix.hal.format</code></dt> +<dd>Optional. Value can be one of: + <ul> + <li><code>"hidl"</code>: HIDL HALs. This is the default.</li> + <li><code>"native"</code>: native HALs.</li> + </ul> +</dd> +<dt><code>compatibility-matrix.hal.optional</code></dt> +<dd>Attribute is optional and defaults to false. States whether this HAL is +optional to the owner of the compatibility matrix (framework or device). If a +<code><p;hal></code> entry is marked as optional, it means the owner can +work with this HAL, if present, but does not require it to be present.</dd> +<dt><code>compatibility-matrix.hal.name</code></dt> +<dd>Required. Full package name of this HAL. Examples: + <ul> + <li><code>android.hardware.camera</code> (HIDL HAL)</li> + <li><code>GLES</code> (native HAL, requires name only)</li> + </ul> +</dd> +<dt><code>compatibility-matrix.hal.version</code></dt> +<dd>Required, can repeat without duplicates. A list of version ranges (see +<a href="/devices/architecture/vintf/match-rules.html#hals">HAL matches</a>) +that defines what versions the owner of the compatibility matrix (framework or +device) expects.</dd> +<dt><code>compatibility-matrix.hal.interface</code></dt> +<dd>Optional, can repeat. A list of required interfaces of this HAL.</dd> +<dt><code>compatibility-matrix.hal.interface.name</code></dt> +<dd>Required. Name of the interface.</dd> +<dt><code>compatibility-matrix.hal.interface</code></dt> +<dd>Optional, can repeat. A list of required instances of this interface.</dd> +<dt><code>compatibility-matrix.kernel.version</code></dt> +<dd>Required. Kernel version. Format is +<code>{version}.{major-revision}.{minor-revision}</code>. Version and major +revision must match exactly, minor-revision defines the minimum LTS version of +the kernel the framework expects.</dd> +<dt><code>compatibility-matrix.kernel.config</code></dt> +<dd>Optional, can repeat. Lists <code>CONFIG</code> items that must be +matched for this kernel version. Each <code>CONFIG</code> item is a key-value +pair; config items are distinguished by key.</dd> +<dt><code>compatibility-matrix.kernel.config.key</code></dt> +<dd>Required. Key name of the <code>CONFIG</code> item. Starts with +<code>CONFIG_</code>.</dd> +<dt><code>compatibility-matrix.kernel.config.value</code></dt> +<dd>Required. Value of the <code>CONFIG</code> item. Format depends on type: + <ul> + <li><code>string</code>. Quotes are omitted.</li> + <li><code>int</code>. Decimal and hexadecimal (must start with <code>0x</code> + or <code>0X)</code>values are accepted. Interpreted as an 64-bit integer; + overflows result in truncation. (Parser accepts values from -2<sup>64</sup> + 1 + to 2<sup>64</sup> - 1, 65th bit is truncated; for details refer to the + <a href="http://man7.org/linux/man-pages/man3/strtoul.3.html" class="external">strtoull + man page</a>.)</li> + <li><code>range</code>. Format is <code>[int]-[int]</code>, e.g. + <code>10-20</code>. Hexadecimal values are accepted and must start with + <code>0x</code> or <code>0X</code>. Two boundaries must be an unsigned 64-bit + integer.</li> + <li><code>tristate</code>. Valid values are <code>y</code>, <code>m</code> and + <code>n</code>.</li> + </ul> +</dd> +<dt><code>compatibility-matrix.kernel.config.value.type</code></dt> +<dd>Required. Type of the value of the <code>CONFIG</code> item, one of: + <ul> + <li><code>string</code></li> + <li><code>int</code></li> + <li><code>range</code></li> + <li><code>tristate</code></li> + </ul> +</dd> +<dt><code>compatibility-matrix.sepolicy</code></dt> +<dd>Required. Contains all sepolicy-related entries. Used only by the +framework compatibility matrix.</dd> +<dt><code>compatibility-matrix.sepolicy.sepolicy-version</code></dt> +<dd>Required, can repeat. Describes the requirement on sepolicy version. +Corresponds to <code>manifest.sepolicy.version</code>. Each instance of an +element defines a range of sepolicy versions.</dd> +<dt><code>compatibility-matrix.sepolicy.kernel-sepolicy-version</code></dt> +<dd>Required. Declares the <code>policydb</code> version the framework works +with.</dd> +<dt><code>compatibility-matrix.avb.vbmeta-version</code></dt> +<dd>Optional; used only by the framework compatibility matrix. Declares the +<a href="/devices/architecture/vintf/match-rules.html#avb-version">AVB +version</a> used to sign <code>system.img</code>.</dd> +</dl> + + </body> +</html> diff --git a/en/devices/architecture/vintf/index.html b/en/devices/architecture/vintf/index.html new file mode 100644 index 00000000..b0517a32 --- /dev/null +++ b/en/devices/architecture/vintf/index.html @@ -0,0 +1,113 @@ +<html devsite> + <head> + <title>Vendor Interface Object</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This document describes the design of the <em>vendor interface object</em> +(VINTF object), which aggregates relevant information about a device and makes +that information available through a <em>queryable API</em>.</p> + + +<h2 id=about-objects>VINTF object design</h2> +<p>A VINTF object gathers some of the information it needs directly from the +device. Other aspects, such as the manifests, are described statically in +XML.</p> + +<img src="../images/treble_vintf_mm.png"> +<figcaption><strong>Figure 1.</strong> Manifests, compatibility matrices, and +runtime-collectible information.</figcaption> + +<p>VINTF object design provides the following for device and framework +components:</p> + +<table> +<tr> +<th style="width:50%">For the Device</th> +<th>For the Framework</th> +</tr> +<tr> +<td> +<ul> +<li>Defines a schema for the static component (the +<a href="/devices/architecture/vintf/objects.html#device-manifest-file">device +manifest file</a>).</li> +<li>Adds build time support for defining the device manifest file for a given +device.</li> +<li>Defines the +<a href="/devices/architecture/vintf/objects.html#queryable-api">queryable +API</a> at runtime that retrieves the device manifest file (along with the other +runtime-collectible information) and packages them into the query result.</li> +</ul> +</td> +<td> +<ul> +<li>Defines a schema for the static component (the +<a href="/devices/architecture/vintf/objects.html#framework-manifest-file">framework +manifest file</a>).</li> +<li>Defines the +<a href="/devices/architecture/vintf/objects.html#queryable-api">queryable +API</a> at runtime that retrieves the framework manifest file and packages it +into the query result.</li> +</ul> +</td> +</tr> +</table> + +<p>The VINTF object must be reliable and provide the same complete information +regardless of when the object is requested (see +<a href="/devices/architecture/vintf/resources.html#caveats">Caveats</a>).</p> + +<h2 id=manifests-matrices>Manifests & matrices</h2> +<p>Android O requires an API at runtime to query what is on the device and send +that information to the <a href="/devices/tech/ota/index.html">Over-the-Air +(OTA)</a> update server and other interested parties (such as CTS +<code>DeviceInfo</code>). Some information is retrieved at runtime and some of +it is statically-defined.</p> + +<ul> +<li>The <strong>device manifest</strong> describes the static component of what +the device can provide to the the framework.</li> +<li>The <strong>framework compatibility matrix</strong> describes what the +Android framework expects from a given device. The matrix is a static entity +whose composition is determined manually during development of the next release +of the Android framework.</li> +<li>The <strong>framework manifest</strong> describes high-level services the +framework can provide to the device.</li> +<li>The <strong>device compatibility matrix</strong> describes the services the +vendor image requires of the framework. Its composition is determined manually +during the development of the device.</li> +</ul> + +<p>These two pairs of manifests and matrices must be reconciled at OTA time to +ensure a device can get framework updates that are compatible with the device's +capabilities. In general, a <em>manifest</em> describes what is provided and a +<em>compatibility matrix</em> describes what is required.</p> + +<p><a href="/devices/architecture/vintf/objects.html">VINTF Object Data</a> +defines the schema for the manifest, +<a href="/devices/architecture/vintf/comp-matrices.html">Compatibility +Matrices</a> defines the schema for the compatibility matrix, and +<a href="/devices/architecture/vintf/match-rules.html">Matching Rules</a> +defines the rules for a successful match between a compatibility matrix +and a manifest.</p> + + </body> +</html> diff --git a/en/devices/architecture/vintf/match-rules.html b/en/devices/architecture/vintf/match-rules.html new file mode 100644 index 00000000..61a5f1a7 --- /dev/null +++ b/en/devices/architecture/vintf/match-rules.html @@ -0,0 +1,366 @@ +<html devsite> + <head> + <title>Matching Rules</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>The two pairs of compatibility matrices and manifests are meant to be +reconciled at <a href="/devices/tech/ota/index.html">OTA</a> time to verify the +framework and vendor implementation can work with each other. This verification +is successful upon a match between the framework compatibility matrix and the +device manifest, as well as between the framework manifest and the device +compatibility matrix. The following sections detail matching rules used by +various components.</p> + +<h2 id="hals">HAL matches</h2> +<p>The HAL-match rule identifies the versions of <code>hal</code> elements in a +manifest file that are considered supported by the owner of the corresponding +compatibility matrix.</p> +<ul> +<li>Multiple <code>version</code> elements are concatenated with +<strong>OR</strong> (see camera example below).</li> +<li>Multiple <code><hal></code> elements with the same name are +concatenated with <strong>AND</strong>.</li> +</ul> + +<h4><strong>Example</strong>: Successful HAL match for Camera module</h4> +<p>For a HAL at version 2.5, the match rule is as follows:</p> + +<table> + <tr> + <th>Matrix</th> + <th>Matching Manifest</th> + </tr> + <tr> + <td><code>2.5</code></td> + <td>2.5-2.∞. Shorthand for 2.5-5.</td> + </tr> + <tr> + <td><code>2.5-7</code></td> + <td>2.5-2.∞. Indicates the following: + <br> + <ul> + <li>2.5 is the minimum required version, meaning a manifest providing HAL + 2.0-2.4 is not compatible.</li> + <li>2.7 is the maximum version that could be requested, meaning the owner of + the compatibility matrix (framework or device) will not request versions + beyond 2.7. The owner of the matching manifest can still serve version 2.10 + (as an example) when 2.7 is requested. The compatibility-matrix owner knows + only that the requested service is compatible with API version 2.7.</li> + <li>-7 is informational only and does not affect the OTA update process.</li> + </ul> + Thus, a device with a HAL at version 2.10 in its manifest file remains + compatible with a framework that states <code>camera:</code> + <code>2.5-7</code> in its compatibility matrix.</td> + </tr> +</table> + +<h4><strong>Example:</strong> Successful HAL match for DRM module</h4> +<p>The framework compatibility matrix states the following version information +for DRM HAL:</p> + +<pre class="prettyprint"><hal> + <name>android.hardware.drm + <version>1.0</version> + <version>3.1-2</version> + <interface> + <name>IDrmFactory</name> + <instance>default</instance> + <instance>specific</instance> + </interface> +</hal> +<hal> + <name>android.hardware.drm + <version>2.0</version> + <interface> + <name>ICryptoFactory</name> + <instance>default</instance> + </interface> +</hal> +</pre> + +<p>A vendor must implement ONE of the following HALs:</p> + +<pre> +android.hardware.drm@1.x::IDrmFactory/default //where x >= 0 +android.hardware.drm@1.x::IDrmFactory/specific //where x >= 0 +</pre> + +OR + +<pre> +android.hardware.drm@3.y::IDrmFactory/default //where y >= 1 +android.hardware.drm@3.y::IDrmFactory/specific //where y >= 1 +</pre> + +<p>... AND must also implement this HAL:</p> + +<pre> +android.hardware.drm@2.z::ICryptoFactory/default //where z >= 0 +</pre> + +<h2 id="kernel">Kernel matches</h2> +<p>The <code><kernel></code> section of the framework compatibility matrix +describes the framework's requirements of the Linux kernel on the device. This +information is meant to be matched at OTA time against the +<a href="/devices/architecture/vintf/objects.html#runtime-collectible-information">information</a> +about the kernel that gets reported by the device's VINTF object.</p> + +<p>A matrix can include multiple <code><kernel></code> sections, each with +a different <code>version</code> attribute using the format:</p> + +<pre class="prettyprint">${ver}.${major_rev}.${kernel_minor_rev}</pre> + +<p>The OTA considers only the <code><kernel></code> section with the same +<code>${ver}</code> and <code>${major_rev}</code> as the device kernel (i.e., +<code>version="${ver}.${major_rev}.${matrix_minor_rev}")</code>; other sections +are ignored. In addition, the minor revision from the kernel must be a value +from the compatibility matrix (<code>${kernel_minor_rev} >= +${matrix_minor_rev}</code>;). If no <code><kernel></code> section meets +these requirements, it is a non-match.</p> + +<p>If the <code><kernel></code> section does match, the process continues +by attempting to match <code>config</code> elements against +<code>/proc/config.gz</code>. For each config element in the compatibility +matrix, it looks up <code>/proc/config.gz</code> to see if the config is +present. When a config item is set to <code>n</code> in the compatibility +matrix for the matching <code><kernel></code> section, it must be absent +from <code>/proc/config.gz</code>. Finally, a config item not in the +compatibility matrix may or may not be present in <code>/proc/config.gz</code>. +</p> + +<p>Examples of matches:</p> +<ul> +<li><code><value type="string">bar</value></code> matches +<code>"bar"</code>. Quotes are omitted in the compatibility matrix but present +in <code>/proc/config.gz</code>.</li> +<li><code><value type="int">4096</value></code> matches +<code>4096</code> or <code>0x1000</code> or <code>0X1000</code>.</li> +<li><code><value type="int">0x1000</value></code> matches +<code>4096</code> or <code>0x1000</code> or <code>0X1000</code>.</li> +<li><code><value type="int">0X1000</value></code> matches +<code>4096</code> or <code>0x1000</code> or <code>0X1000</code>.</li> +<li><code><value type="tristate">y</value></code> matches +<code>y</code>.</li> +<li><code><value type="tristate">m</value></code> matches +<code>m</code>.</li> +<li><code><value type="tristate">n</value></code> means the config +item must NOT exist in <code>/proc/config.gz</code>.</li> +<li><code><value type="range">1-0x3</value></code> matches +<code>1</code>, <code>2</code>, or <code>3</code>, or hexadecimal equivalent. +</li> +</ul> + +<h4><strong>Example:</strong> Successful kernel match</h4> +<p>A framework compatibility matrix has the following kernel information:</p> + +<pre class="prettyprint"> +<kernel version="3.18.51"> + <config> + <key>CONFIG_TRI</key> + <value type="tristate">y</value> + </config> + <config> + <key>CONFIG_NOEXIST</key> + <value type="tristate">n</value> + </config> + <config> + <key>CONFIG_DEC</key> + <value type="int">4096</value> + </config> + <config> + <key>CONFIG_HEX</key> + <value type="int">0XDEAD</value> + </config> + <config> + <key>CONFIG_STR</key> + <value type="string">str</value> + </config> + <config> + <key>CONFIG_EMPTY</key> + <value type="string"></value> + </config> +</kernel> +</pre> + +<p>The kernel version is matched first. If a device in <code>uname()</code> +reports:</p> +<ul> +<li>3.10.73 (no match to matrix unless there is a separate kernel section +with <code><kernel version="3.10.x"></code> where <code>x <= 73</code>) +</li> +<li>3.18.50 (no match to matrix, smaller than <code>version</code>)</li> +<li>3.18.51 (match to matrix)</li> +<li>3.18.52 (match to matrix)</li> +<li>4.1.22 (no match to matrix unless there is a separate kernel section +with <code><kernel version="4.1.x"></code> where <code>x <= 22</code>) +</li> +</ul> + +<p>After the appropriate <code><kernel></code> section is selected, for +each <code><config></code> item with value other than <code>n</code>, we +expect the corresponding entry to be present in <code>/proc/config.gz</code>; +for each <code><config></code> item with value <code>n</code>, we expect +the corresponding entry to not be present in <code>/proc/config.gz</code>. We +expect the content of <code><value></code> to exactly match the text after +the equal sign (including quotes), up to the newline character or +<code>#</code>, with leading and trailing whitespace truncated.</p> + +<p>The following kernel configuration is an example of a successful match:</p> + +<pre class="prettyprint"> +# comments don't matter +CONFIG_TRI=y +# CONFIG_NOEXIST should not exist +CONFIG_DEC = 4096 # trailing comments and whitespaces are fine +CONFIG_HEX=57005 # 0XDEAD == 57005 +CONFIG_STR="str" +CONFIG_EMPTY="" # empty string must have quotes +CONFIG_EXTRA="extra config items are fine too" +</pre> + +<p>The following kernel configuration is an example of an unsuccessful match:</p> + +<pre class="prettyprint"> +CONFIG_TRI="y" # mismatch: quotes +CONFIG_NOEXIST=y # mismatch: CONFIG_NOEXIST exists +CONFIG_HEX=0x0 # mismatch; value doesn't match +CONFIG_DEC="" # mismatch; type mismatch (expect int) +CONFIG_EMPTY=1 # mismatch; expects "" +# mismatch: CONFIG_STR is missing +</pre> + +<h2 id="se-policy">SE policy matches</h2> +<p>SE policy requires the following matches:</p> +<ul> +<li><code><sepolicy-version></code> defines a closed range of minor +versions for every major version. The sepolicy version reported by the device +must fall within one of these ranges to be compatible with the framework. Match +rules are similar to HAL versions; it is a match if the sepolicy version is +higher or equal to the minimum version for the range. The maximum version is +purely informational.</li> +<li><code><kernel-sepolicy-version></code> i.e. policydb version. Must +exactly match the <code>security_policyvers()</code> reported by the device. +</li> +</ul> + +<h4><strong>Example:</strong> Successful SE policy match</h4> +<p>The framework compatibility matrix states the following sepolicy information: +</p> + +<pre class="prettyprint"> + <sepolicy> + <kernel-sepolicy-version>30</kernel-sepolicy-version> + <sepolicy-version>25.0</sepolicy-version> + <sepolicy-version>26.0-3</sepolicy-version> + </sepolicy> +</pre> + +<p>On the device:</p> +<ul> +<li>The value returned by <code>security_policyvers()</code> must exactly equal +30. Otherwise it is not a match.</li> +<li>SE Policy version must be one of 25.0-∞ or 26.0-∞. Otherwise it is not a +match. (The "<code>-3</code>" after "<code>26.0</code>" is purely +informational.)</li> +</ul> + +<h2 id="avb-version">AVB version matches</h2> +<p>The AVB version contains a MAJOR version and MINOR version, with the format +as MAJOR.MINOR (e.g., 1.0, 2.1). For details, refer to +<a href="https://android.googlesource.com/platform/external/avb/#Versioning-and-compatibility" class="external">Versioning +and Compatibility</a>. AVB version has the following system properties:</p> +<ul> +<li><code>ro.boot.vbmeta.avb_version</code> is the <code>libavb</code> version +in bootloader</li> +<li><code>ro.boot.avb_version</code> is the <code>libavb</code> version in +Android OS (<code>init/fs_mgr</code>)</li> +</ul> + +<p>The system property appears only when the corresponding libavb has been used +to verify AVB metadata (and returns OK). It is absent if a verification failure +occurred (or no verification occurred at all).</p> + +<p>A compatibility match compares the following:</p> +<ul> +<li>sysprop <code>ro.boot.vbmeta.avb_version</code> with +<code>avb.vbmeta-version</code> from framework compatibility matrix; + <ul> + <li><code>ro.boot.vbmeta.avb_version.MAJOR == avb.vbmeta-version.MAJOR</code></li> + <li><code>ro.boot.vbmeta.avb_version.MINOR >= avb.vbmeta-version.MINOR</code></li> + </ul> +</li> +<li>sysprop <code>ro.boot.avb_version</code> with +<code>avb.vbmeta-version</code> from framework compatibility matrix. + <ul> + <li><code>ro.boot.avb_version.MAJOR == avb.vbmeta-version.MAJOR</code></li> + <li><code>ro.boot.avb_version.MINOR >= avb.vbmeta-version.MINOR</code></li> + </ul> +</li> +</ul> + +<p>The bootloader or Android OS might contain two copies of <code>libavb</code> +libraries, each with a different MAJOR version for upgrade devices and launch +devices. In this case, the same <em>unsigned</em> system image can be shared but +the final <em>signed</em> system images are different (with different +<code>avb.vbmeta-version</code>):</p> + +<img src="../images/treble_vintf_avb_o_p.png"> +<figcaption><strong>Figure 1. </strong>AVB version matches (<code>/system</code> +is P, all other partitions are O).</figcaption> +<br> +<br> +<img src="../images/treble_vintf_avb_p.png"> +<figcaption><strong>Figure 2.</strong> AVB version matches (all partitions are +P).</figcaption> + +<h4><strong>Example:</strong> Successful AVB version match</h4> +<p>The framework compatibility matrix states the following AVB information:</p> + +<pre class="prettyprint"> +<avb> + <vbmeta-version>2.1</vbmeta-version> +</avb> +</pre> + +<p>On the device:</p> + +<pre class="prettyprint"> +ro.boot.avb_version == 1.0 && +ro.boot.vbmeta.avb_version == 2.1 <font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: red; color: white"> mismatch </font> +</pre> + +<pre class="prettyprint"> +ro.boot.avb_version == 2.1 && +ro.boot.vbmeta.avb_version == 3.0 <font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: red; color: white"> mismatch </font> +</pre> + +<pre class="prettyprint"> +ro.boot.avb_version == 2.1 && +ro.boot.vbmeta.avb_version == 2.3 <font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: green; color: white"> match </font> +</pre> + +<pre class="prettyprint"> +ro.boot.avb_version == 2.3 && +ro.boot.vbmeta.avb_version == 2.1 <font style="font-family: Roboto, Arial, Helvetica, sans-serif; background-color: green; color: white"> match </font> +</pre> + + </body> +</html> diff --git a/en/devices/architecture/vintf/objects.html b/en/devices/architecture/vintf/objects.html new file mode 100644 index 00000000..87abeedf --- /dev/null +++ b/en/devices/architecture/vintf/objects.html @@ -0,0 +1,262 @@ +<html devsite> + <head> + <title>VINTF Object Data</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>A VINTF object aggregates data from +<a href="#device-manifest-file">device manifest</a> and +<a href="#framework-manifest-file">framework manifest</a> files (XML) and from +the device itself at <a href="#runtime-data">runtime</a>. Both manifests share a +format, although not all elements apply to both (for details on the schema, see +<a href="#manifest-file-schema">Manifest file schema</a>).</p> + +<h2 id="device-manifest-file">Device manifest file</h2> +<p>The Device manifest file is provided by the device. It lives in the Android +source tree at <code>device/${VENDOR}/${DEVICE}/manifest.xml</code> and on the +device at +<code><a href="https://android.googlesource.com/platform/system/libhidl/+/master/manifest.xml" class="external">/vendor/manifest.xml</a></code>. +</p> + +<p>Example Device manifest:</p> + +<pre class="prettyprint"> +<?xml version="1.0" encoding="UTF-8"?> +<!-- Comments, Legal notices, etc. here --> +<manifest version="1.0" type="device"> + <hal> + <name>android.hardware.camera</name> + <transport>hwbinder</transport> + <version>3.4</version> + <interface> + <name>ICameraProvider</name> + <instance>legacy/0</instance> + <instance>proprietary/0</instance> + </interface> + </hal> + <hal> + <name>android.hardware.nfc</name> + <transport>hwbinder</transport> + <version>1.0</version> + <version>2.0</version> + <interface> + <name>INfc</name> + <instance>nfc_nci</instance> + </interface> + </hal> + <hal> + <name>android.hardware.nfc</name> + <transport>hwbinder</transport> + <version>2.0</version> + <interface> + <name>INfc</name> + <instance>default</instance> + </interface> + </hal> + <hal format="native"> + <name>EGL</name> + <version>1.1</version> + </hal> + <hal format="native"> + <name>GLES</name> + <version>1.1</version> + <version>2.0</version> + <version>3.0</version> + </hal> + <sepolicy> + <version>25.0</version> + </sepolicy> +</manifest> +</pre> + +<h2 id="framework-manifest-file">Framework manifest file</h2> +<p>The Framework manifest file is provided by Google and is manually generated. +It lives in the Android source tree at <code>system/libhidl/manifest.xml</code> +and on the device under <code>/system/manifest.xml</code>.</p> + +<p>Example Framework manifest (provided by Google):</p> + +<pre class="prettyprint"> +<?xml version="1.0" encoding="UTF-8"?> +<!-- Comments, Legal notices, etc. here --> +<manifest version="1.0" type="framework"> + <hal> + <name>android.hidl.allocator</name> + <transport>hwbinder</transport> + <version>1.0</version> + <interface> + <name>IAllocator</name> + <instance>ashmem</instance> + </interface> + </hal> + <hal> + <name>android.hidl.memory</name> + <transport arch="32+64">passthrough</transport> + <version>1.0</version> + <interface> + <name>IMapper</name> + <instance>ashmem</instance> + </interface> + </hal> + <hal> + <name>android.hidl.manager</name> + <transport>hwbinder</transport> + <version>1.0</version> + <interface> + <name>IServiceManager</name> + <instance>default</instance> + </interface> + </hal> + <hal> + <name>android.framework.sensorservice</name> + <transport>hwbinder</transport> + <version>1.0</version> + <interface> + <name>ISensorManager</name> + <instance>default</instance> + </interface> + </hal> +</manifest> +</pre> + +<h2 id="manifest-file-schema">Manifest file schema</h2> +<dl> +<dt><code>?xml</code></dt> +<dd>Optional. Only provides information to the XML parser.</dd> +<dt><code>manifest.version</code></dt> +<dd>Required. Version of <strong>this</strong> manifest. Describes the elements +expected in the manifest. Unrelated to XML version.</dd> +<dt><code>manifest.type</code></dt> +<dd>Required. Type of this manifest. It has value <code>device</code> for +device manifest file and <code>framework</code> for framework manifest +file.</dd> +<dt><code>manifest.hal</code></dt> +<dd>Optional, can repeat. A single HAL (HIDL or native, such as GL), +depending on the <code>format</code> attribute.</dd> +<dt><code>manifest.hal.format</code></dt> +<dd>Optional. Value can be one of: + <ul> + <li><code>hidl</code>: HIDL HALs. This is the default. + <li><code>native</code>: native HALs.</li> + </ul> +</dd> +<dt><code>manifest.hal.name</code></dt> +<dd>Required. Fully-qualified package name of HAL. Multiple HAL entries can use +the same name. Examples: + <ul> + <li><code>android.hardware.camera</code> (HIDL HAL)</li> + <li><code>GLES</code> (native HAL, requires name only)</li> + </ul> + </dd> +<dt><code>manifest.hal.transport</code></dt> +<dd>Required when <code>manifest.hal.format == "hidl"</code>. Must NOT be +present otherwise. States what transport will be used when an interface from +this package is queried from service manager. Value can be one of: + <ul> + <li><code>hwbinder</code>: binderized mode</li> + <li><code>passthrough</code>: passthrough mode</li> + </ul> +</dd> +<dt><code>manifest.hal.transport.arch</code></dt> +<dd>Required for <code>passthrough</code> and must not be present for +<code>hwbinder</code>. Describes the bitness of the passthrough service being +provided. Value can be one of: + <ul> + <li><code>32</code>: 32-bit mode</li> + <li><code>64</code>: 64-bit mode</li> + <li><code>32+64</code>: both</li> + </ul> +</dd> +<dt><code>manifest.hal.version</code></dt> +<dd>Required, can repeat. A version for the <code>hal</code> tags in a +manifest. Format is <code><var>MAJOR</var>.<var>MINOR</var></code>. For +examples, refer to <code>hardware/interfaces</code>, +<code>vendor/${VENDOR}/interfaces</code>, +<code>framework/hardware/interfaces</code>, or<code> +system/hardware/interfaces</code>. +<br><br> +HIDL and native HALs may use multiple version fields as long as they represent +<strong>distinct major versions</strong>, with only one minor version per major +version provided. For example, 3.1 and 3.2 cannot coexist, but 1.0 and 3.4 can. +This applies for all <code>hal</code> elements with the same name.</dd> +<dt><code>manifest.hal.interface</code></dt> +<dd>Required, can repeat without duplicates. State an interface in the +package that has an instance name. There can be multiple +<code><interface></code> elements in a <code><hal></code>; names +must be distinct.</dd> +<dt><code>manifest.hal.interface.name</code></dt> +<dd>Required. Name of the interface.</dd> +<dt><code>manifest.hal.interface.instance</code></dt> +<dd>Required, can repeat. Instance name of the interface. Can have multiple +instances for an interface but no duplicated <code><instance></code> +elements.</dd> +<dt><code>manifest.sepolicy</code></dt> +<dd>Required. Contains all sepolicy-related entries.</dd> +<dt><code>manifest.sepolicy.version</code></dt> +<dd>Required for device manifest. Declares sepolicy version. It has the +format <var>SDK_INT</var>.<var>PLAT_INT</var>.</dd> +</dl> + +<h2 id=runtime-data>Runtime data</h2> +<p>Some information required for the device manifest can be collected only at +runtime. Information is available via +<code>::android::vintf::VintfObject::GetRuntimeInfo()</code> and includes the +following:</p> + +<ul> +<li>Kernel information, including: + <ul> + <li><code>/proc/config.gz</code>. Zipped full kernel configuration that needs + to be read at runtime and converted to a queryable object.</li> + <li><code>/proc/version</code>. Information available through + <code>uname()</code> system call.</li> + <li><code>/proc/cpuinfo</code>. Format may be different for 32-bit and 64-bit + machine.</li> + <li>policydb version + <ul> + <li><code>/sys/fs/selinux/policyvers</code> (assuming <code>selinuxfs</code> + is mounted at <code>/sys/fs/selinux</code>).</li> + <li><code>security_policyvers()</code> API from <code>libselinux</code> gives + you the same.</li> + </ul> + </li> + </ul> +<li>static libavb version, including: + <ul> + <li>bootloader system property: <code>ro.boot.vbmeta.avb_version</code></li> + <li>init/fs_mgr system property: <code>ro.boot.avb_version</code></li> + </ul> +</li> +</ul> + +<h2 id="queryable-api">Queryable API</h2> +<p>The VINTF object is a system API as the +<code>hwservicemanager</code>, OTA update service, CTS <code>DeviceInfo</code>, +and others need information from this API.</p> + +<ul> +<li>C++ queryable API is in +<a href="https://android.googlesource.com/platform/system/libvintf/+/master/include/vintf/VintfObject.h" class="external"><code>android::vintf::VintfObject</code></a></li> +<li>Java queryable API is in +<a href="https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/os/VintfObject.java" class="external"><code>android.os.VintfObject</code></a> +</ul> + + </body> +</html> diff --git a/en/devices/architecture/vintf/resources.html b/en/devices/architecture/vintf/resources.html new file mode 100644 index 00000000..65828508 --- /dev/null +++ b/en/devices/architecture/vintf/resources.html @@ -0,0 +1,184 @@ +<html devsite> + <head> + <title>Additional Resources</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>The following resources provide details on code locations, tools, testing, +licensing, and caveats.</p> + +<h2 id=query-api-code>Queryable code location</h2> +<p>The code for the queryable vendor interface object goes to +<code><a href="https://android.googlesource.com/platform/system/libvintf/+/master#" class="external">system/libvintf</a></code> +(see the +<a href="/devices/architecture/vintf/objects.html#queryable-api">queryable +API</a>).</p> + +<h2 id="related-tools">Tools</h2> +<p>Handwriting manifest files and compatibility matrices can be tough. Use the +following tools to generate a boilerplate manifest/compatibility matrix to start +from.</p> + +<h3 id="lshal">LSHAL</h3> +<p>LSHAL is a device-side tool that lists all registered HALs to +<code>hwservicemanager</code> and all available passthrough implementations +(e.g. <code>android.hardware.foo@1.0-impl.so</code>) on the device. It can also +generate a <strong>device manifest</strong> file based on the list:</p> + +<pre class="devsite-terminal"> +adb shell su 0 /system/bin/lshal --init-vintf +</pre> + +<p>Note the following:</p> +<ol> +<li>If a package is both registered to <code>hwservicemanager</code> and found +as a passthrough HAL, <code><transport></code> is set to +<code>hwbinder</code>.</li> +<li>A dummy <code><sepolicy><version>0.0</version></sepolicy></code> +element exists at the end of the manifest. It is suggested that the element is +deleted and injected via <code>assemble_vintf</code> as explained below.</li> +<li>The generated HAL manifest file may be inaccurate. Human attention is +required to fix inconsistencies between the device manifest and what +<code>vendor.img</code> actually provides.</li> +</ol> + +<h3 id="assemble_vintf">ASSEMBLE_VINTF</h3> +<p><code>assemble_vintf</code> is a host-side tool that:</p> +<ol> +<li>Verifies a compatibility matrix or manifest file is valid.</li> +<li>Injects variables to manifests/compatibility matrices available at build +time and generates a new file that should be installed to the device.</li> +<li>Checks compatibility between the generated file and its dual.</li> +<li>If a manifest file is given, optionally generates a boilerplate +compatibility matrix that is compatible with the manifest file.</li> +</ol> + +<h4><strong>Example:</strong> Generate <strong>device compatibility +matrix</strong> from a framework manifest file</h4> + +<pre class="devsite-terminal"> +assemble_vintf -m \ + -i system/libhidl/manifest.xml \ + -o device/manufacturer/device_name/compatibility_matrix.xml +</pre> +<p>Note the following:</p> +<ul> +<li>Even though <code><vndk></code> entries are in the output +compatibility matrix, they should be deleted and injected at build time.</li> +<li>All HALs are set to <code>optional="true"</code>.</li> +</ul> + +<h4><strong>Example:</strong> Generate a skeleton framework compatibility +matrix from a device manifest file</h4> + +<pre class="devsite-terminal"> +BOARD_SEPOLICY_VERS=10000.0 assemble_vintf -m \ + -i device/foo/bar/manifest.xml + -o path/to/place/output/compatibility_matrix.xml +</pre> +<p>Note the following:</p> +<ul> +<li>Even though <code><sepolicy></code> and <code><avb></code> are +in the output compatibility matrix, they should be deleted and injected at +build time.</li> +<li>All HALs are set to <code>optional="true"</code>.</li> +</ul> + +<h4><strong>Example:</strong> Generate XML files from variables</h4> + +<p>At build time, if the following variables are +defined in <code>device/manufacturer/device_name/BoardConfig.mk</code>:</p> + +<pre class="prettyprint"> +DEVICE_MANIFEST_FILE := \ + device/manufacturer/device_name/manifest.xml +DEVICE_MATRIX_FILE := \ + device/manufacturer/device_name/compatibility_matrix.xml +</pre> + +<p>Then the following commands (modified to omit implementation details) are +executed to generate all XML files:</p> + +<pre class="prettyprint"> +# device manifest; only when DEVICE_MANIFEST_FILE is set +BOARD_SEPOLICY_VERS=10000.0 assemble_vintf \ + -i device/manufacturer/device_name/manifest.xml \ + -o $(TARGET_OUT_VENDOR)/manifest.xml + +# device compatibility matrix; only when DEVICE_MATRIX_FILE is set +assemble_vintf \ + -i device/manufacturer/device_name/compatibility_matrix.xml \ + -o $(TARGET_OUT_VENDOR)/compatibility_matrix.xml + +# framework manifest +assemble_vintf + -i system/libhidl/manifest.xml \ + -o $(TARGET_OUT)/manifest.xml \ + -c $(TARGET_OUT_VENDOR)/compatibility_matrix.xml + +# framework compatibility matrix +BOARD_SEPOLICY_VERS=$(BOARD_SEPOLICY_VERS) \ +POLICYVERS=$(POLICYVERS) \ +BOARD_AVB_VBMETA_VERSION=$(BOARD_AVB_VBMETA_VERSION) +assemble_vintf \ + -i hardware/interfaces/compatibility_matrix.xml \ + -o $(TARGET_OUT)/compatibility_matrix.xml \ + -c $(TARGET_OUT_VENDOR)/manifest.xml \ +</pre> + +<p>For details, see:</p> + +<pre class="devsite-terminal">assemble_vintf --help</pre> + +<h2 id="testing">Testing</h2> +<p>The <code>platform/system/libvintf</code> project uses +<a href="https://github.com/google/googletest" class="external">GTest</a> for +the serialization, deserialization, and compatibility checking.</p> + +<h2 id="licensing">Licensing</h2> +<ul> +<li><code>tinyxml2</code> (external/tinyxml2) for serializing/deserializing the +object to/from XML. BSD-like license.</li> +<li><code>libselinux</code> (external/selinux/libselinux) for getting policydb +version. Public domain license.</li> +<li><code>libz</code> (external/zlib) for decompressing +<code>/proc/config.gz</code>. BSD-like license.</li> +<li><code>libvintf</code> project uses Apache 2.0 license (with appropriate +MODULE_LICENSE_APACHE2 and NOTICE files).</li> +</ul> + +<h2 id="caveats">Caveats</h2> +<p>It is also possible to determine the HALs at runtime by querying +<code>hwservicemanager</code> (as <code>lshal</code> does). However:</p> +<ul> +<li><code>hwservicemanager</code> does not list passthrough services.</li> +<li>If a service has just crashed and is restarting, it may be missing from the +query result.</li> +<li>It doesn't work for hot pluggable services.</li> +<li><code>hwservicemanager</code> is not available in recovery mode (see +below).</li> +</ul> + +<p>In recovery mode, the API to retrieve the vendor interface object must still +be available to allow the device to check the vendor interface against the +compatibility matrix again.</p> + + </body> +</html> diff --git a/en/devices/architecture/vndk/deftool.html b/en/devices/architecture/vndk/deftool.html new file mode 100644 index 00000000..bfed22c4 --- /dev/null +++ b/en/devices/architecture/vndk/deftool.html @@ -0,0 +1,433 @@ +<html devsite> + <head> + <title>VNDK Definition Tool</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + + +<p> +The VNDK definition tool helps vendors migrate their source tree to an Android O +environment. This tool scans binary files in the system and vendor images then +resolves dependencies. Based on the module dependency graph, the tool can also +detect violations to VNDK concepts and provide insight/suggestions for moving +modules between partitions. If an AOSP system image is specified, the VNDK +definition tool can compare your system image with the AOSP system image and +determine the extended libraries. +</p> +<p> +This section covers three frequently used commands for the VNDK definition tool: +</p> +<ul> +<li><code>vndk</code>. Compute VNDK_SP_LIBRARIES, VNDK_SP_EXT_LIBRARIES, and +EXTRA_VENDOR_LIBRARIES for build system workaround in Android O.</li> +<li><code>check-dep</code>. Check the violating module dependencies from vendor +modules to non-eligible framework shared libraries.</li> +<li><code>deps</code>. Print the dependencies between the shared libraries and +executables.</li> +</ul> + +<p>For more details on advanced command usage, refer to +<a href="https://android.googlesource.com/platform/development/+/master/vndk/tools/definition-tool/README.md" class="external">README.md</a> +file in the VNDK Definition Tool repository.</p> + +<h2 id="vndk">vndk</h2> +<p>The <code>vndk</code> subcommand loads the shared libraries and executables +from the system partition and vendor partitions, then resolves module +dependencies to determine the libraries that must be copied to +<code>/system/lib[64]/vndk-sp</code> and <code>/vendor/lib[64]</code>. Options +for the <code>vndk</code> subcommand include:</p> + +<table> + <tr> + <th>Option</th> + <th>Description</th> + </tr> + <tr> + <td><code>--system</code> + </td> + <td>Point to a directory containing the files that will reside in the system +partition. + </td> + </tr> + <tr> + <td><code>--vendor</code> + </td> + <td>Point to a directory containing the files that will reside in a vendor +partition. + </td> + </tr> + <tr> + <td><code>--aosp-system</code> + </td> + <td>Point to a directory containing the files that will reside in the AOSP +system image. + </td> + </tr> + <tr> + <td><code>--load-extra-deps</code> + </td> + <td>Point to a file that describes the implicit dependencies, such as +<code>dlopen()</code>. + </td> + </tr> +</table> + +<p>For example, to compute the VNDK library sets, run the following +<code>vndk</code> subcommand:</p> + +<pre class="prettyprint"> +<code class="devsite-terminal">./vndk_definition_tool.py vndk \</code> + --system ${ANDROID_PRODUCT_OUT}/system \ + --vendor ${ANDROID_PRODUCT_OUT}/vendor \ + --aosp-system ${ANDROID_PRODUCT_OUT}/../generic_arm64_ab/system\ + --load-extra-deps dlopen.dep +</pre> + +<p>Specify extra dependencies with a simple file format. Each line represents a +relationship, with the file before the colon depending on the file after the +colon. For example:</p> + +<pre class="prettyprint">/system/lib/libart.so: /system/lib/libart-compiler.so</pre> + +<p>This line lets the VNDK definition tool know that <code>libart.so</code> +depends on <code>libart-compiler.so</code>. +</p> + +<h3 id="installation-destination">Installation destination</h3> +<p>VNDK definition tool lists libraries and corresponding install directories +for the following categories:</p> + +<table> + <tr> + <th>Category</th> + <th>Directory</th> + </tr> + <tr> + <td>vndk_sp + </td> + <td>Must install to <code>/system/lib[64]/vndk-sp</code> + </td> + </tr> + <tr> + <td>vndk_sp_ext + </td> + <td>Must install to <code>/vendor/lib[64]/vndk-sp</code> + </td> + </tr> + <tr> + <td>extra_vendor_libs + </td> + <td>Must install to <code>/vendor/lib[64]</code> + </td> + </tr> +</table> + +<h3 id="build-system-templates">Build system templates</h3> +<p>After gathering outputs from VNDK definition tool, a vendor can create an +<code>Android.mk</code> and fill in <code>VNDK_SP_LIBRARIES</code>, +<code>VNDK_SP_EXT_LIBRARIES</code> and <code>EXTRA_VENDOR_LIBRARIES</code> to +automate the process to copy libraries to the designated installation +destination.</p> + +<pre class="prettyprint">ifneq ($(filter $(YOUR_DEVICE_NAME),$(TARGET_DEVICE)),) +VNDK_SP_LIBRARIES := ##_VNDK_SP_## +VNDK_SP_EXT_LIBRARIES := ##_VNDK_SP_EXT_## +EXTRA_VENDOR_LIBRARIES := ##_EXTRA_VENDOR_LIBS_## + +#------------------------------------------------------------------------------- +# VNDK Modules +#------------------------------------------------------------------------------- +LOCAL_PATH := $(call my-dir) + +define define-vndk-lib +include $$(CLEAR_VARS) +LOCAL_MODULE := $1.$2 +LOCAL_MODULE_CLASS := SHARED_LIBRARIES +LOCAL_PREBUILT_MODULE_FILE := $$(TARGET_OUT_INTERMEDIATE_LIBRARIES)/$1.so +LOCAL_STRIP_MODULE := false +LOCAL_MULTILIB := first +LOCAL_MODULE_TAGS := optional +LOCAL_INSTALLED_MODULE_STEM := $1.so +LOCAL_MODULE_SUFFIX := .so +LOCAL_MODULE_RELATIVE_PATH := $3 +LOCAL_VENDOR_MODULE := $4 +include $$(BUILD_PREBUILT) + +ifneq ($$(TARGET_2ND_ARCH),) +ifneq ($$(TARGET_TRANSLATE_2ND_ARCH),true) +include $$(CLEAR_VARS) +LOCAL_MODULE := $1.$2 +LOCAL_MODULE_CLASS := SHARED_LIBRARIES +LOCAL_PREBUILT_MODULE_FILE := $$($$(TARGET_2ND_ARCH_VAR_PREFIX)TARGET_OUT_INTERMEDIATE_LIBRARIES)/$1.so +LOCAL_STRIP_MODULE := false +LOCAL_MULTILIB := 32 +LOCAL_MODULE_TAGS := optional +LOCAL_INSTALLED_MODULE_STEM := $1.so +LOCAL_MODULE_SUFFIX := .so +LOCAL_MODULE_RELATIVE_PATH := $3 +LOCAL_VENDOR_MODULE := $4 +include $$(BUILD_PREBUILT) +endif # TARGET_TRANSLATE_2ND_ARCH is not true +endif # TARGET_2ND_ARCH is not empty +endef + +$(foreach lib,$(VNDK_SP_LIBRARIES),\ + $(eval $(call define-vndk-lib,$(lib),vndk-sp-gen,vndk-sp,))) +$(foreach lib,$(VNDK_SP_EXT_LIBRARIES),\ + $(eval $(call define-vndk-lib,$(lib),vndk-sp-ext-gen,vndk-sp,true))) +$(foreach lib,$(EXTRA_VENDOR_LIBRARIES),\ + $(eval $(call define-vndk-lib,$(lib),vndk-ext-gen,,true))) + + +#------------------------------------------------------------------------------- +# Phony Package +#------------------------------------------------------------------------------- + +include $(CLEAR_VARS) +LOCAL_MODULE := $(YOUR_DEVICE_NAME)-vndk +LOCAL_MODULE_TAGS := optional +LOCAL_REQUIRED_MODULES := \ + $(addsuffix .vndk-sp-gen,$(VNDK_SP_LIBRARIES)) \ + $(addsuffix .vndk-sp-ext-gen,$(VNDK_SP_EXT_LIBRARIES)) \ + $(addsuffix .vndk-ext-gen,$(EXTRA_VENDOR_LIBRARIES)) +include $(BUILD_PHONY_PACKAGE) + +endif # ifneq ($(filter $(YOUR_DEVICE_NAME),$(TARGET_DEVICE)),) +</pre> + +<h2 id="check-dep">check-dep</h2> +<p>The <code>check-dep</code> subcommand scans vendor modules and checks their +dependencies. If it detects violations, it prints the violating dependant +library and symbol usages:</p> + +<pre class="prettyprint"> +<code class="devsite-terminal">./vndk_definition_tool.py check-dep \</code> + --system ${ANDROID_PRODUCT_OUT}/system \ + --vendor ${ANDROID_PRODUCT_OUT}/vendor \ + --tag-file eligible-list.csv \ + --module-info ${ANDROID_PRODUCT_OUT}/module-info.json \ + 1> check_dep.txt \ + 2> check_dep_err.txt +</pre> + +<p>For example, the following sample output shows a violating dependency from +<code>libRS_internal.so</code> to <code>libmediandk.so</code>:</p> + +<pre class="prettyprint"> +/system/lib/libRS_internal.so + MODULE_PATH: frameworks/rs + /system/lib/libmediandk.so + AImageReader_acquireNextImage + AImageReader_delete + AImageReader_getWindow + AImageReader_new + AImageReader_setImageListener +</pre> + +<p>Options for the <code>check-dep</code> subcommand include:</p> + +<table> + <tr> + <th style="width:25%">Option</th> + <th>Description</th> + </tr> + <tr> + <td><code>--tag-file</code> + </td> + <td>Must refer to an eligible library tag file (described below), which is a +Google-provided spreadsheet that described categories of framework shared +libraries. + </td> + </tr> + <tr> + <td><code>--module-info</code> + </td> + <td>Points to the <code>module-info.json</code> generated by Android build +system. It helps the VNDK definition tool associate binary modules with source +code. + </td> + </tr> +</table> + +<h3 id="eligible-library-tag-file">Eligible library tag file</h3> +<p>Google provides an eligible VNDK spreadsheet (e.g. +<code>eligible-list.csv</code>) that tags the framework shared libraries that +can be used by vendor modules:</p> + +<table> + <tr> + <th style="width:25%">Tag</th> + <th>Description</th> + </tr> + <tr> + <td>LL-NDK</td> + <td>Low-level NDK libraries that can be used by both framework and vendor +modules.</td> + </tr> + <tr> + <td>LL-NDK-Indirect</td> + <td>Private dependencies of LL-NDK libraries. Vendor modules must not access +these libraries directly.</td> + </tr> + <tr> + <td>SP-NDK</td> + <td>Same-process NDK libraries that can be used by both framework and vendor +modules.</td> + </tr> + <tr> + <td>SP-NDK-Indirect</td> + <td>Private dependencies of SP-NDK libraries. Vendor modules must not access +these libraries directly.</td> + </tr> + <tr> + <td>VNDK-SP</td> + <td>SP-HAL framework shared libraries dependencies.</td> + </tr> + <tr> + <td>VNDK-SP-Indirect</td> + <td>VNDK-SP dependencies that are not directly accessible to SP-HAL, but can +be accessed by other vendor modules (except SP-HAL and SP-HAL-Dep)</td> + </tr> + <tr> + <td>VNDK-SP-Indirect-Private</td> + <td>VNDK-SP dependencies that are not directly accessible to all vendor +modules.</td> + </tr> + <tr> + <td>VNDK</td> + <td>Framework shared libraries that are available to vendor modules (except +SP-HAL and SP-HAL-Dep).</td> + </tr> + <tr> + <td>FWK-ONLY</td> + <td>Framework-only shared libraries that must not be accessed by vendor +modules (neither directly nor indirectly).</td> + </tr> + <tr> + <td>FWK-ONLY-RS</td> + <td>Framework-only shared libraries that must not be accessed by vendor +modules (except for RS usages).</td> + </tr> +</table> + +<p>The following table describes tags used for vendor shared libraries:</p> + +<table> + <tr> + <th style="width:25%">Tag</th> + <th>Description</th> + </tr> + <tr> + <td>SP-HAL</td> + <td>Same-process HAL implementation shared libraries.</td> + </tr> + <tr> + <td>SP-HAL-Dep</td> + <td>SP-HAL vendor shared libraries dependencies (a.k.a. SP-HAL dependencies +excluding LL-NDK, SP-NDK, and VNDK-SP)</td> + </tr> + <tr> + <td>VND-ONLY</td> + <td>Framework-invisible shared libraries that must not be accessed by +framework modules. The copied extended VNDK libraries will be tagged as VND-ONLY +as well.</td> + </tr> +</table> + +<p>Relationships between tags:</p> + +<img src="../images/treble_vndk_design.png"> +<figcaption><strong>Figure 1.</strong> Relationships between tags.</figcaption> + +<h2 id="deps">deps</h2> +<p>To debug the library dependencies, the <code>deps</code> subcommand prints +the module dependencies:</p> + +<pre class="prettyprint"> +<code class="devsite-terminal">./vndk_definition_tool.py deps \</code> + --system ${ANDROID_PRODUCT_OUT}/system \ + --vendor ${ANDROID_PRODUCT_OUT}/vendor +</pre> + +<p>The output consists of multiple lines. The line without a tab character +starts a new section. The line with a tab character depends on the preceding +section. For example:</p> + +<pre class="prettyprint"> +/system/lib/ld-android.so +/system/lib/libc.so + /system/lib/libdl.so +</pre> + +<p>This output shows that <code>ld-android.so</code> does not have a dependency +and <code>libc.so</code> depends on <code>libdl.so</code>.</p> + +<p>When specifying the <code>--revert</code> option, <code>deps</code> +subcommand prints the <strong>usages of libraries</strong> (reversed +dependencies):</p> + +<pre class="prettyprint"> +<code class="devsite-terminal">./vndk_definition_tool.py deps \</code> + --revert \ + --system ${ANDROID_PRODUCT_OUT}/system \ + --vendor ${ANDROID_PRODUCT_OUT}/vendor</pre> + +<p>For example:</p> + +<pre class="prettyprint"> +/system/lib/ld-android.so + /system/lib/libdl.so + </pre> + +<p>This output shows that <code>ld-android.so</code> is used by +<code>libdl.so</code>, or in other words, <code>libdl.so</code> depends on +<code>ld-android.so</code>. In addition, this output shows that +<code>libdl.so</code> is the sole user of <code>ld-android.so</code>.</p> + +<p>When specifying the <code>--symbol</code> option, the <code>deps</code> +subcommand prints the symbols being used:</p> + +<pre class="prettyprint"> +<code class="devsite-terminal">./vndk_definition_tool.py deps \</code> + --symbol \ + --system ${ANDROID_PRODUCT_OUT}/system \ + --vendor ${ANDROID_PRODUCT_OUT}/vendor + </pre> + +<p>For example:</p> + +<pre class="prettyprint"> +/system/lib/libc.so + /system/lib/libdl.so + android_get_application_target_sdk_version + dl_unwind_find_exidx + dlclose + dlerror + dlopen + dlsym +</pre> + +<p>This output shows that <code>libc.so</code> depends on 6 functions exported +from <code>libdl.so</code>. If both the <code>--symbol</code> option and +<code>--revert</code> option are specified, the symbols used by the user +will be printed.</p> + + </body> +</html> diff --git a/en/devices/architecture/vndk/dir-rules-sepolicy.html b/en/devices/architecture/vndk/dir-rules-sepolicy.html new file mode 100644 index 00000000..0cc2ceae --- /dev/null +++ b/en/devices/architecture/vndk/dir-rules-sepolicy.html @@ -0,0 +1,189 @@ +<html devsite> + <head> + <title>Directories, Rules, and sepolicy</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>This page describes the directory layout for devices running Android O, VNDK +rules, and associated sepolicy.</p> + +<h2 id="directory">Directory layout</h2> +<p>The <em>Degenerated Directory Layout</em> consists of the following +directories:</p> +<ul> +<li><code>/system/lib[64]</code> contains all framework shared libraries, +including LL-NDK, SP-NDK, VNDK, and framework-only libraries (including +LL-NDK-Indirect, SP-NDK-Indirect, and some libraries with the same names as the +ones in VNDK-SP).</li> +<li><code>/system/lib[64]/vndk-sp</code> contains VNDK-SP libraries for +same-process HALs.</li> +<li><code>/vendor/lib[64]</code> contains the extended VNDK libraries (either +DXUA or DXUX VNDK libraries), same-process HAL implementations, and other vendor +shared libraries.</li> +<li><code>/vendor/lib[64]/vndk-sp</code> may contain extra libraries that are +used by VNDK-SP libraries.</li> +</ul> + +<p>Vendor modules load the VNDK libraries from <code>/system/lib[64]</code>.</p> + +<h2 id="rules">VNDK rules</h2> +<p>This section provides a comprehensive list of VNDK rules:</p> + +<ul> +<li>Framework processes must not load non-SP-HAL shared libraries from vendor +partitions (not strictly enforced in Android O but will be in a future release). +</li> +<li>Vendor processes must not load non-LL-NDK, non-SP-NDK, non-VNDK-SP, and +non-VNDK libraries from the system partition. (not strictly enforced in Android +O but will be in a future release).</li> + +<p class="note"><strong>NOTE</strong>: To benefit from the framework-only OTA +beyond Android O, this rule must not be violated in devices launched with +Android O.</p> + +<li>Installed VNDK libraries must be a subset of Google-defined eligible VNDK +libraries.</li> +<li>The outer dependencies of SP-HAL and SP-HAL-Dep must be restricted to +LL-NDK, SP-NDK, or Google-defined VNDK-SP libraries. + <ul> + <li>The dependencies of an SP-HAL shared library must be restricted to LL-NDK + libraries, SP-NDK libraries, SP-NDK libraries, Google-defined VNDK-SP + libraries, other SP-HAL libraries, and/or other vendor shared libraries that + can be labeled as SP-HAL-Dep libraries.</li> + <li>A vendor shared library can be labeled as a SP-HAL-Dep library only if it + is not an AOSP library and its dependencies are restricted to LL-NDK libraries, + SP-NDK libraries, Google-defined VNDK-SP libraries, SP-HAL libraries, and/or + other SP-HAL-Dep libraries.</li> + </ul> +</li> +<li>VNDK-SP must be self-contained. <code>libRS_internal.so</code> gets special +treatment in Android O, but will be revisited in a future release.</li> +<li>No framework-vendor communication through non-HIDL interfaces, including +(but not limited to) binder, sockets, shared memories, files, etc.</li> +<li>The size of the system partition must be large enough to contain two copies +of all eligible VNDK libraries and a copy of ineligible framework shared +libraries.</li> +</ul> + +<h2 id="sepolicy">sepolicy</h2> +<p>Framework processes described in this section correspond to +<code>coredomain</code> in sepolicies while vendor processes correspond to +<code>non-coredomain</code>. For example, <code>/dev/binder</code> can be +accessed only in <code>coredomain</code> and <code>/dev/vndbinder</code> can be +accessed only in non-<code>coredomain</code>.</p> + +<p>Similar policies restrict the access to the shared libraries on system and +vendor partitions. The following table shows the rights to access shared +libraries of different categories:</p> + +<table> + <tr> + <th style="width:35%">Category</th> + <th>Partition</th> + <th>Accessible from<br>coredomain</th> + <th>Accessible from<br>non-coredomain</th> + </tr> + <tr> + <td>LL-NDK</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>LL-NDK-Indirect</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>SP-NDK</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>SP-NDK-Indirect</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>VNDK-SP/VNDK-SP-Indirect/VNDK-SP-Indirect-Private</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>VNDK-SP-Ext/VNDK-SP-Indirect-Ext</td> + <td>Vendor</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>VNDK</td> + <td>System</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>VNDK-Ext</td> + <td>Vendor</td> + <td>N</td> + <td>Y</td> + </tr> + <tr> + <td>FWK-ONLY</td> + <td>System</td> + <td>Y</td> + <td>N</td> + </tr> + <tr> + <td>FWK-ONLY-RS</td> + <td>System</td> + <td>Y</td> + <td>N</td> + </tr> + <tr> + <td>SP-HAL</td> + <td>Vendor</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>SP-HAL-Dep</td> + <td>Vendor</td> + <td>Y</td> + <td>Y</td> + </tr> + <tr> + <td>VND-ONLY</td> + <td>Vendor</td> + <td>N</td> + <td>Y</td> + </tr> +</table> + +<p>LL-NDK-Indirect, SP-NDK-Indirect, VNDK-SP-Indirect, and +VNDK-SP-Indirect-Private must be accessible from both domains because +non-<code>coredomain</code> will indirectly access them. Similarly, SP-HAL-Dep +must be accessible from <code>coredomain</code> because SP-HAL relies on it.</p> + + </body> +</html> diff --git a/en/devices/architecture/vndk/extensions.html b/en/devices/architecture/vndk/extensions.html new file mode 100644 index 00000000..2cc895c7 --- /dev/null +++ b/en/devices/architecture/vndk/extensions.html @@ -0,0 +1,150 @@ +<html devsite> + <head> + <title>VNDK Extensions</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Android device manufacturers change the source code of AOSP libraries for +various reasons. Some vendors reimplement functions in AOSP libraries to +boost the performance while other vendors add new hooks, new APIs, or new +functionalities to AOSP libraries. This section provides guidelines for +extending AOSP libraries in a way that does not break CTS/VTS.</p> + +<h2 id="drop-in-replacement">Drop-in replacement</h2> +<p>All modified shared libraries must be <strong>binary-compatible</strong>, +<strong>drop-in replacements</strong> of their AOSP counterpart. All existing +AOSP users must be able to use the modified shared library without +recompilations. This requirement implies the following:</p> +<ul> +<li>AOSP functions must not be removed.</li> +<li>Structures must not be altered if such structures are exposed to their +users.</li> +<li>Pre-condition of functions must not be strengthened.</li> +<li>Functions must provide equivalent functionalities.</li> +<li>Post-condition of functions must not be weakened.</li> +</ul> + +<h2 id="extended-module-classifications">Extended module classifications</h2> +<p>Classify modules by the functionalities they <strong>define</strong> and +<strong>use</strong>.</p> +<p class="note"><strong>Note</strong>: <em>Functionalities</em> is used here +instead of API/ABI because it is possible to add functionality without changing +any API/ABI.</p> + +<p>Depending on the functionalities defined in a module, modules can be +classified into <strong>DA-Module</strong> and <strong>DX-Module</strong>:</p> +<ul> +<li><em>Defining-only-AOSP Modules (DA-Module)</em> do not define new +functionalities which were not in the AOSP counterpart. + <ul> + <li><em>Example 1.</em> An intact unmodified AOSP library is a DA-Module.</li> + <li><em>Example 2.</em> If a vendor rewrites the functions in + <code>libcrypto.so</code> with SIMD instructions (without adding new + functions), then the modified <code>libcrypto.so</code> will be a DA-Module. + </li> + </ul> +</li> +<li><em>Defining-Extension Modules (DX-Module)</em> either define new +functionalities or do not have an AOSP counterpart. + <ul> + <li><em>Example 1.</em> If a vendor adds a helper function to + <code>libjpeg.so</code> to access some internal data, then the modified + <code>libjpeg.so</code> will be a DX-Lib and the newly added function will be + the extended portion of the library.</li> + <li><em>Example 2.</em> If a vendor defines a non-AOSP library named + <code>libfoo.so</code>, then <code>libfoo.so</code> will be a DX-Lib.</li> + </ul> +</li> +</ul> + +<p>Depending on the functionalities used by a module, modules can be classified +into <strong>UA-Module</strong> and <strong>UX-Module</strong>.</p> +<ul> +<li><em>Using-only-AOSP Modules (UA-Module)</em> only use AOSP functionalities +in their implementations. They do not rely on any non-AOSP extensions. + <ul> + <li><em>Example 1.</em> An intact unmodified AOSP library is an UA-Module.</li> + <li><em>Example 2.</em> If a modified shared library <code>libjpeg.so</code> + only relies on other AOSP APIs, then it will be an UA-Module.</li> + </ul> +</li> +<li><em>Using-Extension Modules (UX-Module)</em> rely on some non-AOSP +functionalities in their implementations. + <ul> + <li><em>Example 1.</em> If a modified <code>libjpeg.so</code> relies on another + non-AOSP library named <code>libjpeg_turbo2.so</code>, then the modified + <code>libjpeg.so</code> will be an UX-Module.</li> + <li><em>Example 2.</em> If a vendor adds a new function to their modified + <code>libexif.so</code> and their modified <code>libjpeg.so</code> uses the + newly added function from <code>libexif.so</code>, then their modified + <code>libjpeg.so</code> will be an UX-Module.</li> + </ul> +</li> +</ul> + +<p>Definitions and usages are independent from each other:</p> +<table> + <tr> + <td rowspan="2" colspan="2" class="columns"></td> + <th colspan="2">Used Functionalities</th> + </tr> + <tr> + <td>Only AOSP (UA)</td> + <td>Extended (UX)</td> + </tr> + <tr> + <th rowspan="2">Defined Functionalities</th> + <td>Only AOSP (DA)</td> + <td>DAUA</td> + <td>DAUX</td> + </tr> + <tr> + <td>Extended (DX)</td> + <td>DXUA</td> + <td>DXUX</td> + </tr> +</table> + +<h2 id="vndk-extension-mechanism">VNDK extension mechanism</h2> +<p> +Vendor modules that rely on extended functionalities won't work because the AOSP +library with the same name does not have the extended functionality. If vendor +modules directly or indirectly depend on extended functionalities, vendors +should copy DAUX, DXUA, and DXUX shared libraries to the vendor partition +(vendor processes always look for shared libraries in the vendor partition +first). However, LL-NDK and SP-NDK libraries must not be copied, so vendor +modules must not rely on the extended functionalities defined by the modified +LL-NDK and SP-NDK libraries. +</p> +<p> +DAUA shared libraries can remain on the system partition if the corresponding +AOSP library can provide the same functionality and vendor modules continue to +work when the system partition is overwritten by an AOSP system image. +</p> +<p> +Drop-in replacement is important because the unmodified VNDK libraries in the +AOSP system image will link with the modified shared libraries on name +collision. If the AOSP libraries are modified in an API/ABI incompatible manner, +the AOSP libraries in the AOSP system image might fail to link or result in +undefined behaviors. +</p> + + </body> +</html> diff --git a/en/devices/architecture/vndk/index.html b/en/devices/architecture/vndk/index.html new file mode 100644 index 00000000..a0d4af59 --- /dev/null +++ b/en/devices/architecture/vndk/index.html @@ -0,0 +1,267 @@ +<html devsite> + <head> + <title>Vendor Native Development Kit (VNDK)</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>The Vendor Native Development Kit (VNDK) is a set of libraries +exclusively for vendors to implement their HALs. The VNDK ships in +<code>system.img</code> and is dynamically linked to vendor code at runtime.</p> + +<h2 id=why-vndk>Why VNDK?</h2> +<p>Android O enables framework-only updates in which the system partition can be +upgraded to the latest version while vendor partitions are left unchanged. This +implies that binaries built at different times must be able to work with each +other; VNDK covers API/ABI changes across Android releases.</p> + +<p>Framework-only updates include the following challenges:</p> + +<ul> +<li><strong>Dependency between framework modules and vendor modules</strong>. +Before Android O, modules from both sides could link with modules from the other +side. However, dependencies from vendor modules imposed undesired restrictions +to framework modules development.</li> +<li><strong>Extensions to AOSP libraries</strong>. Android O requires all +Android devices to pass CTS when the system partition is replaced with a +standard AOSP system image. However, as vendors extend AOSP libraries to boost +performance or to add extra functionalities for their HIDL implementations, +flashing the system partition with a standard AOSP system image might break a +vendor's HIDL implementation. (For guidelines on preventing such breakages, see +<a href="/devices/architecture/vndk/extensions.html">VNDK extensions</a>.)</li> +</ul> + +<p>To address these challenges, Android O introduces several techniques such as +VNDK (described in this section), +<a href="/devices/architecture/hidl/index.html">HIDL</a>, hwbinder, +<a href="/devices/architecture/dto/index.html">device tree overlay</a>, and +sepolicy overlay.</p> + +<h2 id=resources>VNDK resources</h2> +<p>This section includes the following VNDK resources:</p> +<ul> +<li><em><a href="#vndk-concepts">VNDK concepts</a></em> (below) describes +framework shared libraries, same-process HALs (SP-HALs), and VNDK terminology. +</li> +<li><em><a href="/devices/architecture/vndk/extensions.html">VNDK +extensions</a></em> classifies vendor-specific changes into categories. For +example, libraries with extended functionalities on which vendor modules rely +must be copied into the vendor partition, but ABI-incompatible changes are +prohibited.</li> +<li>The <em><a href="/devices/architecture/vndk/deftool.html">VNDK Definition +Tool</a></em> helps migrate your source tree to Android O.</li> +<li>The <em><a href="/devices/architecture/vndk/linker-namespace.html">Linker +Namespace</a></em> provides fine-grained control over shared library linkages. +</li> +<li><em><a href="/devices/architecture/vndk/dir-rules-sepolicy.html">Directories, +Rules, and sepolicy</a></em> defines the directory structure for devices running +Android O, VNDK rules, and associated sepolicy.</li> +<li>The <em><a href="../images/vndk_design_android_o.pdf">VNDK Design in Android +O</a></em> presentation illustrates fundamental VDNK concepts used in Android +O.</li> +</ul> + +<h2 id="concepts">VNDK concepts</h2> +<p>In an ideal Android O world, framework processes do not load vendor shared +libraries, all vendor processes load only vendor shared libraries (and a portion +of framework shared libraries), and communications between framework processes +and vendor processes are governed by HIDL and hardware binder.</p> + +<p>Such a world includes the possibility that stable, public APIs from +framework shared libraries might not be sufficient for vendor module developers +(although APIs can change between Android releases), requiring that some portion +of framework shared libraries be accessible to vendor processes. In addition, as +performance requirements can lead to compromises, some response-time-critical +HALs must be treated differently.</p> + +<p>The following sections detail how VNDK handles framework shared libraries for +vendors and Same-Process HALs (SP-HALs).</p> + +<h3 id="framework-shared-libraries">Framework shared libraries for vendor</h3> +<p>This section describes the criteria for classifying shared libraries that are +accessible to vendor processes. There are two approaches to support vendor +modules across multiple Android releases:</p> + +<ol> +<li><strong>Stabilize the ABIs/APIs of the framework shared libraries</strong>. +New framework modules and old vendor modules can use the same shared library to +reduce memory footprint and storage size. A unique shared library also avoids +several double-loading issues. However, the development cost to maintain stable +ABIs/APIs is high and it is unrealistic to stabilize all ABIs/APIs exported by +every framework shared library.</li> +<li><strong>Copy old framework shared libraries</strong>. Comes with the strong +restriction against side channels, defined as all mechanisms to communicate +among framework modules and vendor modules, including (but not limited to) +binder, socket, pipe, shared memory, shared file, and system properties. There +must be no communication unless the communication protocol is frozen and stable +(e.g. HIDL through hwbinder). Double-loading shared libraries might cause +problems as well; for example, if an object created by the new library is passed +into the functions from the old library, an error may occur as these libraries +may interpret the object differently.</li> +</ol> + +<p>Different approaches are used depending on the characteristics of the shared +libraries. As a result, framework shared libraries are classified into three +sub-categories:</p> + +<ul> +<li><em>LL-NDK</em> and <em>SP-NDK</em> are <em>Framework Shared Libraries</em> +that are known to be stable. Their developers are committed to maintain their +API/ABI stabilities. + <ul> + <li>LL-NDK includes the following libraries: <code>libandroid_net.so</code>, + <code>libc.so</code>, <code>libstdc++.so</code>, <code>libdl.so</code>, + <code>liblog.so</code>, <code>libm.so</code>, <code>libz.so</code>, and + <code>libvndksupport.so</code>.</li> + <li>SP-NDK includes the following libraries: <code>libEGL.so</code>, + <code>libGLESv1_CM.so</code>, <code>libGLESv2.so</code>, + <code>libGLESv3.so</code>, <code>libvulkan.so</code>, + <code>libnativewindow.so</code>, and <code>libsync.so</code>.</li> + </ul> +</li> +<li><em>Eligible VNDK Libraries (VNDK)</em> are <em>Framework Shared +Libraries</em> that are safe to be copied twice. <em>Framework Modules</em> and +<em>Vendor Modules</em> can link with their own copies. A framework shared +library can become an eligible VNDK library only if it satisfies the following +criteria: + <ul> + <li>It does not send/receive IPCs to/from the framework.</li> + <li>It is not related to ART virtual machine.</li> + <li>It does not read/write files/partitions with unstable file formats.</li> + <li>It does not have special software license which requires legal reviews.</li> + <li>Its code owner does not have objections to vendor usages.</li> + </ul> +</li> +<li><em>Framework-Only Libraries (FWK-ONLY)</em> are <em>Framework Shared +Libraries</em> that do not belong to the categories mentioned above. These +libraries: + <ul> + <li>Are considered framework internal implementation details.</li> + <li>Must not be accessed by vendor modules.</li> + <li>Have unstable ABIs/APIs and no API/ABI compatibility guarantees.</li> + <li>Are not copied.</li> + </ul> +</li> +</ul> + +<h3 id="sp-hal">Same-Process HAL (SP-HAL)</h3> +<p> +<em>Same-Process HAL</em> (<em>SP-HAL</em>) is a set of predetermined HALs +implemented as <em>Vendor Shared Libraries</em> and loaded into <em>Framework +Processes</em>. SP-HALs are isolated by a linker namespace (controls the +libraries and symbols that are visible to the shared libraries). SP-HALs must +depend only on <em>LL-NDK</em>, <em>SP-NDK</em>, and <em>VNDK-SP</em>.</p> + +<p>VNDK-SP is a predefined subset of eligible VNDK libraries. VNDK-SP libraries +are carefully reviewed to ensure double-loading VNDK-SP libraries into framework +processes does not cause problems. Both SP-HALs and VNDK-SPs are defined by +Google.</p> + +<p>The following libraries are approved SP-HALs:</p> + +<ul> +<li><code>libGLESv2_${driver}.so</code></li> +<li><code>libGLESv1_CM_${driver}.so</code></li> +<li><code>libEGL_${driver}.so</code></li> +<li><code>vulkan.${driver}.so</code></li> +<li><code>android.hardware.renderscript@1.0-impl.so</code></li> +<li><code>android.hardware.graphics.mapper@2.0-impl.so</code></li> +</ul> + +<p>The following libraries are VNDK-SP libraries that are accessible by SP-HALs: +</p> + +<ul> +<li><code>android.hardware.graphics.allocator@2.0.so</code></li> +<li><code>android.hardware.graphics.common@1.0.so</code></li> +<li><code>android.hardware.graphics.mapper@2.0.so</code></li> +<li><code>android.hardware.renderscript@1.0.so</code> (Renderscript)</li> +<li><code>libRS_internal.so</code> (Renderscript)</li> +<li><code>libbase.so</code></li> +<li><code>libc++.so</code></li> +<li><code>libcutils.so</code></li> +<li><code>libhardware.so</code></li> +<li><code>libhidlbase.so</code></li> +<li><code>libhidltransport.so</code></li> +<li><code>libhwbinder.so</code></li> +<li><code>libutils.so</code></li> +</ul> + +<p> +The following <em>VNDK-SP dependencies (VNDK-SP-Indirect)</em> are invisible to +<em>SP-HALs</em>: +</p><ul> +<li><code>libRSCpuRef.so</code> (Renderscript)</li> +<li><code>libRSDriver.so</code> (Renderscript)</li> +<li><code>libbacktrace.so</code></li> +<li><code>libblas.so</code> (Renderscript)</li> +<li><code>libbcinfo.so</code> (Renderscript)</li> +<li><code>liblzma.so</code></li> +<li><code>libunwind.so</code></li> +</ul> + +<p>The following <em>private VNDK-SP dependency (VNDK-SP-Indirect-Private)</em> +is invisible to all vendor modules:</p> + +<ul> +<li><code>libcompiler_rt.so</code> (Renderscript)</li> +</ul> + +<p>The following are <em>framework-only libraries with RS exceptions +(FWK-ONLY-RS)</em>:</p> +<ul> +<li><code>libft2.so</code> (Renderscript)</li> +<li><code>libmediandk.so</code> (Renderscript)</li> +</ul> + + +<h2 id="vndk-terminology">VNDK terminology</h2> +<ul> +<li><em>Modules</em> refer to either <em>Shared Libraries</em> or +<em>Executables</em>.</li> +<li><em>Processes</em> are operating system tasks spawned from +<em>Executables</em>.</li> +<li><em>Framework</em>-qualified terms refer to the concepts related to the +<strong>system</strong> partition.</li> +<li><em>Vendor</em>-qualified terms refer to the concepts related to +<strong>vendor</strong> partitions.</li> +</ul> + +<p>For example:</p> +<ul> +<li><em>Framework Executables</em> refer to executables in +<code>/system/bin</code> or <code>/system/xbin</code>.</li> +<li><em>Framework Shared Libraries</em> refer to shared libraries under +<code>/system/lib[64]</code>.</li> +<li><em>Framework Modules</em> refer to both <em>Framework Shared Libraries</em> +and <em>Framework Executables</em>.</li> +<li><em>Framework Processes</em> are processes spawned from <em>Framework +Executables</em> (e.g. <code>/system/bin/app_process</code>).</li> +<li><em>Vendor Executables</em> refer to executables in <code>/vendor/bin</code> +<li><em>Vendor Shared Libraries</em> refer to shared libraries under +<code>/vendor/lib[64]</code>.</li> +<li><em>Vendor Modules</em> refer to both <em>Vendor Executables</em> and +<em>Vendor Shared Libraries</em>.</li> +<li><em>Vendor Processes</em> are processes spawned from <em>Vendor +Executables</em> (e.g.</li> +<code>/vendor/bin/android.hardware.camera.provider@2.4-service</code>).</li> +</ul> + + </body> +</html> diff --git a/en/devices/architecture/vndk/linker-namespace.html b/en/devices/architecture/vndk/linker-namespace.html new file mode 100644 index 00000000..d0ef3421 --- /dev/null +++ b/en/devices/architecture/vndk/linker-namespace.html @@ -0,0 +1,154 @@ +<html devsite> + <head> + <title>Linker Namespace</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p>Linker namespace is a mechanism to control the shared library search path and +the permission to open the libraries in a directory.</p> + +<h2 id="framework-processes">Framework processes</h2> +<h3 id="default-namespace">Default namespace</h3> +<p>Search paths and permitted paths:</p> +<ol> +<li><code>/system/lib[64]</code></li> +<li><code>/vendor/lib[64]</code> # Deprecated in Android O (for legacy modules +only)</li> +</ol> + +<h3 id="sp-hal-namespace">SP-HAL namespace</h3> +<p>Search paths and permitted paths:</p> +<ol> +<li><code>/vendor/lib[64]/egl</code></li> +<li><code>/vendor/lib[64]/hw</code></li> +<li><code>/vendor/lib[64]</code></li> +</ol> + +<p>Shared libraries imported from other namespaces:</p> +<ul> +<li>Exported by <strong>default</strong> namespace: + <ul> + <li>LL-NDK: <code>libc.so</code>, <code>libm.so</code>, <code>libdl.so</code>, + <code>libstdc++.so</code>, <code>liblog.so, libz.so</code></li> + <li>SP-NDK: <code>libEGL.so</code>, <code>libGLESv1_CM.so</code>, + <code>libGLESv2.so</code>, <code>libnativewindow.so</code>, + <code>libsync.so</code>, <code>libvndksupport.so</code></li> + </ul> +</li> +<li>Exported by <strong>vndk</strong> namespace: +<code>android.hardware.renderscript@1.0.so</code>, +<code>android.hardware.graphics.allocator@2.0.so</code>, +<code>android.hardware.graphics.mapper@2.0.so</code>, +<code>android.hardware.graphics.common@1.0.so</code>, +<code>libhwbinder.so</code>, <code>libbase.so</code>, <code>libcutils.so</code>, +<code>libhardware.so</code>, <code>libhidlbase.so</code>, +<code>libhidltransport.so</code>, <code>libutils.so</code>, +<code>libc++.so</code> +</li> +<li>Exported by <strong>rs</strong> namespace: <code>libRS_internal.so</code> +</li> +</ul> + +<h3 id="vndk-vndk-sp-namespace">VNDK (VNDK-SP) namespace</h3> +<p>Search paths:</p> +<ol> +<li><code>/vendor/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]/vndk-sp</code></li> +<li><code>/vendor/lib[64]</code></li> +</ol> + +<p>Permitted paths:</p> +<ul> +<li><code>/vendor/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]/vndk-sp</code></li> +<li><code>/vendor/lib[64]</code></li> +<li><code>/vendor/lib[64]/hw</code></li> +<li><code>/vendor/lib[64]/egl</code></li> +</ul> + +<p>Shared libraries imported from other namespaces:</p> +<ul> +<li>Exported by <strong>default</strong> namespace: + <ul> + <li>LL-NDK: <code>libc.so</code>, <code>libm.so</code>, <code>libdl.so</code>, + <code>libstdc++.so</code>, <code>liblog.so</code>,<code> libz.so</code> + <li>SP-NDK: <code>libEGL.so</code>, <code>libnativewindow.so</code>, + <code>libsync.so</code>, <code>libvndksupport.so</code> + </li> + </ul> +</li> +</ul> + +<h3 id="rs-namespace">RS namespace</h3> +<p>This special namespace allows <code>libRS_internal.so</code> to use +<code>libmediandk.so</code>, which is not accessible from either SP-HAL +namespace or VNDK-SP namespace (this is a relaxation in Android O).</p> + +<p>Search paths:</p> +<ol> +<li><code>/vendor/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]/vndk-sp</code></li> +<li><code>/vendor/lib[64]</code></li> +</ol> + +<p>Permitted paths:</p> +<ul> +<li><code>/vendor/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]/vndk-sp</code></li> +<li><code>/vendor/lib[64]</code></li> +<li><code>/data</code> # Shared libraries JIT-compiled by RS CPU reference +implementation</li> +</ul> + +<p>Shared libraries imported from other namespaces:</p> +<ul> +<li>Exported by <strong>default</strong> namespace + <ul> + <li>LL-NDK: <code>libc.so</code>, <code>libm.so</code>, <code>libdl.so</code>, + <code>libstdc++.so</code>, <code>liblog.so</code></li> + <li>SP-NDK: <code>libEGL.so</code>, <code>libGLESv1_CM.so</code>, + <code>libGLESv2.so</code>, <code>libnativewindow.so</code>, + <code>libsync.so</code>, <code>libvndksupport.so</code></li> + <li>Other Lib: <code>libmediandk.so</code>, <code>libui.so</code></li> + </ul> +</li> +<li>Exported by <strong>vndk</strong> namespace: +<code>android.hardware.renderscript@1.0.so</code>, +<code>android.hardware.graphics.allocator@2.0.so</code>, +<code>android.hardware.graphics.mapper@2.0.so</code>, +<code>android.hardware.graphics.common@1.0.so</code>, +<code>libhwbinder.so</code>, <code>libbase.so</code>, <code>libcutils.so</code>, +<code>libhardware.so</code>, <code>libhidlbase.so</code>, +<code>libhidltransport.so</code>, <code>libutils.so</code>, +<code>libc++.so</code></li> +</ul> + +<h2 id="vendor-processes">Vendor processes</h2> +<h3 id="default-linker-namespace">Default linker namespace</h3> +<p>Search paths and permitted paths:</p> +<ol> +<li><code>/vendor/lib[64]</code></li> +<li><code>/vendor/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]/vndk-sp</code></li> +<li><code>/system/lib[64]</code> # For degenerated VNDK libraries</li> +</ol> + + </body> +</html> diff --git a/en/devices/architecture/vndk/renderscript.html b/en/devices/architecture/vndk/renderscript.html new file mode 100644 index 00000000..82f3db16 --- /dev/null +++ b/en/devices/architecture/vndk/renderscript.html @@ -0,0 +1,449 @@ +<html devsite> + <head> + <title>Renderscript</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p><em>RenderScript</em> is a framework for running computationally intensive +tasks at high performance on Android. It is designed for use with data-parallel +computation, although serial workloads can benefit as well. The RenderScript +runtime parallelizes work across processors available on a device, such as +multi-core CPUs and GPUs, enabling developers to focus on expressing algorithms +rather than scheduling work. RenderScript is especially useful for applications +performing image processing, computational photography, or computer vision.</p> + +<p>Android O devices use the following RenderScript framework and vendor HALs: +</p> +<img src="../images/treble_rs_linking.png"> +<figcaption><strong>Figure 1.</strong> Vendor code linking to internal libs. +</figcaption> + +<p>Differences from RenderScript in Android 7.x and earlier include:</p> +<ul> +<li>Two instances of RenderScript internal libs in a process. One set is for CPU +fallback path and is from directly at <code>/system/lib</code>; the other set is +for GPU path and is from <code>/system/lib/vndk-sp</code>.</li> +<li>RS internal libs in <code>/system/lib</code> are built as part of the +platform and are updated as <code>system.img</code> is upgraded. However, libs +in <code>/system/lib/vndk-sp</code> are built for the vendor and are not updated +when <code>system.img</code> is upgraded (while they can be updated for a +security fix, their ABI remains the same).</li> +<li>Vendor code (RS HAL, RS driver, and the bcc plugin) are linked against the +RenderScript internal libs located at <code>/system/lib/vndk-sp</code>. They +cannot link against libs in <code>/system/lib</code> because libs in that +directory are built for the platform and thus may not be compatible with the +vendor code (i.e., symbols may be removed). Doing so would make a framework-only +OTA impossible.</li> +</ul> + +<p>For more details, see +<a href="https://developer.android.com/guide/topics/renderscript/compute.html" class="external">Renderscript</a> +on developer.android.com.</p> + +<h2 id="design">Design</h2> +<p>The following sections detail RenderScript design in Android O.</p> + +<h3 id="renderscript-libs-available-to-vendors">RenderScript libs available to +vendors</h3> +<p>This section lists the RenderScript libs (known as Vendor NDK for +Same-Process HALs or VNDK-SP) that are available to vendor code and which can be +linked against. It also details additional libraries that are unrelated to +RenderScript but which are also provided to vendor code.</p> + +<p>While the following list of libraries might differ between Android release, +it is immutable for a specific Android release; for an up-to-date list of +available libraries, refer to <code>/system/etc/ld.config.txt</code>.</p> + +<aside class="note"><strong>Note:</strong> Libraries not listed below cannot be +used by any vendor code; i.e. <code>libLLVM.so</code> cannot be used by the +vendor's bcc plugin as the lib is not in the list.</aside> + +<table> +<tr> +<th style="width:50%">RenderScript Libs</th> +<th>Non-RenderScript Libs</th> +</tr> +<tr> +<td><ul> +<li><code>android.hardware.graphics.renderscript@1.0.so</code></li> +<li><code>libRS_internal.so</code></li> +<li><code>libRSCpuRef.so</code></li> +<li><code>libblas.so</code></li> +<li><code>libbcinfo.so</code></li> +<li><code>libcompiler_rt.so</code></li> +<li><code>libRSDriver.so</code></li> +</ul></td> +<td><ul> +<li><code>libc.so</code></li> +<li><code>libm.so</code></li> +<li><code>libdl.so</code></li> +<li><code>libstdc++.so</code></li> +<li><code>liblog.so</code></li> +<li><code>libnativewindow.so</code></li> +<li><code>libsync.so</code></li> +<li><code>libvndksupport.so</code></li> +<li><code>libbase.so</code></li> +<li><code>libc++.so</code></li> +<li><code>libcutils.so</code></li> +<li><code>libutils.so</code></li> +<li><code>libhardware.so</code></li> +<li><code>libhidlbase.so</code></li> +<li><code>libhidltransport.so</code></li> +<li><code>libhwbinder.so</code></li> +<li><code>liblzma.so</code></li> +<li><code>libz.so</code></li> +<li><code>libEGL.so</code></li> +<li><code>libGLESv1_CM.so</code></li> +<li><code>libGLESv2.so</code></li> +</ul></td> + </tr> +</table> + +<h3 id="linker-namespace-configuration">Linker namespace configuration</h3> +<p>The linking restriction that prevents libs not in VNDK-SP from being used by +vendor code is enforced at runtime using the linker namespace. (For details, +refer to the +<a href="/devices/architecture/images/vndk_design_android_o.pdf">VNDK Design in +Android O</a> presentation.)</p> + +<p>On a device running Android O, all Same-Process HALs (SP-HALs) <em>except +RenderScript</em> are loaded inside the linker namespace <code>sphal</code>. +RenderScript is loaded into the RenderScript-specific namespace <code>rs</code>, +a location that enables a slightly looser enforcement for RenderScript libs. +Because the RS implementation needs to load the compiled bitcode, +<code>/data/*/*.so</code> is added to the path of the <code>rs</code> namespace +(other SP-HALs are not allowed to load libs from the data partition).</p> + +<p>In addition, the <code>rs</code> namespace allows more libs than provided for +by other namespaces. <code>libmediandk.so</code> and <code>libft2.so</code> are +exposed to the <code>rs</code> namespace because <code>libRS_internal.so</code> +has an internal dependency to these libraries.</p> + +<img src="../images/treble_rs_namespace.png"> +<figcaption><strong>Figure 2.</strong> Namespace configuration for linker. +</figcaption> + +<h3 id="loading-drivers">Loading drivers</h3> + +<h4>CPU fallback path</h4> +<p>Depending on the existence of the <code>RS_CONTEXT_LOW_LATENCY</code> bit +when creating an RS context, either the CPU or GPU path is selected. When the +CPU path is selected, <code>libRS_internal.so</code> (the main implementation of +the RS framework) is directly <code>dlopen</code>ed from the default linker +namespace where the platform version of RS libs are provided.</p> + +<p>The RS HAL implementation from the vendor is not used at all when the CPU +fallback path is taken, and an <code>RsContext</code> object is created with +null <code>mVendorDriverName</code>. <code>libRSDriver.so</code> is (by default) +<code>dlopen</code>ed and the driver lib is loaded from the <code>default</code> +namespace because the caller (<code>libRS_internal.so</code>) is also loaded in +the <code>default</code> namespace.</p> + +<img src="../images/treble_rs_cpu_fallback.png"> +<figcaption><strong>Figure 4.</strong> CPU fallback path.</figcaption> + +<h4 id="gpu-path">GPU path</h4> +<p>For the GPU path, the <code>libRS_internal.so</code> is loaded differently. +First, <code>libRS.so</code> uses +<code>android.hardware.renderscript@1.0.so</code> (and its underlying +<code>libhidltransport.so</code>) to load +<code>android.hardware.renderscript@1.0-impl.so</code> (a vendor implementation +of RS HAL) into a different linker namespace called <code>sphal</code>. The RS +HAL then <code>dlopen</code>s <code>libRS_internal.so</code> in a another linker +namespace called <code>rs</code>.</p> + +<p>Vendors can provide their own RS driver by setting the build-time flag +<code>OVERRIDE_RS_DRIVER</code>, which is embedded into the RS HAL +implementation +(<code>hardware/interfaces/renderscript/1.0/default/Context.cpp</code>). This +driver name is then <code>dlopen</code>ed for the RS context for the GPU path. +</p> + +<p>The creation of the <code>RsContext</code> object is delegated to the RS HAL +implementation. The HAL calls back to the RS framework using +<code>rsContextCreateVendor()</code> function with the name of the driver to use +as an argument. The RS framework then loads the specified driver when the +<code>RsContext</code> is initialized. In this case, the driver library is +loaded into the <code>rs</code> namespace because the <code>RsContext</code> +object is created inside the <code>rs</code> namespace and +<code>/vendor/lib</code> is in the search path of the namespace.</p> + +<img src="../images/treble_rs_gpu_fallback.png"> +<figcaption><strong>Figure 5.</strong> GPU fallback path.</figcaption> + +<p>When transitioning from the <code>default</code> namespace to the +<code>sphal</code> namespace, <code>libhidltransport.so</code> uses the +<code>android_load_sphal_library()</code> function to explicitly order the +dynamic linker to load the <code>-impl.so</code> library from the +<code>sphal</code> namespace.</p> + +<p>When transitioning from the <code>sphal</code> namespace to the <code>rs</code> +namespace, loading is done indirectly by the following line in +<code>/system/etc/ld.config.txt</code>:</p> + +<pre class="prettyprint"> +namespace.sphal.link.rs.shared_libs = libRS_internal.so +</pre> + +<p>This line specifies the dynamic linker should load +<code>libRS_internal.so</code> from the <code>rs</code> namespace when the lib +can't be found/loaded from the <code>sphal</code> namespace (which is always the +case because <code>sphal</code> namespace does not search +<code>/system/lib/vndk-sp</code> where <code>libRS_internal.so</code> resides). +With this configuration, a simple <code>dlopen()</code> call to +<code>libRS_internal.so</code> is enough to make the namespace transition.</p> + +<h3 id="loading-bcc-plugin">Loading bcc plugin</h3> +<p><code>bcc plugin</code> is a vendor-provided library loaded into the +<code>bcc</code> compiler. Because <code>bcc</code> is a system process in the +<code>/system/bin</code> directory, the <code>bcc plugin</code> library can be +considered an SP-HAL (i.e., a vendor HAL that can be directly loaded into the +system process without being binderized). As an SP-HAL, the +<code>bcc-plugin</code> library:</p> +<ul> +<li>Cannot link against framework-only libraries such as +<code>libLLVM.so</code>.</li> +<li>Can link against only the VNDK-SP libraries available to the +vendor.</li> +</ul> + +<p>This restriction is enforced by loading the bcc plugin into the +<code>sphal</code> namespace using the <code>android_sphal_load_library()</code> +function. In previous versions of Android, the plugin name was specified using +the <code>-load</code> option and the lib was loaded using the simple +<code>dlopen()</code> by <code>libLLVM.so</code>. In Android O, this is +specified in <code>-plugin</code> option and the lib is directly loaded by the +<code>bcc</code> itself. This option enables a non-Android-specific path to +the open source LLVM project.</p> + +<img src="../images/treble_rs_bcc_plugin_old.png"> +<figcaption><strong>Figure 6.</strong> Loading bcc plugin, Android 7.x and +earlier.</figcaption> +<br> +<br> +<img src="../images/treble_rs_bcc_plugin_new.png"> +<figcaption><strong>Figure 7. </strong>Loading bcc plugin, Android O. +</figcaption> + +<h3 id="search-paths-for-ld-mc">Search paths for ld.mc</h3> +<p>When executing <code>ld.mc</code>, some RS runtime libs are given as inputs +to the linker. The RS bitcode from the app is linked against the runtime libs +and when the converted bitcode is loaded into an app process, the runtime libs +are again dynamically linked from the converted bitcode.</p> + +<p>Runtime libs include:</p> +<ul> +<li><code>libcompiler_rt.so</code></li> +<li><code>libm.so</code></li> +<li><code>libc.so</code></li> +<li>RS driver (either <code>libRSDriver.so</code> or +<code>OVERRIDE_RS_DRIVER</code>)</li> +</ul> + +<p>When loading the compiled bitcode into the app process, we must provide the +exact same library that was used by <code>ld.mc</code>. Otherwise, the compiled +bitcode may not find a symbol which was available when it was linked.</p> + +<p>To do so, RS framework uses different search paths for the runtime libs when +executing <code>ld.mc</code>, depending on whether the RS framework itself is +loaded from <code>/system/lib</code> or from <code>/system/lib/vndk-sp</code>. +This can be determined by reading the address of an arbitrary symbol of a RS +framework lib and using <code>dladdr()</code> to get the file path mapped to the +address.</p> + +<h3 id="selinux-policy">SELinux policy</h3> +<p>As a result of the SELinux policy changes in Android O, you must follow +specific rules (enforced through <code>neverallows</code>) when labelling +additional files in <code>vendor</code> partition:</p> +<ul> +<li><code>vendor_file</code> must be the default label in for all files in +<code>vendor</code> partition. The platform policy requires this to access +passthrough HAL implementations.</li> +<li>All new <code>exec_types</code> added in <code>vendor</code> partition +through vendor SEPolicy must have <code>vendor_file_type</code> attribute. This +is enforced through <code>neverallows</code>.</li> +<li>To avoid conflicts with future platform/framework updates, avoid labelling +files other than <code>exec_types</code> in <code>vendor</code> partition. +<li>All library dependencies for AOSP-identified same process HALs must be +labelled as <code>same_process_hal_file</code>.</li> +</ul> + +<aside class="note"><strong>Note:</strong> For details on Android 8.0 SELinux, +see <a href="/security/selinux/images/SELinux_Treble.pdf">SELinux for Android +8.0</a>.</aside> + +<h3 id="abi-compatibility-for-bitcode">ABI compatibility for bitcode</h3> +<p>If no new APIs are added, which means no HAL version bump, the RS frameworks +will keep using the existing GPU (HAL 1.0) driver.</p> + +<p>For minor HAL changes (HAL 1.1) not affecting bitcode, the frameworks should +fallback to CPU for these newly added APIs and keep using GPU (HAL 1.0) driver +elsewhere.</p> + +<p>For major HAL changes (HAL 2.0) affecting bitcode compilation/linking, RS +frameworks should choose not to load vendor-provided GPU drivers and instead use +the CPU or Vulkan path for acceleration.</p> + +<p>Consuming RenderScript bitcode occurs in three stages:</p> + +<table> +<tr> +<th>Stage</th> +<th>Details</th> +</tr> +<tr> +<td><em>Compile</em></td> +<td><ul> +<li>The input bitcode (.bc) for <code>bcc</code> must be in +<code>LLVM 3.2</code> bitcode format and <code>bcc</code> must be backward +compatible with existing (legacy) apps.</li> +<li>However, the meta-data in .bc could change (there could new runtime +functions, e.g, Allocation setters ∓ getters, math functions, etc.). Part +of the runtime functions lives in <code>libclcore.bc</code>, part of them +lives in LibRSDriver or vendor equivalent.</li> +<li>New runtime functions or breaking meta-data changes require incrementing +the bitcode API level. Because vendor drivers won't be able to consume it, the +HAL version must also be incremented.</li> +<li>Vendors may have their own compilers, but the conclusions/requirements for +<code>bcc</code> also apply to those compilers.</li> +</ul></td> +</tr> +<tr> +<td><em>Link</em></td> +<td><ul> +<li>The compiled .o will be linked with vendor driver, e.g, +<code>libRSDriver_foo.so</code>, and <code>libcompiler_rt.so</code>. The CPU +path will link with <code>libRSDriver.so</code>.</li> +<li>If the .o requires a new runtime API from <code>libRSDriver_foo</code>, the +vendor driver has to be updated to support it.</li> +<li>Certain vendors may have their own linkers, but the argument for +<code>ld.mc</code> also apply to them.</li> +</ul></td> +</tr> +<tr> +<td><em>Load</em></td> +<td><ul> +<li><code>libRSCpuRef</code> loads the the shared object. If there are changes +to this interface, a HAL version bump is needed.</li> +<li>Vendors would either rely on <code>libRSCpuRef</code> to load the shared +object, or implement their own.</li> +</ul></td> +</tr> +</table> + +<p>In addition to the HAL, runtime APIs and the exported symbols are also +interfaces. Neither interface has changed since Android 7.0 (API 24) and there +are no immediate plans to change it in Android O and beyond. However, if the +interface does change, the HAL version will also increment.</p> + +<h2 id="vendor-implementations">Vendor implementations</h2> +<p>Android O requires some GPU driver changes for the GPU driver to work +correctly.</p> + +<h3 id="driver-modules">Driver modules</h3> +<ul> +<li>Driver modules must not depend on any system libraries that are not in +<a href="#renderscript-libs-available-to-vendors">the list</a>.</li> +<li>Driver must provide its own +<code>android.hardware.renderscript@1.0-impl_{NAME}</code>, or declare the +default implementation <code>android.hardware.renderscript@1.0-impl</code> as +its dependency.</li> +<li>CPU implementation <code>libRSDriver.so</code> is a good example of how to +remove non-VNDK-SP dependencies.</li> +</ul> + +<h3 id="bitcode-compiler">Bitcode compiler</h3> +<p>You can compile RenderScript bitcode for the vendor driver in two ways:</p> +<ol> +<li>Invoke vendor-specific RenderScript compiler in <code>/vendor/bin/</code> +(preferred method of GPU compilation). Similar to other driver modules, the +vendor compiler binary cannot depend on any system library that is not in the +list of <a href="#renderscript-libs-available-to-vendors">RenderScript libs +available to vendors</a>.</li> +<li>Invoke system bcc: <code>/system/bin/bcc</code> with a vendor-provided bcc +plugin. The <code>bcc plugin</code> cannot depend on any system library that is +not in the list of +<a href="#renderscript-libs-available-to-vendors">RenderScript libs available to +vendors</a>.</li> +</ol> + +<p>If the vendor <code>bcc plugin</code> needs to interfere with the CPU +compilation and its dependency on <code>libLLVM.so</code> cannot be easily +removed, the vendor should copy <code>bcc</code> (and all the non-LL-NDK +dependencies, including <code>libLLVM.so</code>, <code>libbcc.so</code>) into +<code>/vendor</code> partition.</p> + +<p>In addition, vendors need to make the following changes:</p> +<img src="../images/treble_rs_vendor_driver.png"> +<figcaption><strong>Figure 8.</strong> Changes to vendor driver.</figcaption> +<ol> +<li>Copy <code>libclcore.bc</code> to <code>/vendor</code> partition. This +ensures <code>libclcore.bc</code>, <code>libLLVM.so</code>, and +<code>libbcc.so</code> are in sync.</li> +<li>Change the path to the <code>bcc</code> executable by setting +<code>RsdCpuScriptImpl::BCC_EXE_PATH</code> from the RS HAL implementation.</li> +</ol> + +<aside class="note"><strong>Note:</strong> Restrictions for +<code>/vendor/bin/*</code> processes are not fully implemented in Android O. +While not recommended, it is theoretically possible to just copy +<code>bcc</code> to <code>/vendor/bin/ </code>without copying its dependencies. +</aside> + +<h3 id="selinux-policy">SELinux policy</h3> +<p>SELinux policy affects both the driver and the compiler executables. All +driver modules must be labeled <code>same_process_hal_file</code> in the +device's <code>file_contexts</code>. For example:</p> + +<pre class="prettyprint"> +/vendor/lib(64)?/libRSDriver_EXAMPLE\.so u:object_r:same_process_hal_file:s0 +</pre> + +<p>The compiler executable must be able to be invoked by an app process, as does +the vendor copy of bcc (<code>/vendor/bin/bcc</code>). For example:</p> + +<pre class="prettyprint"> +device/vendor_foo/device_bar/sepolicy/file.te: +type renderscript_exec, exec_type, file_type; + +device/vendor_foo/device_bar/sepolicy/app.te: +allow appdomain renderscript_exec:file { read open getattr execute execute_no_trans }; + +device/vendor_foo/device_bar/sepolicy/file_contexts: +/vendor/bin/bcc u:object_r:renderscript_exec:s0 +</pre> + +<h3 id="legacy-devices">Legacy devices</h3> +<p>Legacy devices are those that satisfy the following conditions:</p> +<ol> +<li><em>PRODUCT_SHIPPING_API_LEVEL</em> is lower than 26.</li> +<li><em>PRODUCT_FULL_TREBLE_OVERRIDE</em> is not defined.</li> +</ol> + +<p>For legacy devices, the restrictions detailed throughout this document are +not enforced when upgrading to Android O, meaning the drivers can continue to +link to libraries in <code>/system/lib[64]</code>. However, because of the +architecture change related with <code>OVERRIDE_RS_DRIVER</code>, +<code>android.hardware.renderscript@1.0-impl</code> must be installed to +<code>/vendor</code> partition; failing to do so forces RenderScript runtime +fallback to CPU path.</p> + + </body> +</html> |
