1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
|
<html devsite>
<head>
<title>Reducing OTA Size</title>
<meta name="project_path" value="/_project.yaml" />
<meta name="book_path" value="/_book.yaml" />
</head>
<body>
<!--
Copyright 2017 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<p>This page describes build changes added to AOSP to reduce unnecessary file changes between
builds. Device implementers who maintain their own build system can use this information as a
guide for reducing over-the-air (OTA) update size.</p>
<p>Android OTAs occasionally contain changed files that do not correspond to code changes but are
instead artifacts of the build system. This can occurs when the same code built at different
times, from different directories, or on different machines, produces a large number of changed
files. These excess files not only increase the size of an OTA, but make it difficult to
determine which code is changed in the OTA.</p>
<p>To make the contents of an OTA more transparent, AOSP includes build system changes designed to
reduce OTA size by eliminating unnecessary file changes between builds. The aim is to reduce
the size of OTAs to include only the files that relate to the patches contained in the OTA. AOSP
also includes a <a href="#the_build_diff_tool">build diff tool</a> that filters out common
build-related file changes and provides a cleaner build file diff.</p>
<p>The build system can create unnecessary file diffs in several ways. The following sections
discuss some of these issues and solutions, providing examples of fixes in AOSP when possible).</p>
<h2 id=file_order>File order</h2>
<p><strong>Problem</strong>: Filesystems don’t guarantee a file order when asked for a list of files
in a directory, though it’s commonly the same for the same checkout. Tools such as <code>ls</code>
sort the results by default, but the wildcard function used by commands such as <code>find</code>
and <code>make</code> do not. Before using these tools, you must sort the outputs.</p>
<p><strong>Solution</strong>: Users of tools such as <code>find</code> and <code>make</code> with
wildcard must sort the output of these commands before using them. Use of
<code>$(wildcard )</code> or <code>$(shell find )</code> in Android.mk files should also be
sorted. Some tools, such as Java, do sort inputs so verify sorting is necessary.</p>
<p><strong>Examples:</strong> Many instances were fixed in the core build system using the builtin
<code>all-*-files-under</code> macro, which includes <code>all-cpp-files-under</code> (as several
definitions were spread out in other makefiles). For details, refer to the following CLs:</p>
<ul>
<li><a href="https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f">https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f</a>
<li><a href="https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410">https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410</a>
<li><a href="https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653">https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653</a>
<li><a href="https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c">https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c</a>
</ul>
<h2 id=build_directory>Build directory</h2>
<p><strong>Problem:</strong> Changing the directory in which things are built can cause the binaries
to be different. Most paths in the Android build are relative paths so <code>__FILE__</code> in
C/C++ isn’t a problem. However, the debug symbols encode the full pathname by default, and the
<code>.note.gnu.build-id</code> is generated from hashing the pre-stripped binary, so it will
change if the debug symbols change.</p>
<p><strong>Solution:</strong> AOSP now makes debug paths relative. For details, refer to CL:
<a href="https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02">https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02</a>.</p>
<h2 id=timestamps>Timestamps</h2>
<p><strong>Problem:</strong> Timestamps in the build output result in unnecessary file changes. This
is likely to happen in the following locations:</p>
<ul>
<li><code> __DATE__/__TIME__/__TIMESTAMP__ </code> macros in C or C++ code.</li>
<li>Timestamps embedded in zip-based archives.</li>
</ul>
<p><strong>Solutions/Examples:</strong> To remove timestamps from the build output, use the
instructions in the sections below.</p>
<h3 id=date_time_timestamp_in_c_c>__DATE__/__TIME__/__TIMESTAMP__ in C/C++</h3>
<p>These macros always produce different outputs for different builds, so they shouldn’t be used.
Here are a few options on how to eliminate these macros:</p>
<ul>
<li>Just remove them, they often aren’t necessary. For an example, refer to
<a href="https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f">https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f</a></li>
<li>To uniquely identify the running binary, read the build-id from the ELF header.</li>
<li>To know when the OS was built, read the <code>ro.build.date</code> (should work for everything
except incremental builds, which may not update this date). For an example, refer to
<a href="https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84">https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84</a></li>
</ul>
<p class="note"><strong>Note:</strong>We turned on <code>-Werror=date-time</code> so using
timestamps is a build error.</p>
<h3 id=embedded_timestamps_in_zip-based_archives_zip_jar>Embedded timestamps in archives (zip, jar)</h3>
<p>We fixed the problem of embedded timestamps in zip archives by adding <code>-X</code> to all uses
of the <code>zip</code> command, so the UID/GID of the builder and the extended Unix timestamp are
not embedded in the zip file.</p>
<p>A new tool, <code>ziptime</code> (located in <code>
<a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/">/platform/build/+/master/tools/ziptime/</a></code>)
resets the normal timestamps in the zip headers. For details, refer to the
<a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/README.txt">README
file</a>.</p>
<p>The <code>signapk</code> tool sets timestamps for the APK files that may vary depending on the
server timezone. For details, refer to the CL
<a href="https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028">https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028</a>.</p>
<h2 id=version_strings>Version strings</h2>
<p><strong>Problem:</strong> APK version strings often had the BUILD_NUMBER appended to the
hardcoded version. Even if nothing else changed in the APK, the APK would still be different.</p>
<p><strong>Solution:</strong> Remove the build number from the APK version string.</p>
<p><strong>Examples:</strong></p>
<ul>
<li><a href="https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27">https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27</a></li>
<li><a href="https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c">https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c</a></li>
</ul>
<h2 id=consistent_build_tools>Consistent build tools</h2>
<p><strong>Problem:</strong> Tools that generate installed files must be consistent (the same input
should always produce the same output).</p>
<p><strong>Solutions/Examples:</strong> Changes were required in the following build tools:</p>
<ul>
<li><strong>NOTICE file creator</strong>. The NOTICE file creator needed the changes. Refer to CL:
<a href="https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64">https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64</a></li>
<li><strong>Java Android Compiler Kit (Jack)</strong>. The Jack toolchain required an update to
handle an occasional change in generated constructor ordering. Refer to CL:
<a href="https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b">https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b</a></li>
<li><strong>ART AOT compiler (dex2oat)</strong>. The ART compiler binary required an update to
create a deterministic image. Refer to CL:
<a href="https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9">https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9</a></li>
<li><strong>The libpac.so file (V8)</strong>Every build creates a different
<code>/system/lib/libpac.so</code> file because the V8 snapshot changes for each build. The
solution is to remove the snapshot. Refer to CL:
<a href="https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29">https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29</a></li>
<li><strong>Application pre-dexopt’d (.odex) files</strong>. The pre-dexopt’d (.odex) files
contained uninitialized padding on 64-bit systems. Refer to CL:
<a href="https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029">https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029</a></li>
</ul>
<h2 id=the_build_diff_tool>Using the build diff tool</h2>
<p>For cases where it is not possible to eliminate build-related file changes, we supply a build
diff tool,
<code><a href="https://android.googlesource.com/platform/build/+/master/tools/releasetools/target_files_diff.py">target_files_diff.py</a></code>
for use in comparing two file packages. This tool performs a recursive diff between two builds,
excluding common build-related file changes, such as:</p>
<ul>
<li>Expected changes in the build output (for example, due to a build number change).</li>
<li>Changes due to known issues in the current build system.</li>
</ul>
<p>To use the build diff tool, run the following command:</p>
<pre class="devsite-terminal devsite-click-to-copy">
target_files_diff.py dir1 dir2
</pre>
<p><code>dir1</code> and <code>dir2</code> are base directories that contain the extracted target
files for each build.</p>
</body>
</html>
|
