diff options
Diffstat (limited to 'en/devices/tech/debug/kasan-kcov.html')
| -rw-r--r-- | en/devices/tech/debug/kasan-kcov.html | 410 |
1 files changed, 410 insertions, 0 deletions
diff --git a/en/devices/tech/debug/kasan-kcov.html b/en/devices/tech/debug/kasan-kcov.html new file mode 100644 index 00000000..20ba7fce --- /dev/null +++ b/en/devices/tech/debug/kasan-kcov.html @@ -0,0 +1,410 @@ +<html devsite> + <head> + <title>Building a Pixel kernel with KASAN+KCOV</title> + <meta name="project_path" value="/_project.yaml" /> + <meta name="book_path" value="/_book.yaml" /> + </head> + <body> + <!-- + Copyright 2017 The Android Open Source Project + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + +<p> +Kernel Address Sanitizer (<a +href="https://www.kernel.org/doc/html/latest/dev-tools/kasan.html">KASAN</a>) +helps kernel developers and testers find runtime memory-related bugs, such as +out-of-bound read or write operations, and use-after-free issues. While KASAN +isn't enabled on production builds due to its runtime performance +penalties and memory usage increment, it is still a valuable tool for testing +debug builds. +</p> +<p> +When used with another runtime tool called Kernel Coverage (<a +href="https://lwn.net/Articles/671640/">KCOV</a>), KASAN-sanitized and +KCOV-instrumented code helps developers and testers to detect runtime memory +errors and obtain code coverage information. In the scenario of kernel fuzz +testing, e.g. via <a href="https://github.com/google/syzkaller">syzkaller</a>, +KASAN helps to determine the root cause of crashes, and KCOV provides code +coverage information to the fuzzing engine to help in test-case or corpus +deduplication. +</p> +<p> +This page does not discuss the inner workings or mechanics of KASAN. Rather, it +serves as a guide to build and modify the Android Open Soure Project (AOSP) and +Pixel's kernel source to boot with KASAN and KCOV turned on. +</p> +<h2 id="setting-up-your-build-environment">Setting up your build +environment</h2> +<p> +Follow the steps in the <a +href="/source/requirements">Downloading and +Building</a> section to set up your build environment. +</p> +<h2 id="building-aosp">Building AOSP</h2> +<p> +Download the <a href="/source/downloading">Android source code</a>. For the +purpose of building KASAN images, choose a stable build that is not in active +development. Often, the latest available release/stable branch is a good choice. +More information about build and branch can be found at <a +href="/source/build-numbers#source-code-tags-and-builds">Source +Code Tags and Builds</a>. +</p> + +<p> +After you have successfully checked out the source code, download the necessary +device blobs that correspond to the device and branch you are using from <a +href="https://developers.google.com/android/drivers">Driver Binaries for +Nexus and Pixel Devices</a>. Download both the vendor image and the set of +binaries from the System-on-Chip (SOC) manufacturer. Then, unarchive the +downloaded tarballs, run the scripts they contain, and accept the licenses. +</p> +<aside class="note"> +<strong>Tip</strong>: Double check that you have the <a +href="/source/initializing#installing-the-jdk">right +version of JDK</a> installed on your system before proceeding further. +</aside> +<p> +Then clean up, set up your build environment, and choose your build target, +following the steps in +<a href="/source/building#cleaning-up">Preparing to Build</a>. +</p> + +<p> +To establish a working base, make your first build without modifications: +</p> + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="~/src/aosp$ ">make -j48</code> +</pre> +<p> +Flash your build result to a test device (for example, marlin) and let it boot: +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal" data-terminal-prefix="~/src/aosp$ ">cd out/target/product/marlin</code> +<code class="devsite-terminal" data-terminal-prefix="~/src/aosp/out/target/product/marlin$ ">ANDROID_PRODUCT_OUT=`pwd` fastboot flashall -w</code> +</pre> +<p> +After booting to the homescreen, you might see a pop-up that says: +</p> +<p> +<code>There's an internal problem with your device. Contact your manufacturer +for details.</code> This pop-up likely means that the build fingerprint of your +vendor and your system partition do not match. Because this build is just for +development and testing, and not a release build, just ignore it. +</p> +<h2 id="building-the-kernel">Building the kernel</h2> +<p>To build the kernel, you need to check out the correct source code, +cross-compile it, and then build the kernel image in the correct AOSP +directory.</p> +<h3 id="checking-out-kernel-source-code">Checking out kernel source code</h3> +<p> +Create a directory to store the kernel source code and clone the AOSP kernel git +repository to your local storage. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy">mkdir ~/src/marlin-kernel-src</code> +<code class="devsite-terminal devsite-click-to-copy">cd ~/src/marlin-kernel-src</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/marlin-kernel-src$ ">git clone https://android.googlesource.com/kernel/msm</code> +</pre> +<p> +After you are done, there should be an empty directory named <code>msm</code>. +</p> +<p> +Enter the <code>msm</code> directory and <code>git checkout</code> the branch +that corresponds to the source code you are building. For the list of available +branches and tags, see the <a href="https://android.googlesource.com/kernel/msm/">Android msm +kernel source tree</a>. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/marlin-kernel-src$ ">cd msm</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/marlin-kernel-src$ ">git checkout <var>TAG_NAME</var></code> +</pre> +<p> +After completing this step, the <code>msm</code> directory should have content. +</p> +<h3 id="performing-cross-compilation">Performing cross compilation</h3> +<p> +Next you need to compile the Android kernel. +</p> +<h5 id="setting-up-your-cross-compiler">Setting up your cross-compiler</h5> +<p> +To build the kernel, you need to set up a cross-compiler. The current +recommended and tested toolchain is Android's NDK toolchain latest stable +version. To download the Android NDK, visit the official <a +href="https://developer.android.com/ndk/downloads/index.html">Android NDK +website</a>. Download the appropriate zip archive for your platform, and unzip +it. This results in a directory resembling +<code>android-ndk-<var>NDK_VERSION</var></code>. +</p> +<h5 id="downloading-the-lz4c-tool">Downloading the LZ4c tool</h5> +<p> +The Pixel kernel uses <a hre="//lz4.github.io/lz4/">LZ4 compression</a>, +so the <code>lz4c</code> tool is required when you build your kernel. If you +use Ubuntu, install the <code>lz4c</code> tool by: +</p> + + +<pre class="devsite-terminal devsite-click-to-copy">sudo apt-get install liblz4-tool +</pre> +<h4 id="building-your-kernel">Building your kernel</h4> +<p> +Set up your build environment from the <code>marlin-kernel-src/msm</code> +directory with: +</p> + + +<pre> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">export ARCH=arm64</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">export CROSS_COMPILE=<var>PATH_TO_NDK</var>/android-ndk-<var>NDK_VERSION</var>/toolchains/aarch64-linux-android-<var>TOOLCHAIN_VERSION</var>/prebuilt/linux-x86_64/bin/aarch64-linux-android-</code> +</pre> +<p> +Then build an unmodified version of your kernel to establish a working base: +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">make marlin_defconfig</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">make -j48</code> +</pre> +<p> +The result of the build process can be found at: +<code>arch/arm64/boot/Image.lz4-dtb</code> +</p> +<h4 id="rebuilding-the-boot-image-in-aosp">Rebuilding the boot image in +AOSP</h4> +<p> +After you have built the kernel image, copy the result over into AOSP's +<code>device/google/marlin-kernel</code> directory, with: +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">cp ${marlin-kernel-src}/msm/arch/arm64/boot/Image.lz4-dtb device/google/marlin-kernel</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">source build/envsetup.sh</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">lunch aosp_marlin-userdebug</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">make -j48</code> +</pre> +<p> +After a successful build, flash the target device with: +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">cd out/target/product/marlin</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp/out/target/product/marlin$ ">fastboot flashall -w</code> +</pre> +<p> +After flashing, your device should boot. Verify the image you flashed to +the device is the kernel image you built by checking <code>Kernel +version</code> under <code>Settings -> System -> About phone</code> +after the device finishes booting. +</p> +<h2 id="modifying-the-kernel">Modifying the kernel</h2> +<h3 id="enabling-kasan-and-kcov-compile-options">Enabling KASAN and KCOV compile +options</h3> +<p> +KASAN and KCOV codes are guarded by compilation flags, which are not turned on +for normal builds. To enable them, add KASAN and KCOV options to the config +file, but drop the LZ4 config. +</p> +<p> +To do this, make a copy of the default config file, for example, +<code>marlin_defconfig</code>: +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">cd arch/arm64/configs</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm/arch/arm64/configs$ ">cp marlin_defconfig marlin-kasan_defconfig</code> +</pre> +<p> +In the new config file, remove this flag <code>CONFIG_KERNEL_LZ4=y</code> and +add these flags: +</p> + +<pre class="devsite-click-to-copy">CONFIG_KASAN=y +CONFIG_KASAN_INLINE=y +CONFIG_KCOV=y +CONFIG_SLUB=y +CONFIG_SLUB_DEBUG=y +</pre> + +<h2 id="recompiling-the-kernel-with-new-configuration">Recompiling the kernel +with new configuration</h2> +<p> +After you've finished modifying your copy of the config file, recompile the +kernel. +</p> +<h3 id="reconfiguring-the-kernel">Reconfiguring the kernel</h3> +<p> +Set up your <a href="/source/building-kernels#building">build environment</a>. +Build your modified <code>defconfig</code> and check if the newly added flags +are present in the produced <code>.config</code> file. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">make marlin-kasan_defconfig</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">grep KASAN .config +CONFIG_HAVE_ARCH_<strong>KASAN</strong>=y +CONFIG_<strong>KASAN</strong>=y +# CONFIG_<strong>KASAN</strong>_OUTLINE is not set +CONFIG_<strong>KASAN</strong>_INLINE=y</code> +</pre> +<p> +You should see the KASAN flags. Compile your kernel: +</p> + + +<pre class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">make -j48 +</pre> +<h3 id="checking-the-modified-kernel-image">Checking the modified kernel +image</h3> +<p> +After a successful compilation, navigate to the <code>arch/arm64/boot</code> +directory to view the compilation results. Generally, the +<code>Image.gz-dtb</code> is about 23MB and larger than that of a standard build. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm$ ">cd arch/arm64/boot</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src/msm/arch/arm64/boot$ ">ls -lh Image.gz-dtb +-rw-r--r-- 1 username groupname 23M Aug 11 13:59 Image.gz-dtb</code> +</pre> +<p> +To see if KCOV was properly compiled, perform additional analysis on the +produced <code>vmlinux</code> at the root of the kernel source tree. If you run +an <code>objdump</code> on vmlinux, you should see numerous calls to +<code>__sanitizer_cov_trace_pc()</code>. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="marlin-kernel-src$ ">sh -c '${CROSS_COMPILE}objdump -d vmlinux' | grep sanitizer +ffffffc000082030: 94040658 bl ffffffc000183990 <__sanitizer_cov_trace_pc> +ffffffc000082050: 94040650 bl ffffffc000183990 <__sanitizer_cov_trace_pc> +ffffffc000082078: 94040646 bl ffffffc000183990 <__sanitizer_cov_trace_pc> +ffffffc000082080: 94040644 bl ffffffc000183990 <__sanitizer_cov_trace_pc> +ffffffc0000820ac: 94040639 bl ffffffc000183990 <__sanitizer_cov_trace_pc> +</code></pre> +<h2 id="modifying-aosp-code">Modifying AOSP code</h2> +<p> +Before plugging in the new boot image, you need to adjust certain parameters in +AOSP's source code that govern how the device boots. This is mainly necessary to +ensure the new (inflated) image boots properly. +</p> +<h3 id="adjusting-board-parameters">Adjusting board parameters</h3> +<p> +Adjust the boot parameters defined in the device's <code>BoardConfig.mk</code> +file. It is located at <code>device/google/marlin/marlin</code> relative to the +root of your AOSP source code. +</p> + + +<pre class="devsite-click-to-copy"> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp$ ">cd device/google/marlin/marlin</code> +<code class="devsite-terminal devsite-click-to-copy" data-terminal-prefix="~/src/aosp/device/google/marlin/marlin$ ">vim BoardConfig.mk</code> +</pre> +<aside class="caution"> +<p> +<strong>Caution</strong>: Make sure you have a backup of the original +<code>BoardConfig.mk</code> file before proceeding, in case something goes +wrong. +</p> +<p> +The adjustments to be made can be summarized as follows through a +<code>git diff</code> result: +</p> + + +<pre>diff --git a/marlin/BoardConfig.mk b/marlin/BoardConfig.mk +index 31533fb9..81caf05d 100644 +--- a/marlin/BoardConfig.mk ++++ b/marlin/BoardConfig.mk +@@ -116,15 +116,10 @@ BOARD_EGL_CFG := device/google/marlin/egl.cfg + + BOARD_KERNEL_BASE := 0x80000000 + BOARD_KERNEL_PAGESIZE := 4096 +<var>-ifneq ($(filter marlin_kasan, $(TARGET_PRODUCT)),)</var> + BOARD_KERNEL_OFFSET := 0x80000 + BOARD_KERNEL_TAGS_OFFSET := 0x02500000 + BOARD_RAMDISK_OFFSET := 0x02700000 + BOARD_MKBOOTIMG_ARGS := --kernel_offset $(BOARD_KERNEL_OFFSET) --ramdisk_offset $(BOARD_RAMDISK_OFFSET) --tags_offset $(BOARD_KERNEL_TAGS_OFFSET) +<var>-else +-BOARD_KERNEL_TAGS_OFFSET := 0x02000000 +-BOARD_RAMDISK_OFFSET := 0x02200000 +-endif</var> + + TARGET_KERNEL_ARCH := arm64 + TARGET_KERNEL_HEADER_ARCH := arm64 +</pre> +</aside> + +<p> +If you do not wish to modify the <code>BoardConfig.mk</code> +file, you could instead create a new lunch target that contains the name +<code>marlin_kasan</code>. For more information about this process, see +<a href="/source/add-device">Adding a New Device</a>. +</p> + +<h3 id="adjusting-the-kernel-target-in-the-local-makefile">Adjusting the kernel +target in the local Makefile</h3> +<p> +The new kernel uses LZ4 compression to improve speed, but KASAN requires gzip +for better compression ratio. To accommodate this, tell the build machinery +which kernel to bundle into the final target by modifying where the +<code>LOCAL_KERNEL</code> variable points to in +<code>device/google/marlin/device-common.mk</code>. +</p> +<h2 id="rebuilding-boot-image">Rebuilding the boot image</h2> +<p> +To rebuild the boot image, copy the new kernel image into the AOSP tree in the +device-specific folder (e.g. <code>device/google/marlin-kernel</code>). Make +sure this is where the build system expects the kernel target image to be +at, according to how you modified it earlier. +</p> +<p> +Next, rebuild your flashable images, similar to how you <a +href="#building-aosp">built AOSP</a> earlier. Upon successful build, flash all +built images as usual. +</p> +<h2 id="booting-your-device-with-a-modified-kernel-image">Booting your device +with a modified kernel image</h2> +<p> +You should now have a build that boots and enters the home screen. From here, +check the device's <code>dmesg</code> output for a "<code>KernelAddressSanitizer +initialized</code>" message in the very early boot stage. That message means +KASAN is initialized during boot time. Also, you can confirm +<code>/sys/kernel/debug/kcov</code> is present on the device (you will have to +be root to do that). +</p> +<h2 id="troubleshooting">Troubleshooting</h2> +<p> +You can experiment with different kernel versions, starting with a standard +build as a working base, before turning on KASAN+KCOV compile options. When +things break, first check if the bootloader and baseband version on your device +matches those required by the new build. Finally, you might have to +catch up with a newer branch of the Android tree altogether if you venture too +far ahead with the kernel version. +</p> +</body> +</html> |