1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
|
<html devsite>
<head>
<title>Debugging Native Android Platform Code</title>
<meta name="project_path" value="/_project.yaml" />
<meta name="book_path" value="/_book.yaml" />
</head>
<body>
<!--
Copyright 2017 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<p>This section summarizes useful tools and related commands for debugging,
tracing, and profiling native Android platform code when developing
platform-level features. This page covers use of <code>debuggerd</code>, a
daemon process for collecting error information after applications crash, and
the GNU Project debugger (GDB).</p>
<p>Other pages in this section explore system services with
<a href="/devices/tech/debug/dumpsys.html">Dumpsys</a>, viewing
<a href="/devices/tech/debug/native-memory.html">native memory</a>,
<a href="/devices/tech/debug/netstats.html">network</a>, and
<a href="/devices/tech/debug/procstats.html">RAM</a> usage, using
<a href="/devices/tech/debug/asan.html">AddressSanitizer</a> to detect memory
bugs in native code, evaluating
<a href="/devices/tech/debug/eval_perf.html"> performance issues</a> (includes
<a href="/devices/tech/debug/systrace">systrace</a>), and several other
debugging tools.</p>
<h2 id=debuggerd>Using debuggerd</h2>
<p>The <code>debuggerd</code> process dumps registers and unwinds the stack.
When a dynamically linked executable starts, several signal handlers are
registered that connect to <code>debuggerd</code> (or <code>debuggerd64)</code>
in the event that signals (such as SIGSEGV or SIGABRT) are sent to the process.</p>
<p>It's possible for <code>debuggerd</code> to attach only if nothing else is
already attached, which means using tools such as <code>strace</code> or
<code>gdb</code> will prevent <code>debuggerd</code> from working. You can also
explicitly prevent <code>debuggerd</code> from attaching by calling
<code>prctl(PR_SET_DUMPABLE, 0)</code>, which can be useful when you need to
opt out of crash reporting.</p>
<p>Example <code>debuggerd</code> output (with timestamps and extraneous
information removed):</p>
<pre class="no-pretty-print">
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'Android/aosp_angler/angler:7.1.1/NYC/enh12211018:eng/test-keys'
Revision: '0'
ABI: 'arm'
pid: 17946, tid: 17949, name: crasher >>> crasher <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
r0 0000000c r1 00000000 r2 00000000 r3 00000000
r4 00000000 r5 0000000c r6 eccdd920 r7 00000078
r8 0000461a r9 ffc78c19 sl ab209441 fp fffff924
ip ed01b834 sp eccdd800 lr ecfa9a1f pc ecfd693e cpsr 600e0030
backtrace:
#00 pc 0004793e /system/lib/libc.so (pthread_mutex_lock+1)
#01 pc 0001aa1b /system/lib/libc.so (readdir+10)
#02 pc 00001b91 /system/xbin/crasher (readdir_null+20)
#03 pc 0000184b /system/xbin/crasher (do_action+978)
#04 pc 00001459 /system/xbin/crasher (thread_callback+24)
#05 pc 00047317 /system/lib/libc.so (_ZL15__pthread_startPv+22)
#06 pc 0001a7e5 /system/lib/libc.so (__start_thread+34)
Tombstone written to: /data/tombstones/tombstone_06
</pre>
<p>The last line of <code>debuggerd</code> output dumps a summary to the log and
writes a full <em>tombstone</em> to disk. The tombstone is simply a file with
extra data about the crashed process; it contains information that can be
helpful in debugging a crash, in particular the stack traces for all threads in
the crashing process (not just the thread that caught the signal) and a full
memory map.</p>
<p>Assuming the unstripped binaries can be found, you can get a more detailed
unwind with line number information by pasting the above example into
<code>development/scripts/stack</code>:</p>
<p class="key-point"><strong>Tip:</strong> For convenience, if you've lunched
<code>stack</code> will be on your $PATH already so you don't need to give the
full path.</p>
<pre>
$ development/tools/stack
Reading native crash info from stdin
03-02 23:53:49.477 17951 17951 F DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
03-02 23:53:49.477 17951 17951 F DEBUG : Build fingerprint: 'Android/aosp_angler/angler:7.1.1/NYC/enh12211018:eng/test-keys'
03-02 23:53:49.477 17951 17951 F DEBUG : Revision: '0'
03-02 23:53:49.477 17951 17951 F DEBUG : ABI: 'arm'
03-02 23:53:49.478 17951 17951 F DEBUG : pid: 17946, tid: 17949, name: crasher >>> crasher <<<
03-02 23:53:49.478 17951 17951 F DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
03-02 23:53:49.478 17951 17951 F DEBUG : r0 0000000c r1 00000000 r2 00000000 r3 00000000
03-02 23:53:49.478 17951 17951 F DEBUG : r4 00000000 r5 0000000c r6 eccdd920 r7 00000078
03-02 23:53:49.478 17951 17951 F DEBUG : r8 0000461a r9 ffc78c19 sl ab209441 fp fffff924
03-02 23:53:49.478 17951 17951 F DEBUG : ip ed01b834 sp eccdd800 lr ecfa9a1f pc ecfd693e cpsr 600e0030
03-02 23:53:49.491 17951 17951 F DEBUG :
03-02 23:53:49.491 17951 17951 F DEBUG : backtrace:
03-02 23:53:49.492 17951 17951 F DEBUG : #00 pc 0004793e /system/lib/libc.so (pthread_mutex_lock+1)
03-02 23:53:49.492 17951 17951 F DEBUG : #01 pc 0001aa1b /system/lib/libc.so (readdir+10)
03-02 23:53:49.492 17951 17951 F DEBUG : #02 pc 00001b91 /system/xbin/crasher (readdir_null+20)
03-02 23:53:49.492 17951 17951 F DEBUG : #03 pc 0000184b /system/xbin/crasher (do_action+978)
03-02 23:53:49.492 17951 17951 F DEBUG : #04 pc 00001459 /system/xbin/crasher (thread_callback+24)
03-02 23:53:49.492 17951 17951 F DEBUG : #05 pc 00047317 /system/lib/libc.so (_ZL15__pthread_startPv+22)
03-02 23:53:49.492 17951 17951 F DEBUG : #06 pc 0001a7e5 /system/lib/libc.so (__start_thread+34)
03-02 23:53:49.492 17951 17951 F DEBUG : Tombstone written to: /data/tombstones/tombstone_06
Reading symbols from /huge-ssd/aosp-arm64/out/target/product/angler/symbols
Revision: '0'
pid: 17946, tid: 17949, name: crasher >>> crasher <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
r0 0000000c r1 00000000 r2 00000000 r3 00000000
r4 00000000 r5 0000000c r6 eccdd920 r7 00000078
r8 0000461a r9 ffc78c19 sl ab209441 fp fffff924
ip ed01b834 sp eccdd800 lr ecfa9a1f pc ecfd693e cpsr 600e0030
Using arm toolchain from: /huge-ssd/aosp-arm64/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/
Stack Trace:
RELADDR FUNCTION FILE:LINE
0004793e pthread_mutex_lock+2 bionic/libc/bionic/pthread_mutex.cpp:515
v------> ScopedPthreadMutexLocker bionic/libc/private/ScopedPthreadMutexLocker.h:27
0001aa1b readdir+10 bionic/libc/bionic/dirent.cpp:120
00001b91 readdir_null+20 system/core/debuggerd/crasher.cpp:131
0000184b do_action+978 system/core/debuggerd/crasher.cpp:228
00001459 thread_callback+24 system/core/debuggerd/crasher.cpp:90
00047317 __pthread_start(void*)+22 bionic/libc/bionic/pthread_create.cpp:202 (discriminator 1)
0001a7e5 __start_thread+34 bionic/libc/bionic/clone.cpp:46 (discriminator 1)
</pre>
<p class="note"><strong>Note:</strong> Some system libraries are built with
<code>LOCAL_STRIP_MODULE := keep_symbols</code> to provide usable backtraces
directly from <code>debuggerd</code> without taking up anywhere near as much
space as an unstripped version.</p>
<p>You can also <code>stack</code> an entire tombstone. Example:</p>
<pre>
$ stack < FS/data/tombstones/tombstone_05
</pre>
<p>This is useful if you've just unzipped a bugreport in the current directory.
For more information about diagnosing native crashes and tombstones, see
<a href="/devices/tech/debug/native-crash.html">Diagnosing Native Crashes</a>.
</p>
<h3>Getting a stack trace/tombstone from a running process</h3>
<p>You can also use <code>debuggerd</code> on a running process. From the
command line, invoke <code>debuggerd</code> using a process ID (PID) to dump the
full tombstone to <code>stdout</code>. To get just the stack for every thread in
the process, include the <code>-b</code> or <code>--backtrace</code> flag.
<h2 id=native>Using GDB</h2>
<p>The GNU Project debugger (GDB) is a commonly used Unix debugger.</p>
<h3 id=running>Debugging a running app</h3>
<p>To connect to an already-running app or native daemon, use
<code>gdbclient</code> with a PID. For example, to debug the process with PID
1234, run:</p>
<pre class="no-pretty-print">
$ gdbclient 1234
</pre>
<p>The script sets up port forwarding, starts the appropriate
<code>gdbserver</code> on the device, starts the appropriate <code>gdb</code> on
the host, configures <code>gdb</code> to find symbols, and connects
<code>gdb</code> to the remote <code>gdbserver</code>.</p>
<h3 id=starts>Debugging a native process as it starts</h3>
<p>To debug a process as it starts, use <code>gdbserver</code> or
<code>gdbserver64</code> (for 64-bit processes). For example:</p>
<pre class="no-pretty-print">
$ adb shell gdbserver :5039 /system/bin/<em>my_test_app</em>
Process my_test_app created; pid = 3460
Listening on port 5039
</pre>
<p>Next, identify the application PID from the <code>gdbserver</code> output and
use it in another terminal window:</p>
<pre class="no-pretty-print">
$ gdbclient <em><app pid></em>
</pre>
<p>Finally, enter <strong>continue</strong> at the <code>gdb</code> prompt.</p>
<p class="note"><strong>Note:</strong> If you use the wrong
<code>gdbserver</code>, you'll get an unhelpful error message (such as
"<code>Reply contains invalid hex digit 59</code>").</p>
<h3 id=crash>Debugging processes that crash</h3>
<p>If you want <code>debuggerd</code> to suspend crashed processes so you can
attach <code>gdb</code>, set the appropriate property:</p>
<pre class="no-pretty-print">
# Android 7.0 Nougat and later.
$ adb shell setprop debug.debuggerd.wait_for_gdb true
# Android 6.0 Marshmallow and earlier.
$ adb shell setprop debug.db.uid 999999
</pre>
<p>At the end of the usual crash output, <code>debuggerd</code> provides
instructions on how to connect <code>gdb</code> using the command:
<pre class="no-pretty-print">
$ gdbclient <pid>
</pre>
<h3 id=symbols>Debugging without symbols</h3>
<p>For 32-bit ARM, if you don’t have symbols, <code>gdb</code> can get confused
about the instruction set it is disassembling (ARM or Thumb). To specify the
instruction set chosen as the default when symbol information is missing, set
the following property:</p>
<pre class="no-pretty-print">
$ set arm fallback-mode arm # or thumb
</pre>
</body>
</html>
|
