aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-04-28interceptor: Skip --param.HEADmastermainYifan Hong
E.g. for the following command: clang [...] --param asan-instrument-allocas=1 [...] Test: TH Bug: 221278445 Change-Id: I6117d7d8d08b7fe9b5139c0ec0740a8155b93ffd
2021-12-20Add -x to skipped argument for clang.Yifan Hong
clang -x <language>: Treat subsequent input files as having type <language> Test: builds with cuttlefish Fixes: 211512820 Change-Id: I32f72a2b1ebfc12d6a0aecc80274009a8659c1b5
2021-12-14add readme.mdYifan Hong
Change-Id: Ie82d4e210ba8595412c05689e9fb34df88f7a5ac
2021-12-09main: ensure the correct return code is emittedMatthias Maennich
std::system's return value needs additional treatment to make it the expected return value resembling what the wrapped process returned. Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ib335a383ce07d0311df1b3f6ba8e7addd26360ee
2021-12-09interceptor: give analyzers namesMatthias Maennich
This is useful to recognize the used analyzer from the interceptor log. Bug: 205577427 Change-Id: Iaf536b87341acc533fdf2675c5b1b933e6061450
2021-12-09interceptor: add an initial implementation for fake executionMatthias Maennich
Add an option to allow fake execution. This is enabled by running the interceptor with --fake | -f, which it will enable analyzers during execution to rewrite the command before execution (but after logging it away). The default_fake implementation replaces the entire command by an equivalent of `sh -c "truncate --size 0 <output> <output> ..."`. That effectively touches all outputs leaving empty files behind. This is enabled for the existing analyzers and possibly a good default for many more. Yet there is no guarantee that this will actually make the build succeed if downstream rules that are not captured by analyzers stumble when working with the empty files. The reason for wrapping the `truncate` in /bin/sh is to avoid having to do the PATH lookup - $(which truncate) - ourselves. Hermetic toolchains might provide a different `truncate` than /usr/bin/truncate, e.g. the toybox variant. Bug: 205735739 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ib8478f52940071d07f6544d8ac4804048404ca64
2021-12-08interceptor: store relative current directoriesMatthias Maennich
In one way or the other, the current directories of a command are always relative to the root_dir. Hence, just make them relative at collection time and remove this redundant information. This reduces the command log of a kernel build by ~10%. Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I6c4edb2a67b1d37a91e21e68d603876d96a64761
2021-12-08Merge "interceptor_analysis supports --relative option."Treehugger Robot
2021-12-08interceptor: add FixdepAnalyzerMatthias Maennich
This implements tracing for the fixdep command of kernel builds. In particular it allows us to run this command in dry-run mode later. Bug: 205735739 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I4ea3f41edcd64150be0a890e59356e48de4b036d
2021-12-08interceptor: tidy up default_make_relativeMatthias Maennich
This removes some outstanding code smells identified in aosp/1898997. Some correctness fixes, some cleanups. Bug: 207620569 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ic31b2ac1ef18b2a93b99dc2e303f7c6ccd66d1d6
2021-12-08interceptor: consolidate the execution for execve and posix_spawnMatthias Maennich
Too much code duplication accumulated, which can now be consolidated. Do that and bring back the execute() function that can now deal with a callback to the original libc method. Bug: 209401200 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I85b4c029a6a6153bbabf852cc44c03ff52d2edb1
2021-12-08interceptor: add support for intercepting posix_spawnMatthias Maennich
posix_spawn is another common way of spawning subprocesses via libc. Add support for this method as well as `make` makes use of it when spawning subprocesses. Bug: 209401200 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ib4e1cc6bba19c9b52d819dac4b5140a39fb83f83
2021-12-08interceptor: allow recognized commands to terminate recursionMatthias Maennich
If a recognized command is detected, e.g. a compiler invocation, allow analyzers to stop recursion of tracing. That will effectively not instrument any subprocesses with LD_PRELOAD any longer to avoid needless (and confusing) recording of internal subprocess. Examples of those are the linker/assembler invocations of compilers. Bug: 209401200 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I92e60f309de38438bc2edb85c750b565dd464c5a
2021-12-08interceptor: refactor argv_vector() out of execute_execveMatthias Maennich
This allows later reuse when adding support for intercepting posix_spawn(). Bug: 209401200 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I035d4e2a2e2539c073a9c8bcf919872137fb2d28
2021-12-08interceptor: fix reporting of missing inputsMatthias Maennich
When missing inputs are reported, the Command's debug representation was printed before the inputs/outputs where filled. That is not ideal. Fix that by doing that check after the assignment. Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I8cabd3243c9decee480daad151d69d4c77852b60
2021-12-08interceptor: make the execve execution explicitMatthias Maennich
Track transformations through analyzers (currently only through make_relative()) and based on that, fall back to the original execve with either the transformed command or just with the original data passed in. This avoids unnecessary execute_execve calls if the command is unmodified and prepares for later support of additional intercepted libc calls for subprocess execution, such as posix_spawn(). Bug: 209401200 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I5c8ce3d5df55fc365b12e5406ac575145d6c035d
2021-12-07interceptor_analysis supports --relative option.Yifan Hong
This allows creating compile_commands.json with relative paths so that Kythe won't search for non-existing sandbox directories. Test: build kythe Bug: 201801372 Change-Id: Iffcfa9a6e98ad062595fc52fd45d16d1e9b8e5f8
2021-12-07Recognize -t text.Yifan Hong
Test: manual Change-Id: Ib28f10e0688a96c93a451f509f73d7e9b38d2692
2021-12-02interceptor: avoid concurrency on the command logMatthias Maennich
Using the protobuf util to serialize to the stream can lead to corrupted intermediate logs due to concurrency. Ensure only one process is writing to the log file at a time by directly writing to the log file in a single write call. Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I043f349c130cfe307bd51030a1708c6afd122fdb
2021-12-02interceptor: add error handling for unparseable entriesMatthias Maennich
In case entries are unparseable, skip over them and emit an error message. Since the delimited log is always |<sizeof(message1)><message1>|<sizeof(messsage2)><message2>|..., it is safe to skip entries that can't be read in order to recover most of the log. But since there is an underlying problem, ensure we emit an error message. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I8d1233bb22889c36bf1ecc3e6bd5b144333e0e80
2021-12-01compdb: add mode to emit a "command" instead of "arguments"Matthias Maennich
Add this variant through the mode "compdb_commands" to support indexing systems that require "command" to be set instead of "arguments". Bug: 205731786 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ic9d425abce1b0ad5d48183fd555b72871a53de88
2021-12-01interceptor: relocate utility functions into separate static libMatthias Maennich
This allows using the utils without having to link the entire libinterceptor from the analysis tooling. Bug: 205731786 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ieec738c4be9b95d33af3d09546b0e5964008ed34
2021-11-29interceptor: selectively enable making commands relativeMatthias Maennich
Enabling/Disabling are poor choices for making commands relative. Making all relative breaks some, making none relative misses out on the ones that can benefit from it. Hence, selectively enable make_relative for commands that are known to be safe. Bug: 206894758 Change-Id: I737aa2d06b303cae836abc453e34f28056120291 Signed-off-by: Matthias Maennich <maennich@google.com>
2021-11-29interceptor: refactor instantiation of analyzersMatthias Maennich
Make analyzers classes that are instantiated from a command and that implement common functionality, like determining inputs and outputs. That way the differences between certain commands can be defined much more precise. This change does not intend a change of behaviour. Bug: 206894758 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I42d1006b713371581f0159357b512f5a58a6842c
2021-11-26interceptor: put rewriting commands under flag controlMatthias Maennich
Not every command can be rewritten to execute relative. For now, make this feature opt-in. In a later iteration we can enable it for "safe" commands, like compilers/linkers that we can entirely judge. Bug: 206894758 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Id624d5486dcd68a07b62c02d55d86492c75cca0f
2021-11-26interceptor: generalize environment variable accessMatthias Maennich
Generalize the access to single environment variable values as this is a commonly used functionality. command_getenv returns an optional to indicate whether the value had been set at all and hence allows contextual boolean use, e.g. if (command_getenv(command, kEnvVariable)) { // something } Bug: 206894758 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Icda8d0709b6fe394630729c245a0486fcf718157
2021-11-24interceptor: update license documentationMatthias Maennich
Bug: 206149924 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I3340d58c6a14e98f1fe48ef104e00321fac56881
2021-11-24interceptor: relocate the interceptor to tools/interceptorMatthias Maennich
This makes adjustments to move the interceptor out of the prebuilts and into its final destination. Bug: 206149924 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I86e7ee9cb86ddf153e10c00a5d0e9a42a48b3e59
2021-11-24interceptor: protect from unexpectedly empty argument listsMatthias Maennich
Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Iab3179321d4c727d741d0dee96e3b2c7440201fb
2021-11-24interceptor: use consistent non-abbreviated namingMatthias Maennich
This aligns the proto naming as well as the internal naming for constants and variables. Bug: 207620215 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I31dbf812d48abafe3343cac8cccd5ea4727b8fb9
2021-11-24interceptor: consistent constant namingMatthias Maennich
Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I884c21e272e6a3f1ede59b2c5833fdb3983e2879
2021-11-24interceptor: consistently use curly braces for all statementsMatthias Maennich
Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I80b2773557ec561b944d63dd826fa5a08c0bb849
2021-11-24interceptor: replace repr() method by proper operator<<Matthias Maennich
That gets rid of the legacy repr() implementation of Command. Replace it with a proper ostream operator<<. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ie96b33f9067841da28d5ef8082140ee67c433a76
2021-11-24interceptor: replace custom Command class by proto classMatthias Maennich
Though this removes the lazy evaluation of args and env to a be always done now, this hardly every was truly omitted. Hence, this is not a regression. Instead the code could be tidied up to have only one data format that also serializes. No functional change intended. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ia3790d36c5477eb39dd6980d35b638803726d49d
2021-11-23interceptor: Support for emitting compile_commands.jsonMatthias Maennich
The analysis command learned how to emit compile_command.json databases based on the recorded command log. A sample execution is: $ interceptor_analysis -l /tmp/log -o compile_command.json -t compdb Bug: 205731786 Bug: 201801372 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I00d08d309f51b9263900cab517baa824c3be45c2
2021-11-22interceptor: record root dir in logMatthias Maennich
That is required to reconstruct the full working dir again later. The root dir is either deducted from the environment ($ROOT_DIR) or is cwd() of the interceptor process otherwise. Bug: 205731786 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I12d826d7459a83864ecd024d85c801f1011aee87
2021-11-22Merge changes I75f52e68,Icff5de48Matthias Maennich
* changes: interceptor: Android.bp: add missing license header interceptor: Add CMakeLists.txt build description for local development
2021-11-22Merge changes I10b3e46d,I29cde728,I383723e5,Idbecb166,I0f70c545Matthias Maennich
* changes: Interceptor: initial analysis tooling interceptor: replace JSON log by proto log interceptor: add initial proto log format interceptor: factor out the command line string creation interceptor: introduce default compilation options
2021-11-21interceptor: Android.bp: add missing license headerMatthias Maennich
Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I75f52e68e6f2f08306f30821b64204a2ecb7aa4a
2021-11-21interceptor: Add CMakeLists.txt build description for local developmentMatthias Maennich
This makes it easier to locally develop the interceptor with different configurations, such as debug, sanitized, etc. Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Icff5de48e70e27441f69ce48ce34b4e22111e8d8
2021-11-20Interceptor: initial analysis toolingMatthias Maennich
Add analysis tooling to process interceptor logs. As a first utility, add --output-type text to dump the log as pb text. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I10b3e46df6b7deb070710aa4ac9ab94a5486d43a
2021-11-20interceptor: replace JSON log by proto logMatthias Maennich
This implements serialization of commands to a corresponding protobuf based log protocol. Child processes append log::Message to the log (currently this implies a 'Command'). The parent process will reread the log before terminating and will combine the log to a more compact log::Log for further processing. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I29cde728601fc3796f9b3ff0ff1b1d030bb86678
2021-11-20interceptor: add initial proto log formatMatthias Maennich
This serves as a stable binary format for communication of state. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I383723e5665a47769d7fbfc4187b7864476b59a1
2021-11-20interceptor: factor out the command line string creationMatthias Maennich
Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Idbecb16660ab055056269184091569264160bcc8
2021-11-20interceptor: introduce default compilation optionsMatthias Maennich
Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I0f70c54538b670f15ece913f7874f8ae15b84f71
2021-11-19interceptor: add OWNERSMatthias Maennich
Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I7f23a401f265d0d91006f408a907c131cddcf887
2021-11-15Replace interceptor starter with C++ binaryMatthias Maennich
This avoids the shell script to grow and allows for additional features to be implemented in the future, e.g. communication from the starter to the child processes. Bug: 205732988 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I055530151ce00c600926994fd87392f404ceb375
2021-11-09Interceptor: log inputs/outputs as wellMatthias Maennich
Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ie6b123d97c10a6a1931c8888d85fe63a68aa72e8
2021-11-09Interceptor: add command analysisMatthias Maennich
Add first version of the command analysis utilities. We match the program executed against a set of (command line) analyzers and delegate the detection of any inputs/outputs that can be derived from the command line. For the moment, this only implements analyzers for clang/gcc compilers/linkers as well as archivers (ar, llvm-ar). Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: I72791d05a946a4210be6322358003a7250912b80
2021-11-09Interceptor: relativate execution based on `cwd`Matthias Maennich
`ROOT_DIR` - if set, otherwise assume `cwd` - is assumed to be the directory all execution is done relatively to. Hence, when rewriting commands, do it with that directory as a base. The interceptor wrapper is determining the INTERCEPTOR_root_dir based on either ROOT_DIR or falls back to `cwd`. Bug: 205577427 Signed-off-by: Matthias Maennich <maennich@google.com> Change-Id: Ifef91ce7ae446d1a6cea7a744f36d0ff27cb4b3e
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-06 15:09:22 +0000