An IR-based Fuzzing Approach for Finding Context-Aware Bugs in API-based systems
Author(s)
Xu, Wen
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
API-based systems, a large group of security-critical software programs including web browser and OS kernels, accept program inputs being composed of API calls. Considering the scale and complexity of an API-based system, fuzzing proves to be the most effective approach for bug detection in practice. To effectively discover new bugs in an API-based system nowadays, a fuzzer needs to generate syntactically and semantically correct API calls, which are not declined at an early stage.
Grammar-based API fuzzers generate random API calls in various syntaxes described by context-free grammars. Nevertheless, context-free grammars are unable to deliver certain API semantics in an API program, especially how an API call interacts with the objects in the program. Therefore, the random API calls generated by such fuzzers largely have reference errors, type errors or state errors.
To effectively fuzz an API-based system, we present a context-aware fuzzing approach, which relies on RPG IR to generate random API calls. RPG IR is a formal and contextual representation that defines an object-based context for an API program and models not only the syntax but also the context-based semantics of every API call in the program. Hence, the generated API calls in RPG IR have reduced semantic errors and are more likely to trigger bugs in an API-based system. To evaluate the effectiveness of RPG IR in API fuzzing, we present FreeDom and Janus, two IR-based context-aware fuzzers targeting web browsers and file systems, respectively. In particular, FreeDom has revealed 24 previously unknown bugs in Apple Safari, Mozilla Firefox, and Google Chrome, 10 of which are assigned with CVEs. Meanwhile, FreeDom largely outperforms the grammar-based DOM fuzzer, Domato, with 3× more unique crashes. On the other hand, Janus visits at most 4.19× more code paths compared to the state-of-the-art system call fuzzer, Syzkaller, by generating context-aware file operations. More importantly, Janus has found 90 bugs in eight Linux file systems with 32 CVEs assigned.
We further present RPG (Random Program Generator), a more generic approach to conduct context-aware API fuzzing via RPG IR against different API-based systems. In particular, RPG accepts API description in ASL (API Specification Language), a formal language for developers to describe APIs that can be modeled by RPG IR. RPG manages to compile ASL files into a context-aware API fuzzer based on RPG IR specifically targeting the described APIs. We implement a prototype of RPG, which is evaluated by fuzzing WebKit with the ASL files that describe DOM and SVG specifications. As a domain-agnostic approach, RPG manages to discover a similar number of code blocks and unique crashes compared to FreeDom.
Sponsor
Date
2021-05-03
Extent
Resource Type
Text
Resource Subtype
Dissertation