Operating System Interface Obfuscation and the Revealing of Hidden Operations

Thumbnail Image
Srivastava, Abhinav
Lanzi, Andrea
Giffin, Jonathon
Associated Organizations
Organizational Unit
Organizational Unit
Supplementary to
Many software security solutions—including malware analyzers, information flow tracking systems, auditing utilities, and host-based intrusion detectors—rely on knowledge of standard system call interfaces to reason about process execution behavior. In this work, we first obfuscate the Windows and Linux system call interfaces to degrade the effectiveness of these tools. Our attack, called Illusion, invokes privileged kernel operations in the kernel at the request of user-level processes without requiring those processes to call the actual system calls corresponding to the operations. The Illusion interface hides system operations from user-, kernel-, and hypervisor-level monitors mediating the conventional system-call interface. Illusion alters neither static kernel code nor read-only dispatch tables, remaining elusive from tools protecting kernel memory. We then consider the problem of Illusion attacks and augment system call data with kernel-level execution information to expose the hidden kernel operations. We present a Xen-based monitoring system, Sherlock, that adds kernel execution watchpoints to the stream of system call events. Sherlock automatically adapts its sensitivity based on security requirements to remain performant on desktop systems.
Date Issued
Resource Type
Resource Subtype
Technical Report
Rights Statement
Rights URI