Deobfuscation of malicious software using LLVM intermediate representation

Software security
Authors:
Abstract:

The problem of automating deobfuscation of malicious software is considered. A method based on the LLVM intermediate representation is proposed that combines dynamic unpacking with tracing, hybrid (traceassisted) restoration of the control flow graph and iterative devirtualization. A software prototype has been developed that implements the proposed method. An experimental evaluation was carried out, confirming the applicability of the approach to removing class obfuscation: packaging, control flow distortion, instruction obfuscation, and code virtualization.