Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

As most malware is infectious, anti-analysis and packing techniques supported by commercial protectors are conventionally applied to hinder analysis. When analyzing to detect and block such protected malware, it is necessary to do so in a virtual environment to prevent infection. In terms of packing, it is necessary to analyze using dynamic binary instrumentation (DBI), a dynamic analysis tool, which is advantageous for unpacking because DBI inserts code at run time and analyzes it dynamically. However, malware terminates on its own when it detects a virtual environment or DBI due to anti-analysis techniques. Therefore, it is necessary to also bypass anti-VM and anti-DBI techniques in order to successfully analyze malware in a virtual environment using DBI. It is very difficult for analysts to bypass anti-VM and anti-DBI techniques that are used in commercial protectors because analysts generally have little information on what methods are used or how to even bypass these techniques. In this paper, we suggest guidelines to aid in easy analysis of malware protected by anti-VM and anti-DBI techniques supported by commercial protectors. We analyzed the techniques used by five of the most common commercial protectors, and herein present how to bypass anti-VM and anti-DBI techniques supported by commercial protectors via a detailed algorithm analysis. We performed a bypass experiment after applying each commercial protector to 1573 executable files containing vulnerabilities provided by the National Institute of Standards and Technology (NIST). To our knowledge, this is the first empirical study to suggest detailed bypassing algorithms for anti-VM and anti-DBI techniques used in commercial protectors.

Automated summary

This study found that in order to successfully analyze protected malware, dynamic binary instrumentation (DBI) should be used in a virtual environment, while bypassing anti-VM and anti-DBI techniques supported by commercial protectors. It is generally difficult for analysts to bypass these techniques used in commercial protectors.