Trusted detection of ransomware in a private cloud using machine learning methods leveraging meta-features from volatile memory

Cloud computing is one of today's most popular and important IT trends. Currently, most organizations use cloud computing services (public or private) as part of their computer infrastructure. Virtualization technology is at the core of cloud computing, and virtual resources, such as virtual servers, are commonly used to provide services to the entire organization. Due to their importance and prevalence, virtual servers in an organizational cloud are constantly targeted by cyber-attackers who try to inject malicious code or malware into the server (e.g., ransomware). Many times, server administrators are not aware that the server has been compromised, despite the presence of detection solutions on the server (e.g., antivirus engine). In other cases, the breach is detected after a long period of time when significant damage has already occurred. Thus, detecting that a virtual server has been compromised is extremely important for organizational security. Existing security solutions that are installed on the server (e.g., antivirus) are considered untrusted, since malware (particularly sophisticated ones) can evade them. Moreover, these tools are largely incapable of detecting new unknown malware. Machine learning (ML) methods have been shown to be effective at detecting malware in various domains. In this paper, we present a novel methodology for trusted detection of ransomware in virtual servers on an organization's private cloud. We conducted trusted analysis of volatile memory dumps taken from a virtual machine (memory forensics), using the Volatility framework, and created general descriptive meta-features. We leveraged these meta-features, using machine learning algorithms, for the detection of unknown ransomware in virtual servers. We evaluated our methodology extensively in five comprehensive experiments of increasing difficulty, on two different popular servers (IIS server and an email server). We used a collection of real-world, professional, and notorious ransomware and a collection of legitimate programs. The results show that our methodology is able to detect anomalous states of a virtual machine, as well as the presence of both known and unknown ransomware, obtaining the following results: TPR =1, FPR= 0.052, F-measure = 0.976, and AUC= 0.966, using the Random Forest classifier. Finally, we showed that our proposed methodology is also capable of detecting an additional type of malware known as a remote access Trojan (RAT), which is used to attack organizational VMs. (C) 2018 Elsevier Ltd. All rights reserved.