Malware Analysis and Evasion

Dr. Greg Bernstein

March 16th, 2021

Malware Analysis


  • Malware and Attack Technologies Knowledge Area Issue 1.0, Wenke Lee Georgia Institute of Technology, 2019. PDF CyBOK Malware

  • Ori Or-Meir, Nir Nissim, Yuval Elovici, and Lior Rokach. 2019. Dynamic Malware Analysis in the Modern Era—A State of the Art Survey. ACM Comput. Surv. 52, 5, Article 88 (October 2019), 48 pages. open access Dynamic Analysis Survey


From CyBOK Malware

“malware analysis is the basis for detecting and responding to cyberattacks”

Why: Remediation

From CyBOK Malware

First, we can understand the intended malicious activities to be carried out by the malware. This will allow us to update our network and endpoint sensors to detect and block such activities, and identify which machines have the malware and take corrective actions such as removing it or even completely wiping the computer clean and reinstalling everything.

Why: Attribution

From CyBOK Malware

Second, by analyzing the malware structure (e.g., the libraries and toolkits that it includes) and coding styles, we may be able to gain information that is potentially useful to attribution, which means being able to identify the likely author and operator.

Why: Trend Analysis

From CyBOK Malware

Third, by comparing it with historical as well as geo-location data, we can better understand and predict the scope and trend of malware attacks, e.g., what kinds of activities (e.g., mining cryptocurrencies) are on the rise and if a cybercrime is moving from one region to another.

Malware Acquisition: Intentional 1

From CyBOK Malware

There are ways to ‘capture’ malware instances on the infection sites. A network sensor can examine traffic (e.g., web traffic, email attachment) to identify possible malware (e.g., payload that contains binary or program-like data from a website with a low reputation)… If a network sensor is able to detect outgoing malicious traffic from an internal host, a host-based sensor can further identify the program, i.e., the malware, responsible for such traffic.

Malware Acquisition: Intentional 2

From CyBOK Malware

There are also malware collection and sharing efforts where trusted organisations can upload malware samples found in their networks and also receive samples contributed by other organizations. Academic researchers can typically just obtain malware samples without needing to contribute.

Static Analysis

From CyBOK Malware

Static analysis involves examining the code (source, intermediate, or binary) to assess the behaviors of a program without actually executing it. …One limitation is that the analysis output may not be consistent with the actual malware behaviors (at runtime). This is because in many cases it is not possible to precisely determine a program’s behaviors statically (i.e., without the actual run-time input data).

Static Analysis: Additional Info

Dynamic Analysis 1

From CyBOK Malware

Dynamic analysis monitors the behaviors of malware execution in order to identify malicious behaviors. Dynamic analysis identifies the precise program behaviors per the test input cases but misses behaviors that are not triggered by the input.

Dynamic Analysis 2

From CyBOK Malware

Additionally, dynamical analysis can defeat code obfuscation techniques designed to evade static analysis. For example, when malware at run-time unpacks and executes its packed code, dynamic analysis is able to identify the (run-time) malicious behaviors in the originally packed code.


  • Dynamic analysis involves running the malware
  • One MUST guard against infection spreading
  • Malware can attempt to detect the analyis environment and shut itself down

Analysis and Detection Evasion

Signature Based Virus Detection Issues

From Wikipedia:Signature Based Virus Detection

Although the signature-based approach can effectively contain malware outbreaks, malware authors have tried to stay a step ahead of such software by writing “oligomorphic”, “polymorphic” and, more recently, “metamorphic” viruses, which encrypt parts of themselves or otherwise modify themselves as a method of disguise, so as to not match virus signatures in the dictionary.

Binary Classification

Wikipedia: Binary Classification

  • true positives TP (correct positive assignments),
  • true negatives TN (correct negative assignments),
  • false positives FP (incorrect positive assignments),
  • and false negatives FN (incorrect negative assignments).

Why use file hashes?

If a small or trivial change to a file results in a very different cryptographic hash it seems very easy to modify malware to produce a different hash. Why do we continue to use file hashes?

Static Analysis Evasion References

Static Analysis/Detection Evasion: Packers 1

From Explained: Packer, Crypter, and Protector

This usually is short for “runtime packers” which are also known as “self-extracting archives”. Software that unpacks itself in memory when the “packed file” is executed. This type of compression was invented to make files smaller. But given the current size of portable media and internet speeds, the need for smaller files is not that urgent anymore.

Static Analysis/Detection Evasion: Packers 2

From Explained: Packer, Crypter, and Protector

So when you see some packers being used nowadays, it is almost always for malicious purposes. In essence to make reverse engineering more difficult, with the added benefit of a smaller footprint on the infected machine.

Static Analysis/Detection Evasion: Crypter 1

From Explained: Packer, Crypter, and Protector

The crudest technique for crypters is usually called obfuscation. More complex methods use actual encryption. Most crypters do not only encrypt the file, but the crypter software offers the user many other options to make the hidden executable as hard to detect by security vendors as possible.

Static Analysis/Detection Evasion: Crypter 2

From Explained: Packer, Crypter, and Protector

… FUD (Fully Undetectable) which is the ultimate goal for malware authors. Being able to go undetected by any security vendor is the holy grail for malware authors.

Crypter Example

From bitcrypter


Identifying the Analysis Environments

From CyBOK Malware

Malware often uses system and network artifacts that suggest that it is running in an analysis environment rather than a real, infected system . These artifacts are primarily categorised into four classes: virtualization, environment, process introspection, and user.

Detecting Virtualization

From CyBOK Malware

In virtualization fingerprinting, evasive malware tries to detect that it is running in a virtualized environment. For example, it can use red pill testing, which entails executing specific CPU instruction sequences that cause overhead, unique timing skews, and discrepancies when compared with executions on a bare-metal (i.e., non-virtualized) system.

Detecting Environmental/Process Artifacts

From CyBOK Malware

Regarding environment artifacts, virtual machines and emulators have unique hardware and software parameters including device models, registry values, and processes.

In process introspection, malware can check for the presence of speci€c programs on operating systems, including monitoring tools provided by anti-virus companies and virtual machine vendors.

Detecting User Artifacts

From CyBOK Malware

Lastly, user artifacts include specific applications such a web browser (or lack thereof), web browsing history, recently used files, interactive user prompts, mouse and keyboard activities etc. These are signals for whether a real human uses the environment for meaningful tasks.

// reveal.js plugins