Malware Analysis

Advanced Static Analysis

Deep dive into advanced static analysis techniques including IDA Pro disassembly, Ghidra decompilation, and identifying malicious code patterns without executing the binary.

Contents

Advanced Static Analysis

Goal: Understand what a binary does without ever running it — using disassemblers, decompilers, and manual code analysis.


1. Tool Selection — Which Disassembler to Use

ToolTypeBest ForCost
GhidraDisassembler + DecompilerFull RE, malware, CTFFree (NSA)
IDA ProDisassembler + DecompilerIndustry standard, best x86 analysis$$$$
IDA FreeDisassembler onlyQuick disassembly, no decompilerFree
Cutter / RizinDisassembler + Decompiler (r2-based)Lightweight, Linux-friendly, CTFFree
Binary NinjaDisassembler + DecompilerClean UI, good HLIL$$
x64dbgDebugger (dynamic)Runtime analysis on WindowsFree
radare2CLI DisassemblerScripting, automation, CTFFree

2. Loading a Binary into Cutter / Ghidra

Cutter Workflow

  1. Open Cutter and load FileName.exe
  2. Select analysis options (keep defaults for most malware)
  3. Wait for auto-analysis to complete
  4. Locate the main function — this is the entry point of the program
  5. Switch between Disassembly, Graph, and Decompiler views

Ghidra Workflow

  1. Create a new project → Import File → select the binary
  2. Double-click the file → opens CodeBrowser
  3. Say Yes to auto-analysis when prompted
  4. Navigate to entry → follow call to __libc_start_main → first argument = main
  5. Use the Decompile window (right panel) to read pseudo-C code

3. Identifying Key Functions

Finding main()

  • PE (Windows): Look for entry__mainCRTStartupmain or WinMain
  • ELF (Linux): entry__libc_start_main(main, argc, argv, ...) — first arg is main
  • Stripped binaries: No symbol names — look for the function called by the entry stub

Recognizing Important Functions

What You SeeWhat It Probably Does
CreateFile / WriteFileFile operations (dropper, ransomware)
VirtualAlloc + memcpyShellcode unpacking/injection
socket / connect / send / recvNetwork C2 communication
RegSetValueExRegistry persistence
CreateProcess / ShellExecuteSpawning child processes
CryptEncrypt / CryptDecryptCrypto operations (ransomware)
XOR loops in a tight functionCustom encryption / obfuscation
LoadLibrary + GetProcAddressDynamic API resolution (evasion)
IsDebuggerPresentAnti-debugging check

4. Control Flow Analysis

Reading the Graph View

The graph view (CFG — Control Flow Graph) shows basic blocks connected by arrows:

  • Green arrow = conditional branch TAKEN (true)
  • Red arrow = conditional branch NOT TAKEN (false)
  • Blue arrow = unconditional jump

Common Patterns in Graph View

If/Else:

   [cmp eax, 0]
   /          \
 [jne]      [je]
  |           |
[block A]  [block B]
  \          /
   [continue]

While Loop:

[condition check] <----+
       |               |
   [loop body] --------+
       |
   [exit loop]

Switch/Case:

[cmp/ja → jump table]
  / | | | \
[c0][c1][c2][c3][default]

5. String Analysis in Disassemblers

Where to Find Strings

  • Ghidra: Window → Defined Strings (or Search → For Strings)
  • Cutter: Strings panel (izz/iz)
  • IDA: View → Open Subviews → Strings (Shift+F12)

What Strings Reveal

String PatternIntelligence
http://, https://C2 server URLs
cmd.exe, /bin/shShell command execution
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunPersistence registry key
password, credential, loginCredential stealer
encrypt, decrypt, AES, RSACrypto/ransomware
Base64 blobs (= padding)Encoded payloads or configs
mutex_, Global\Mutex for singleton check
Error messages with developer pathsDeveloper machine info leak
.onion, torTor-based C2

6. Import & Export Analysis

Imports Table

The Import Address Table (IAT) tells you which DLL functions the binary uses statically:

In Ghidra: Window → Symbol Table → filter by “EXTERNAL”
In Cutter: Imports panel
In IDA: View → Open Subviews → Imports

Suspicious Import Combinations

Import CombinationLikely Behavior
VirtualAllocEx + WriteProcessMemory + CreateRemoteThreadProcess injection
OpenProcess + ReadProcessMemoryMemory reading (credential theft)
InternetOpenA + InternetConnectA + HttpSendRequestAHTTP C2
WSAStartup + socket + connectRaw socket C2
CryptAcquireContext + CryptEncryptEncryption (ransomware)
Only LoadLibrary + GetProcAddressDynamic resolution — hiding real API usage
NtQuerySystemInformationAnti-analysis / system enumeration

Exports Table

  • Legitimate DLLs export functions for other programs to use
  • Malicious DLLs often export functions like DllMain, ServiceMain, or names that mimic legit software
  • Look for exported functions with suspicious names or ordinals-only exports

7. Cross-References (XREF) Analysis

Cross-references show who calls a function and what a function calls:

  • Ghidra: Right-click → References → Show References To
  • IDA: Press x on any function/address
  • Cutter: Right-click → Show X-Refs

Why XREFs Matter

  • Find all callers of VirtualAlloc → trace shellcode allocation
  • Find all callers of send() → trace C2 exfiltration
  • Find what calls a decryption function → trace payload unpacking
  • Dead code (functions with 0 XREFs) → possibly unused or dynamically resolved

8. Identifying Packed / Obfuscated Binaries

Signs of Packing

IndicatorWhat It Means
Very few imports (only LoadLibrary, GetProcAddress)Packed — real imports resolved at runtime
Section names like .upx0, .aspack, .themidaKnown packer sections
High entropy (>7.0) in sectionsCompressed/encrypted data
Virtual size >> raw size in PE sectionsUnpacking will expand data in memory
Entry point in non-standard section (not .text)Stub executes before real code
Very few stringsStrings encrypted or compressed

Common Packers

PackerDetectionUnpacking
UPXupx -t file.exe, section names .UPX0/.UPX1upx -d file.exe
ASPackPEiD, Detect It EasyManual or ASPackDie
ThemidaPEiD, section .themidaVery hard, use dynamic analysis
VMProtect.vmp sections, virtualized codeExtremely hard
Custom packersNo signature match, weird entropyDump from memory at OEP

Entropy Analysis

  • Normal code (.text section): entropy ~5.0–6.5
  • Packed/encrypted: entropy ~7.0–8.0
  • Use Detect It Easy (DiE) or pestudio for entropy visualization

9. Advanced Techniques

Identifying Encryption Algorithms

  • Look for magic constants:
    • 0x67452301, 0xEFCDAB89 → MD5
    • 0x6A09E667 → SHA-256
    • Rijndael S-box (0x63, 0x7C, 0x77, 0x7B...) → AES
    • 0x9E3779B9 → TEA/XTEA
  • Ghidra: Use FindCrypt plugin to auto-detect crypto constants
  • IDA: Use FindCrypt2 or Signsrch

Reconstructing Structures

  1. Identify clusters of data accesses at fixed offsets from a base pointer
  2. Create a struct in Ghidra: Data Type Manager → right-click → New Structure
  3. Apply the struct to the pointer → code becomes readable

Patching Binaries

  • Ghidra: Right-click instruction → Patch Instruction → Export patched binary
  • Cutter: Edit → Patch → write bytes
  • Common patches: NOP out anti-debug checks, change JE to JMP to skip password checks

10. Static Analysis Workflow Checklist

□ 1. File identification (file, DIE, PEiD)
□ 2. Check if packed → unpack if needed
□ 3. Hash the sample (MD5/SHA256) → check VirusTotal
□ 4. Extract strings (strings/FLOSS)
□ 5. Analyze imports/exports → categorize behavior
□ 6. Load into disassembler (Ghidra/IDA/Cutter)
□ 7. Find entry point → locate main()
□ 8. Read decompiled code for high-level understanding
□ 9. Follow XREFs for critical functions
□ 10. Identify crypto, C2, persistence, injection
□ 11. Extract IOCs (IPs, domains, file paths, registry keys)
□ 12. Document findings → write report

Pro Tips

Rename everything — As you identify functions, rename them (decrypt_payload, connect_c2, inject_shellcode). Future you will thank present you.

Comment liberally — Add comments in the disassembler for every discovery. Analysis is iterative.

Cross-reference with dynamic analysis — Static tells you what CAN happen. Dynamic tells you what DOES happen. Use both.

Use YARA rules — Write YARA signatures from your static findings to detect variants of the same malware family.