HTB Writeup

Malware Analysis Report: Simple Encryptor

HackTheBox Simple Encryptor challenge writeup, reversing a PRNG-based file encryption scheme by recovering the seed and decrypting the flag.

Contents

Malware Analysis Report: Simple Encryptor

1. Executive Summary

This report documents the reverse engineering of a binary named simple-encryptor. The binary performs a file encryption routine using a pseudo-random number generator (PRNG). The analysis reveals a vulnerability in the seed generation, allowing for the decryption of the target file (flag.enc) through a brute-force attack on the seed value.

2. Initial Assessment

The provided archive contains two files:

  • encrypt: An ELF executable.
  • flag.enc: An encrypted data file.

Initial file identification: File Identification

3. Static Analysis

Decompilation of the encrypt binary using Ghidra reveals the core logic within the main function.

Decompilation Overview: Ghidra Overview

Main Function Logic: Main Function

The analysis of the main function identifies the following operations:

  1. Seed Generation: srand() is called to initialize the PRNG.
  2. File Operations: The program opens the flag file and determines its size.
  3. Memory Allocation: Memory is allocated to store the file contents.
  4. Encryption: The data is encrypted using values generated by rand().

4. Cryptographic Weakness

The encryption relies on the C standard library’s rand() function. The security of this scheme depends entirely on the unpredictability of the seed passed to srand().

According to the C documentation:

srand() seeds the pseudo-random number generator used by rand(). If rand() is used before any calls to srand(), rand() behaves as if it was seeded with srand(1). Each time rand() is seeded with srand(), it must produce the same sequence of values.

This determinism implies that if the seed value can be recovered or guessed, the entire sequence of random numbers can be reproduced, allowing for decryption.

Encryption Logic: Encryption Logic

5. Decryption Strategy

To recover the original data, the encryption process must be reversed. Since the operation is likely a simple XOR or arithmetic operation with the random stream, the decryption logic mirrors the encryption.

Decryption Logic: Decryption Logic

Exploit Development

The analysis indicates the seed is a 4-byte integer. A brute-force approach was selected to identify the correct seed. The exploit iterates through possible seed values (specifically checking a range of 4-digit numbers as identified in the binary analysis), generates the corresponding random sequence, and attempts to decrypt the content.

Exploit Code: Exploit Code

6. Results

The exploit successfully recovered the plaintext flag by identifying the correct seed.

Recovered Flag: Recovered Flag


Analysis generated on December 7, 2025.