Bot Defense Serving Zip Bombs to Crash Crawlers

Web traffic is largely bots; zip bombs, small compressed files expanding massively, can defend against malicious bots by overwhelming their memory and causing crashes.

The Web is Full of Bots

• Most web traffic is from bots, used for discovering new content (RSS, search engines, AI for LLMs).
• Malicious bots from spammers, content scrapers, and hackers exist.
• Example: A bot exploited a WordPress vulnerability, injected a malicious script, and turned the server into a botnet.

Zip Bombs for Defense

• Zip bomb: a small compressed file that expands into a very large file, overwhelming a machine.

How Compression Works

• Gzip compression was developed to compress data for faster transmission.
• Example: A 50 KB HTML file can compress to 10 KB, saving 40 KB in transmission time.
• Browsers signal support for compression via headers: Accept-Encoding: gzip, deflate.
• Web crawlers and bots also support compression to maximize bandwidth.

Serving Zip Bombs

• Detect malicious bots attempting attacks or probing for responses.
• Serve them a gzip response (1MB to 10MB file).
• The bot decompresses the file, which expands until it runs out of memory and crashes.
• A 1MB compressed file can decompress into 1GB, enough to break many bots. A 10MB file can become 10GB.

Creating a Zip Bomb (Warning!)

• Use the following command (at your own risk!):

  dd if=/dev/zero bs=1G count=10 | gzip -c > 10GB.gz
  

• dd: copies or converts data.
• if: Input file, /dev/zero (infinite stream of zero bytes).
• bs: Block size, 1 gigabyte (1G).
• count=10: Process 10 blocks, each 1 GB in size (generates 10 GB of zeroed data).
• gzip: Compresses the output into the file 10GB.gz. The resulting file is around 10MB.

Implementation Example

• Middleware to check if a request is malicious (blacklisted IPs, spamming patterns).
• Code example:

  if (ipIsBlackListed() || isMalicious()) {
      header("Content-Encoding: deflate, gzip");
      header("Content-Length: "+ filesize(ZIP_BOMB_FILE_10G)); // 10 MB
      readfile(ZIP_BOMB_FILE_10G);
      exit;
  }
  

• Serving the 10MB file on some occasions. Decrease it to 1MB for viral articles.

Limitations

• Zip bombs are not foolproof and can be detected.
• Effective for unsophisticated bots blindly crawling the web.

See it in Action

• View server logs: [this replay of my server logs](https://idiallo.com/blog/surviving-the-hug-of-death)