Aqueous: storing already compressed files in less space

The problem

Most photos, screenshots and documents are already compressed when they are saved. JPEG ends with Huffman codingHuffman codingA coding method that gives short bit strings to frequent values and long ones to rare values, so the data takes fewer bits. after its lossy steps, and PNG and ZIP use DEFLATEDEFLATEA compression method that replaces a repeated string with the position and length of an earlier copy, then applies Huffman coding. ZIP, gzip and PNG use it.. Compressed data has few repeated patterns. So these files do not shrink much when you pack them again with an archiver.

The README compares Aqueous with tar plus zstdzstdA general-purpose compression method and tool released by Meta. It compresses and decompresses quickly. and with the zpaq archiver. It does not include 7z. According to 7-Zip’s page on the 7z format, 7z compresses with general-purpose methods such as LZMA2. It adds conversion filters for executables and WAV files. The page lists no filter for JPEG or PNG, which hold compressed data inside. So 7z compresses the bytes of these files as they are, and the compressed parts do not shrink much.

How it solves it

Aqueous was posted as a Show HNShow HNA post on Hacker News, a developer community, in which someone presents what they made. Its title starts with "Show HN:". on Hacker News on October 28, 2026. Its archiver, aqr, detects a file’s format and undoes that format’s compression. It then compresses the contents again with a compression modelcompression modelThe part of a compressor that predicts the next value from earlier data, so values it predicts well take fewer bits. made for that format. According to the README, it handles about twenty formats this way, including JPEG, PNG, MP3, PDF, SQLite, ZIP and gzip.

The same data gives different compressed bytes with different compressors and settings. So the contents alone are not enough to rebuild the original file. aqr predicts the choices the original compressor made and records only the ones it got wrong. For DEFLATE, the choices are which earlier string each repeat points to and where blocks split. On extraction it replays the original compression with that record and rebuilds the file bit for bit.

Each archive holds a small rebuild program for each file, plus the WebAssemblyWebAssemblyA portable binary format for code that browsers and other runtimes can run. The host can limit what the code can access. modules those programs use. According to the README, aqr runs the modules built into its binary as native code. It runs other modules in a sandboxsandboxAn environment that runs a program while blocking its access to system resources such as files and the network..

What is different

The first difference is what gets compressed. 7z compresses a file’s bytes as they are. Aqueous undoes the compression inside the file and compresses the contents with a model for that format. The decoded image or text shows far more repetition and structure than the compressed bytes. But this helps only when aqr can predict the original compressor’s choices well. When many guesses miss, the record grows and the gain disappears. According to the aqr-flate notes in the repository, aqr predicts DEFLATE choices by emulating known compressors such as zlib. When it expects the record of wrong guesses to be large, it stores the stream as it is.

The second difference is what the extractor must know. To open a 7z archive, a program must already implement methods such as LZMA2. Aqueous puts the modules it needs into the archive. So, according to the README, the extractor does not need to know the formats inside. The tradeoff is that the modules add size to every archive. An archive with few files may barely shrink, or end up larger than the input.

In the README, the maker reports a run from October 27, 2026. It compressed the 10 GB benchmark (79,431 files) published by Matt Mahoney, who wrote zpaq, on an Apple M4. aqr produced a 2.63 GB archive, and tar with zstd -3 produced a 4.68 GB archive. zpaq 7.15 produced a 3.70 GB archive. aqr took 136 seconds and zstd took 33 seconds. aqr’s time includes rebuilding every file to check it.

How it was built

• The 539 commits date from October 5 to October 29, 2026. Of these, 31 have a Co-authored-by trailer for Claude.
• In the README, the maker says the project makes heavy use of LLM coding agents. The maker provided the ideas and goals, and the agents wrote the code and ran the experiments.
• The source is 99,033 lines of Rust, and the tests are 11,298 lines. The per-format recompressors are split into 24 Rust packages.

Running it

I cloned the repository and built aqr. I then compressed and extracted 60 files (6.56 MB). They included 22 JPEG screenshots from this site and 14 PNG files. Ten of the PNG files were those screenshots saved again as PNG, and four came from the node_modules and /usr/share folders on the machine I used. The rest were 20 gzip files from Debian docs, 2 SQLite files, 1 ZIP and 1 PDF.

1. Build. The README’s command, cargo build -r -p aqr, failed because it could not read flate-zoo/Cargo.toml. After git submodule update --init, the build took 7 minutes 40 seconds on a 4-core machine. The binary was 166 MB.
2. Compression. aqr compressed the 60 files from 6.56 MB to 4.63 MB (70.6% of the original size) in 0.98 seconds. In a tar archive, the same files came to 85.5% with zstd -3 and 81.8% with xz -9. xz uses LZMA2, the default method of 7z.
3. By format. With a separate archive per format, PNG came to 70.8% (zstd -3: 97.9%), and JPEG came to 70.4% (zstd -3: 77.3%). SQLite came to 13.1%, larger than zstd -3 (7.5%).
4. Extraction. All 60 files extracted with aqr -d had the same SHA-256SHA-256A hash function that computes a 256-bit value from data. Files with the same value are taken to have the same content. hash as the originals. They also matched with --wasm, which runs every module as WebAssembly. Extraction took 1.47 seconds with --wasm and 0.25 seconds without it.
5. gzip. At the default level, aqr stored 18 of the 20 Debian gzip files as they were. Apart from the 60 test files, I made 6 gzip files with GNU gzip and Python’s zlib. aqr stored all of them as they were, and its output did not say why. With -l high, it unpacked and recompressed the two files that held source code.
6. Small archives. For one JPEG (131.8 KB), aqr compressed the image data to 66.1% of its original size. The modules and header added 41.5 KB, so the archive was 128.6 KB. One ZIP (200.6 KB) grew to 245.2 KB.

/tmp/run/out/corpus.aqr: 60 files, 6.56 MB -> 4.63 MB (70.6%) in 0.98 s, 6.7 MB/s
/tmp/run/one/2026-10-11-2.jpg: jpeg, jpeg-recons@2, 131.8 KB -> 87.2 KB (66.1%)
archive overhead (header, modules, metadata): 41.5 KB
/tmp/run/out/one.aqr: 1 file, 131.8 KB -> 128.6 KB (97.6%) in 0.03 s, 3.8 MB/s

Not checked

• MP3, FLAC and PDF recompression. aqr did not recognize the PDF I tested and stored it as it was. Check the ratios on more PDF and music files.
• The 10 GB benchmark. Reproducing the maker’s result requires a 10 GB download. Check the compression time and the memory use during extraction.
• The sandbox. According to the README, modules not built into aqr run in a separate process with limited permissions. Test it with an archive that holds such a module.

Verdict

It works. Every file it compressed came back bit for bit. But the rebuild modules go into every archive, so an archive of a few files was barely smaller than the original files, or even larger. It is worth trying if you back up folders full of photos and screenshots that are already compressed. You also need to accept slower compression than zstd in exchange for less space.