The Memory Leak That Isn’t a Leak: malloc_trim on a Small Box

A notebook that frees every model correctly still watched RAM climb until the container died. Python was not leaking. glibc was holding the arenas.
Author

Benedict Thekkel

Published

July 9, 2026

The setup: a container with 20 GB of system RAM and an RTX 3060 with 12 GB of VRAM. A notebook that loads several diffusion pipelines in sequence, frees each one before loading the next, and still dies around the third or fourth.

Every del was correct. gc.collect() ran. torch.cuda.empty_cache() ran. VRAM was flat. System RAM went up anyway, section after section, until the OOM killer took the container rather than the kernel.

It looks exactly like a Python leak. It is not one.

Two things combine

1. enable_model_cpu_offload() parks weights in system RAM, not VRAM.

It is the right call for keeping VRAM flat: only the executing submodule sits on the GPU, everything else waits in host memory. But that means every pipeline you load this way holds its full weight set in the 20 GB of system RAM until it is released. A diffusers pipeline is 2-7 GB there.

So the memory you carefully kept off the GPU is now competing for the budget you did not think you were spending.

2. glibc does not return freed memory to the operating system.

After del pipe; gc.collect() the Python objects genuinely are gone. But glibc keeps the freed blocks in its own arenas so it can reuse them, and RSS stays high. Because successive models are different sizes, the reuse is imperfect, so RSS does not plateau. It ratchets.

The container’s memory limit is enforced against RSS. So does the OOM killer. Python’s opinion that the objects are gone is not relevant to either of them.

The fix is one line

import ctypes, ctypes.util

ctypes.CDLL(ctypes.util.find_library("c") or "libc.so.6").malloc_trim(0)

malloc_trim(0) hands the arenas back to the OS. Measured on this box, loading Stable Diffusion 1.5 in fp16 with CPU offload three times, and reading RSS after each free:

Without the trim, RSS climbs: 2.00 GB, then 2.48 GB, then still climbing. With it, RSS returns to roughly the same place every time: 1.11, 1.23, 1.25 GB. Flat.

The one-liner belongs inside whatever free_memory() helper you already have, next to gc.collect() and empty_cache(). Wrap it in a try so non-glibc platforms (musl, macOS) skip it quietly rather than crashing.

The framing that makes it stick

torch.cuda.empty_cache() is to VRAM what malloc_trim(0) is to system RAM. On a small box you need both, and almost everyone only knows about the first.

The asymmetry is purely cultural. Everyone doing GPU work learns empty_cache() early, because VRAM is the budget people talk about. System RAM feels like something the OS handles for you, right up until you are in a container with a hard limit and a process whose RSS only goes one direction.

flowchart LR
  A[del pipe] --> B[gc.collect]
  B --> C[Python objects gone]
  C --> D{Where did the RAM go?}
  D -->|VRAM| E[empty_cache -> returned]
  D -->|System RAM| F[glibc arena -> still counted as RSS]
  F --> G[malloc_trim 0 -> returned]

A note on measuring it honestly

psutil reads RSS, which means it only shows the RAM as reclaimed after the malloc_trim. That is sometimes read as the tool lying before the trim.

It is the opposite. RSS is what the kernel and the OOM killer see. If a monitoring helper reported the memory as free before the arenas were returned, it would be telling you a comfortable number while the thing that actually kills your container disagreed.

def memory_report():
    vm = psutil.virtual_memory()
    print(f"RAM  {(vm.total - vm.available) / 1e9:5.2f} / {vm.total / 1e9:5.2f} GB")
    if torch.cuda.is_available():
        print(f"VRAM {torch.cuda.memory_allocated() / 1e9:5.2f} / "
              f"{torch.cuda.get_device_properties(0).total_memory / 1e9:5.2f} GB")

Call it after each heavy stage. A well-behaved notebook returns close to baseline. If it does not, something is still referenced, and it is worth finding rather than leaving the notebook to die on the next run. Jupyter’s own _, __ and Out[...] are a common culprit: a cell whose last expression is a big tensor pins it in RAM for the life of the kernel.

Takeaway

If a long notebook OOMs the container rather than the kernel, and every del looks right, stop hunting for the Python reference. Check whether the RAM was ever handed back to the OS at all.

One line, inside free_memory(), next to the two calls you already make. The rest of the memory rules for this box, along with the download-size trap that catches you before you even get to RAM, are in what actually fits on a 12 GB GPU.

Back to top