The Problem

A Go microservice, designed to load, process, and clear a million records, experienced unexpected memory consumption even after clearing the map. This led to a significant memory leak, resulting in 290MB of excess memory usage.

The Code

The code snippet shows the use of a map with integer keys and byte arrays as values, where data was filled and then deleted after processing. Garbage collection was explicitly called.

treasureChest := make(map[int][128]byte)
Data insertion and processing
delete(treasureChest, i) loop
runtime.GC() call

The Investigation

Despite deleting the map entries and forcing garbage collection, the memory usage didn't decrease. This sparked an investigation into Go's memory management and map implementation.

A Memory Leak Story of Map (Go vs. Rust)

I still remembered it was 3 AM in July 2019, when our production servers caught fire. Not literally, of course, but in the world of software engineering, unexplained memory spikes are just as alarming. I was staring at our monitoring dashboard, bleary-eyed, wondering why our simple microservice was consuming 290MB more memory than it should.

That night began my strange journey into the underbelly of programming language design decisions. A journey that would make me question everything I thought I knew about maps, memory, and the trade-offs we accept without realizing.

The Crime Scene

Our service was simple: load a million records into memory, process them, clear the data, and repeat. Nothing fancy. The code was elegant Go, barely 50 lines. I was proud of it.

// code was sth like this:treasureChest := make(map[int][128]byte)// Fill with a million records...for i := 0; i < 1_000_000; i++ {    treasureChest[i] = [128]byte{}}// Process data...// Then clear everythingfor i := 0; i < 1_000_000; i++ {    delete(treasureChest, i)}runtime.GC() // Explicitly ask Go to clean up

Imagine my surprise when our memory usage refused to drop below 290MB after clearing the map. I triple-checked my code. I forced garbage collection. I questioned my sanity.

Was this article displayed correctly? Not happy with what you see?

See Archived Versions

We located an Open Access version of this article, legally shared by the author or publisher. Open It