How to avoid mixing memory allocators in ClickHouse

⚠️ Mixing memory allocators is asking for trouble. Upgrading chDB to ClickHouse 25.8 exposed exactly this problem: when you embed ClickHouse in Python, you're mixing two different allocators (Python's malloc and jemalloc). If memory allocated by one gets freed by the other, it crashes. 💥 Our solution: runtime memory fingerprinting using je_mallctl("arenas.lookup") to detect which allocator owns each pointer. But arenas.lookup itself would crash on invalid pointers. Auxten Wang fixed this upstream in jemalloc by adding boundary checking, which allows it to handle arbitrary pointers safely. Patch merged, he's now an official jemalloc contributor. 🎉 Bonus: Adding thread-local checks to skip the lookup in hot paths resulted in a 61x speedup on one benchmark query. 🚀 https://lnkd.in/dePp8JRB