How Data Structures Affect Programming Performance

Pop quiz: you're building a text editor and need to pick a data structure to represent the text. What do you choose? If you said "string", keep reading. String is the obvious choice for representing a character array, but the way it's stored (a contiguous block of memory) is terrible for mutable performance if the length is long. Just look at its big O characteristics: • concatenation: O(n + m) • insertion: O(n) • deletion: O(n) • substring: O(m) (n = original string length, m = new string length) A text editor that models file content as strings would be SUPER slow for even moderately sized files. This is what ropes are for. Instead of storing the entire string as a block of memory, a rope represents the string as a balanced binary tree where leaves contain short substrings and parent nodes contain the summed lengths of the *left* subtree. ⚡ Balanced binary trees are MUCH faster for mutability: • concatenation: O(log n + log m) • insertion: O(log n) • deletion: O(log n) • substring: O(log n + log m) By storing left-subtree substring lengths, ropes can efficiently seek around the tree and mostly get log n performance. And that's why your editor responds quickly when you modify the middle of a large file.

Sometimes your data problem is really a Data type problem more so than a compute problem... Back at Motley, we wanted to better predict which product would best fit our users based on user preferences so we started with a clustering algo to get our customers into preliminary groups then KNN to map newer users into groups based on similar behaviors of past clustered users the second they joined Motley's website. This would mean, essentially, "no new users" as everyone would be quickly sorted and we could get them into a member experience funnel early on. The problem with this approach was that KNN is heavy computationally and took a very long time to run each morning in batch. However, taking the same features, turning them into user matrices and taking our products and turning those into item matrices gave us the ability to recommend users to products using a Hybrid Recommendation System that was not as heavy to run as the KNN model. The answer to why that is lies in an understanding of Big O notation, computational complexity of different algos based on dataset size, behavior of data structures, and the understanding that KNN typically requires for loops and for loops run slower than multiplication in Python. Having an understanding of Data Structures can take a Data Scientist far. You don't have to be perfect at it, but knowing why you want to store your Data or train your model using a certain datatype (say an int over a floating point to increase training speed) comes in handy in the long run. This is particularly important when dealing with training for LLMs.

𝐁-𝐭𝐫𝐞𝐞𝐬, 𝐖𝐫𝐢𝐭𝐞 𝐏𝐚𝐭𝐡𝐬, 𝐚𝐧𝐝 𝐌𝐞𝐦𝐨𝐫𝐲-𝐃𝐢𝐬𝐤 𝐈𝐧𝐭𝐞𝐫𝐚𝐜𝐭𝐢𝐨𝐧𝐬 𝐢𝐧 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞 𝐒𝐲𝐬𝐭𝐞𝐦𝐬 In data engineering, we often take the technical details of our infrastructure for granted. I see this materialize in questions like, 'why is Change Data Capture so complicated?' or 'why can't we just do analytics in our database?'. I wanted to break down how databases are designed and optimized for transactional workloads. 𝐁-𝐭𝐫𝐞𝐞𝐬: 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐃𝐚𝐭𝐚 𝐒𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞𝐬 𝐅𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬 Most databases use some sort of B-tree for storage and retrieval. B-trees are self-balancing tree data structures crucial for database systems: Multi-way nodes reduce tree height Balanced structure ensures O(log n) complexity for search, insert, and delete operations. Disk-friendly design minimizes I/O, optimizing performance for large datasets and row-level scans. Common use cases include looking up a user by a key to load their account information in an application. Where as columnar engines (think data warehouses) are optimized for analytical queries. 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞 𝐖𝐫𝐢𝐭𝐞 𝐏𝐚𝐭𝐡 1️⃣ Transaction begins 2️⃣ Changes written to Write-Ahead Log (WAL) 3️⃣ WAL flushed to disk 4️⃣ Transaction committed 5️⃣ Changes applied to in-memory B-tree 6️⃣ Modified B-tree pages periodically written to disk This process ensures durability, atomicity, and efficient recovery in case of system failures. Processes like log-based Change Data Capture tail the WAL to replicate changes, but the log itself is tightly coupled to the database's OS-level operations 𝐁-𝐭𝐫𝐞𝐞𝐬: 𝐁𝐫𝐢𝐝𝐠𝐢𝐧𝐠 𝐌𝐞𝐦𝐨𝐫𝐲 𝐚𝐧𝐝 𝐃𝐢𝐬𝐤 B-trees operate in a dual-mode, existing both in-memory and on-disk: In-Memory: 📖 Hot data and frequently accessed nodes kept in RAM ✍ Allows for fast read and write operations 🐇 Supports quick traversals and modifications 🔹 Reduces disk I/O for common operations On-Disk: 📚 Persistent storage of the entire tree structure 🔖 Organized in pages or blocks for efficient disk access ☠ Enables durability and recovery after system failures 🛳 Supports databases larger than available RAM 𝐈𝐧𝐭𝐞𝐫𝐚𝐜𝐭𝐢𝐨𝐧: ✅ Pages are loaded from disk to memory as needed ✅ Modified in-memory pages are periodically flushed to disk ✅ Buffer pool manages which pages stay in memory ✅ Write-Ahead Logging (WAL) ensures consistency between states This architecture combines B-trees' efficient querying with WAL's durability. Next I'll talk about copy-on-write data structures that are often used in analytics and the tradeoffs they make. 𝐓𝐫𝐚𝐝𝐞𝐨𝐟𝐟𝐬: ❌ Updates go back to a point on disk and re-write it ❌ Analytical queries that scan column values require more hops Credit to my alma mater University of San Francisco Computer Science and David Galles for continuing to maintain this awesome algorithm visualization. #dataengineering #databases #softwareengineering

Cache-Friendly Data Structures in C++ Last week, we conducted a poll, and the winning topic was Cache-Friendly Data Structures. While often overlooked, understanding how data interacts with CPU caches can help C++ developers write code that runs significantly faster — especially when performance is critical. Many developers rely on data structures without ever considering how they map to the CPU cache. But do you know what that means? Cache misses, memory stalls, and serious performance penalties! But… what does cache-friendly really mean? A cache-friendly data structure is one that organizes its data in memory to maximize spatial and temporal locality. In simple terms, it ensures that related data is stored close together in memory, allowing the CPU to load and process data more efficiently using cache lines. Access patterns matter. Sequential access takes full advantage of modern CPU caches, while random access often leads to cache misses — forcing the CPU to fetch data from slower main memory. Each cache miss can cost hundreds of CPU cycles. This becomes critical when dealing with large datasets, game engines, high-performance finance, scientific computing, or real-time systems. Even the choice between std::vector and pointer-based linked structures can make or break your performance. Below, we show a simple example that demonstrates two key points: 1. How sequential access leverages CPU caches for maximum performance. 2. How random access suffers from cache misses, drastically impacting execution time. Have you ever reviewed your data structures to improve cache efficiency? Comment below, we’d love to hear your thoughts! #CppPerformance #CacheFriendly #MemoryOptimization #LowLevelCpp #DataStructures #Cpp23 #CppCommunity #EfficientCoding #TechTips