A Novel Index-Organized Data Layout for Hybrid DRAM-PM Main Memory Database Systems

Large-scale data-intensive applications need massive real-time data processing. Recent hybrid DRAM-PM main memory database systems provide an effective approach by persisting data to persistent memory (PM) in an append-based manner for efficient storage while maintaining the primary database copy in DRAM for high throughput rates. However, they fail to achieve high performance under a hybrid workload because they are unaware of the impact of pointer chasing. In this work, we investigate the impact of chasing pointers on modern main memory database systems to eliminate this bottleneck. We propose Index-Organized data layout that supports efficient reads and updates. We combine two techniques, i.e., cacheline-aligned node layout and cache prefetching, to accelerate pointer chasing, reducing memory access latency. We present four optimizations, i.e., pending versions, fine-grained memory management, Index-SSN, and cacheline-aligned writes, for supporting efficient transaction processing and fast logging. We implement our proposed data layout based on an open-sourced main memory database system. We conduct extensive evaluations on a 20-core machine equipped with Intel Optane DC Persistent Memory Modules. Experimental results demonstrate that Index-Organized obtains up to 3x speedup than the conventional data layouts, i.e., row-store, column-store, and row+column.