Replicated Versioned Data Structures for Wide-Area Distributed Systems

In this work, we investigate the integration of replicated versioned data structures and append-only distributed storage systems. Doing so facilitates high availability and scalability while providing developer access to different versions of program data structures across program executions. Modern distributed systems such as the Internet of Things (IoT) often employ multi-tiered (cloud/edge/sensors) architectures consisting of a wide array of heterogeneous devices generating data frequently. Hence system availability is imperative to avoid data loss, while scalability is required for the efficient operation of the system not only within the same tier but across different tiers as well. Our proposed approach replicates, persists, and versions program data structures such as binary search trees and linked lists for use in distributed IoT applications. The versioning and persistence of these structures aid failure recovery and facilitate system debugging from its inception instead of making such considerations an afterthought. Moreover, our experiments suggest versioned data structures can perform better in applications performing high volumes of temporal queries versus traditional methods of persisting data (e.g., in a database). We empirically evaluate the overheads associated with versioning and storage persistence of program data structures, present experimental results for multiple end-to-end applications, and demonstrate the scalability of this approach.

Automated summary

This work investigates the integration of replicated versioned data structures and append-only distributed storage systems to enhance system availability and scalability. The approach of replicating, persisting, and versioning program data structures enables failure recovery and system debugging, while also improving the performance in applications with high volumes of temporal queries. The experiments provide empirical evaluation of the overheads associated with versioning and storage persistence, as well as demonstrate the scalability of this approach.