(Georgia Institute of Technology, 2013)
Amur, Hrishikesh; Andersen, David G.; Kaminsky, Michael; Schwan, Karsten
The WriteBuffer (WB) Tree is a new write-optimized data
structure that can be used to implement per-node storage in
unordered key-value stores. TheWB Tree provides faster writes
than the Log-Structured Merge (LSM) Tree that is used in many
current high-performance key-value stores. It achieves this by
replacing compactions in LSM Trees, which are I/O-intensive,
with light-weight spills and splits, along with other techniques.
By providing nearly 30 higher write performance compared
to current high-performance key-value stores, while providing
comparable read performance (1-2 I/Os per read using 1-2B per
key of memory), the WB Tree addresses the needs of a class of
increasingly popular write-intensive workloads.
(Georgia Institute of Technology, 2012)
Amur, Hrishikesh; Richter, Wolfgang; Andersen, David G.; Kaminsky, Michael; Schwan, Karsten; Balachandran, Athula; Zawadzki, Erik
Memory is rapidly becoming a precious resource in many
data processing environments. This paper introduces
a new data structure called a Compressed Buffer Tree
(CBT). Using a combination of buffering, compression,
and lazy aggregation, CBTs can improve the memory
efficiency of the GroupBy-Aggregate abstraction which
forms the basis of many data processing models like
MapReduce and databases. We evaluate CBTs in the
context of MapReduce aggregation, and show that CBTs
can provide significant advantages over existing hash-based
aggregation techniques: up to 2x less memory
and 1.5x the throughput, at the cost of 2.5x CPU.