Armando Fox, Steven D. Gribble, Yatin Chawathe, Eric A. Brewer, Paul Gauthier, “Cluster-Based Scalable Network Services,” Sixteenth ACM Symposium on Operating Systems Principles (SOSP), (October, 1997). [PDF]

Summary

This paper identifies scalability (elasticity), availability, and cost effectiveness as the three fundamental requirements of cluster computing. It proposes a layered architecture that separates high-level policies from actual underlying mechanisms using a service programming model consisting of composable workers that can perform Transformation, Aggregation, Caching, and Customization (TACC) to build scalable (online) network services. In addition, the authors propose BASE (Basically Available, Soft State, and Eventual Consistency), a weaker-than-traditional-ACID semantics that trades consistency for availability and relies on soft states for robustness. Under the light of these concepts, the authors discuss the designs of two different online services (TranSend and HotBot) in terms of load balancing, fault management, caching, and evaluate them. Both the services have an ACID component, but primarily manipulate BASE data.

Comments

This work is dated pre-CAP theorem, but it points to some of the observations that I think eventually led to the CAP theorem (e.g., the trade-off between availability and consistency was later extended to include partition tolerance). The focus on cost-effectiveness was a very important observation as well, as it led to the later revolution in datacenters made of commodity components. While user-facing services and relevant content types have changed over time, I think the authors got the basics right and predicted well how modern cluster computing might look like.