Update: Coflow camera-ready is available online! Tell us what you think!

Our position paper to address the lack of a networking abstraction for cluster applications, “Coflow: A Networking Abstraction for Cluster Applications,” has been accepted at the latest workshop  on hot topics in networking. We make the observation that thinking in terms of flows is good, but we can do better if we think in terms of collections of flows using application semantics while optimizing datacenter-scale applications.

Cluster computing applications — frameworks like MapReduce and user-facing applications like search platforms — have application-level requirements and higher-level abstractions to express them. However, there exists no networking abstraction that can take advantage of the rich semantics readily available from these data parallel applications.

We propose coflow, a networking abstraction to express the communication requirements of prevalent data parallel programming paradigms. Coflows make it easier for the applications to convey their communication semantics to the network, which in turn enables the network to better optimize common communication patterns.

This observation is a culmination of some of the work I’ve been fortunate to be part of in last couple of years (specially Orchestra@SIGCOMM’11 and recently Ahimsa@HotCloud’12 and FairCloud@SIGCOMM’12) as well as work done by others in the area (e.g., D3@SIGCOMM’11). Looking forward to constructive discussion in Redmond next month!

FTR, HotNets PC this year accepted 23 papers out of 120 submissions.