Update: Camera-ready is available online! Do let us know what you think in the comments section.

Our exploratory paper on the complexity of a transfer, “Redefining Network Fairness to Support Data Parallelism,” has been accepted for publication at this year’s HotCloud workshop!

In Orchestra, we defined the notion of transfers in the context of cluster computing, and in FairCloud, we argued for fairness across multiple transfers. However, we have so far been considering transfers independently of the computations they enable. Gautam observed that not all transfers are created equal: when we scale-up or -down the input to computations, input to transfers do not always scale linearly (e.g., partitioned transfers like shuffles in a MapReduce program scales linearly, whereas broadcast has a super-linear scaling factor). As a result, network fairness, when defined in terms of bandwidth, does not always match the simple goal of data parallelism: “given n times more resources, a data parallel application can expect to complete n times faster.” Ahimsa explores the notion of network fairness that can match this goal.

This year, HotCloud accepted 24 out of 75 submissions, six of which have at least one Berkeley author :)