Jie (Jimmy) has recently become my third PhD student to graduate with a dissertation titled "Toward Practical Application-Aware Big Data Systems." Over the course of his PhD, Jimmy ended up contributing to most of the major projects of our group over the years, and we will miss him going forward. He's joining … Continue Reading ››
Tag Archives: Big Data Systems
Apache Spark Receives the 2022 ACM SIGMOD Systems Award
Congratulations to the whole Spark community on the prestigious award and its 1800+ contributors and innumerable users. As I look back, memories from years ago continue to remind me how fortunate I was to be able to make small contributions at the infancy of this juggernaut.
Thanks KLA for Supporting Our Research
KLA became our neighbor last year. It turns out building tool-chains for the semiconductor industry requires massive amount of data and image processing in near-realtime as well as addressing corresponding resource and cluster management challenges.
We look forward to working together on related, interesting problems down the line. … Continue Reading ››
QOOP Accepted to Appear at OSDI’2018
Distributed data-parallel queries run on large collections of resources -- within a single datacenter in case of traditional big data queries and across multiple datacenters in case of geo-distributed analytics. The total amount of available resources to a running query changes over time for a variety of reasons, including changes in the fair share of … Continue Reading ››
Two Papers Accepted at APNet’2018 and MAMA’2018 Workshops
Couple more workshop papers got accepted in last few days!
- Pas de Deux: Shape the Circuits, and Shape the Apps Too! -- APNet'18
- Fair Allocation of Heterogeneous and Interchangeable Resources -- MAMA'18
The APNet one is about coflows on optical networks, following my collaborations with Hong and Kai at HKUST (SIGCOMM'16 and SIGCOMM'17), while … Continue Reading ››
Three Papers Accepted at HotCloud’2018 and GRADES-NDA’2018 Workshops
Over the course of last few days, we heard back about the acceptance of three workshops papers:
- To Relay or Not to Relay for Inter-Cloud Transfers? -- HotCloud'18
- Monarch: Gaining Command on Geo-Distributed Graph Analytics -- HotCloud'18
- Bridging the GAP: Towards Approximate Graph Analytics -- GRADES-NDA'18
The HotCloud ones deal with networking for and graph … Continue Reading ››
TWO NSF Proposals as the Lead PI Awarded. Thanks NSF!
The first one is on rack-scale computing using RDMA-enabled networks with Barzan Mozafari at the University of Michigan, and the second is on theoretical and systems implications of long-term fairness in cluster computing with Zhenhua Liu (Stony Brook University).
Thanks NSF!
Combined with the recent awards on geo-distributed analytics from NSF and Continue Reading ››
Carbyne Accepted to Appear at OSDI’2016
Update: Camera-ready version is available here now!
With the wide adoption of distributed data-parallel applications, large-scale resource scheduling has become a constant source of innovation in recent years. There are tens of scheduling solutions that try to optimize for objectives such as user-level fairness, application-level performance, and cluster-level efficiency. However, given the well-known tradeoffs between fairness, performance, and efficiency, these solutions have traditionally focused … Continue Reading ››
EC-Cache Accepted to Appear at OSDI’2016
Update: Camera-ready version is available here now!
In-memory caching is the de facto solution to enable low latency analytics over large datasets. While caching objects, one must be careful about maximizing the number of requests that can be served from memory in the presence of popularity skew, background load imbalance, and server failures. Traditional solutions use selective replication, i.e., … Continue Reading ››
Received SIGCOMM Doctoral Dissertation Award
About a week or so ago Bruce Maggs, SIGCOMM's awards chair, kindly informed me over the phone that my dissertation on coflows has been selected for the 2015 ACM SIGCOMM Doctoral Dissertation Award. The committee for the award included Ratul Mahajan, Dina Papagiannaki, Laurent Vanbever (chair), and Minlan Yu, and the citation reads:
Chowdhury's … Continue Reading ››