Today marks the end of my stay at Facebook. Over the last eight months, on and off, I have re-implemented, tested, evaluated, reproduced the results, and merged Sinbad (SIGCOMM’13) with Facebook’s internal HDFS trunk. Several of the future work promised in the paper have been executed in the process, with varying results. It has been quite a fruitful stint for me; hopefully, it’ll be useful for Facebook in coming days as it rolls out to more production clusters. After all, who doesn’t like faster writes!

I’m really glad to have had this opportunity, and there are many people to thank. It all started with giving a talk last July at the end of my Facebook Fellowship, which resulted in Hairong asking me to implement Sinbad at Facebook. Every single member of the HDFS team have been extremely helpful since I started last October. I’d specially like to mention Hairong, Pritam, Dikang, Weiyan, Justin, Peter, Tom, and Eddie for spending many hours in getting me up to the speed, code reviewing, bug fixing, helping in deploying, troubleshooting, and generally baring with me. I’d also like to thank Lisa from the Networking team to help me add APIs to their infrastructure so that Sinbad can collect rack-level information.

The whole experience was different than my previous internships at research labs for several reasons. First, because it wasn’t an internship, I was on my own schedule and could get my research done while doing this on the side. Second, it was nice to sit with the actual team running some of the largest HDFS clusters on earth. I did learn a lot and found some new problems to explore in the future. Finally, of course, lots of free delicious food!

I’m glad I listened to Ion when I was asked by Hairong. Now, let’s hope Apache Hadoop/YARN picks it up as well.