EECS 582: Advanced Operating Systems (W’16)

 

Announcements

  • 04/14/2016: Final project presentation in regular classroom during class hours on Monday the 18th.
  • 04/04/2016: Updated final project submission requirements. Please carefully read and follow the ‘Final Project’ section below.
  • 03/04/2016: Change of class schedule in the last week.
  • 02/17/2016: Midterm will be a take-home exam instead of a closed-book one.
  • 01/18/2016: All papers have been assigned; double check the schedule below!
  • 01/06/2016: Paper summary site is online. Create an account ASAP with your umich.edu email address to get reviewer access!
  • 12/21/2015: Welcome to EECS 582!

Administrivia

Course EECS 582, Winter 2016, 4 Units, CN: 26787
Meetings 1012 EECS, M/W: 3:00 PM – 4:30 PM (Lectures/Discussion)
1012 EECS, F: 2:30 PM – 3:30 PM (Projects/Makeups)
Instructor Mosharaf Chowdhury (ANY email regarding this course should have a title starting with “EECS582” for a timely response)
Office Hours 4820 BBB. By appointments only.
Paper Summaries Submit at http://eecs582.eecs.umich.edu/papers/

Course Description

EECS 582 will discuss advanced topics and research issues in computer and operating systems. We will take a broad view of systems research and consider common issues that emerge across individual machines to large-scale datacenters. The course schedule can broadly be divided into two parts. In the former, we will consider the traditional computer and the challenges its operating system must address. In the latter, we will consider the datacenter as a computer, and we will observe how the designs of big data and cloud computing systems (e.g., MapReduce, Spark, HDFS) face and address many common challenges. This class will focus on identifying and understanding the enduring principles and practices in computer systems design and implementation, and it will prepare students to carry out substantial independent systems research projects.

Reading List

Date Topic Readings Lead Slides/Notes
 Jan 6 Introduction How to Read a Paper (No review)

Writing Reviews for Systems Conferences (No review)

Mosharaf Slides
Jan 8 Projects Hints for Computer System Design (No review)

Worse is Better (No review)

Mosharaf Slides

Project Ideas

Jan 11 Classics The UNIX Time-Sharing System Mosharaf Slides
Jan 13 A History and Evolution of System R Mosharaf Slides
Jan 15 No Class – Find Project Partners How to Give a Bad Talk (No review)
Jan 18 No Class – MLK Day
Jan 20 Storage and File System A Case for Redundant Arrays of Inexpensive Disks (RAID)

A Fast File System for UNIX

Ofir Weisse

Ya-Yun Lo

Slides

Slides

Jan 22 Analysis and Evolution of Journaling File Systems

The Design and Implementation of a Log-Structured File System (No review)

Andrew Quinn Slides
Jan 25  Kernel Exokernel: An Operating System Architecture for Application-Level Resource Management

The Multikernel: A New OS Architecture for Scalable Multicore Systems

Seth Goldstein

Clyde Byrd III

Slides

Slides

Jan 27 IX: A Protected Dataplane Operating System for High Throughput and Low Latency

The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors

Xianghan Pei

Ning Deng

Slides

Slides

Jan 29 No Class – Work on Project Proposal
Feb 1 Virtual Memory and RPC Memory Coherence in Shared Virtual Memory Systems

Implementing Remote Procedure Calls

Kuangyuan Chen

Chi-Fan Chu

Slides

Slides

Feb 3 Concurrency SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

Cooperative Task Management Without Manual Stack Management

Yang Liu

Hanyun Tao

Slides

Slides

Feb 5 No Class –  Finalize Project Proposal
Feb 8 Scheduling Lottery Scheduling: Flexible Proportional-Share Resource Management

Stride Scheduling: Deterministic Proportional-Share Resource Management

Analysis and Simulation of a Fair Queueing Algorithm (No review)

Nathan Immerman

Dong-Hyeon Park

Slides

Slides

Feb 10 Reliability and Fault Tolerance Eraser: A Dynamic Data Race Detector for Multithreaded Programs

Improving the Reliability of Commodity Operating Systems

Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency (No review)

Chao Kong

Ya-Yun Lo

Slides

Slides

Feb 12 No Class End-To-End Arguments in System Design (No review)
Feb 15 Virtual Machines Xen and the Art of Virtualization

Live Migration of Virtual Machines

Ofir Weisse

Juncheng Gu

Slides

Slides

Feb 17 Memory Resource Management in VMware ESX Server

ReVirt: Enabling Intrusion Analysis Through Virtual-Machine Logging and Replay

Clyde Byrd III

Nathan & Seth

Slides

Slides

Feb 19 No Class How to Write a Great Research Paper (No review)
Feb 22 Midterm Review  Mosharaf Slides
Feb 24 No Class – Take-home Midterm
Feb 26 No Class – Work on Projects
Feb 29 No Class – Winter Break
Mar 2
Mar 4
Mar 7 Datacenter as a Computer The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Second Edition

The Datacenter Needs an Operating System (No review)

 Mosharaf Slides
Mar 9 Datacenter Programming Models MapReduce: Simplified Data Processing on Large Clusters

Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing

Yang Liu

Qi Gao

Slides

Slides

Mar 11 No Class – Work on Projects
Mar 14 Datacenter Operating System Large-Scale Cluster Management at Google with Borg

Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center

YARN: Yet Another Resource Negotiator (No review)

Xianghan Pei

Youngmoon Lee

Slides

Slides

Mar 16 No Class – NSDI
Mar 18
Mar 21 Mid-Semester Project Presentations – 1
Mar 23 Mid-Semester Project Presentations – 2 
Mar 25 No Class – Work on Projects
Mar 28 Datacenter Resource Allocation and Scheduling Omega: Flexible, Scalable Schedulers for Large Compute Clusters

Dominant Resource Fairness: Fair Allocation of Multiple Resource Types

HUG: Multi-Resource Fairness for Correlated and Elastic Demands (No review)

Yikai & Chao

Ning & Hanyun

Slides

Slides

Mar 30 Datacenter File System The Google File System

Flat Datacenter Storage

Chao-Han Tsai

Kuangyuan & Qi

Slides

Slides

Apr 1 No Class – Work on Projects
Apr 4 Datacenter Memory Management PACMan: Coordinated Memory Caching for Parallel Jobs

Scaling Distributed Machine Learning with the Parameter Server

Dong-Hyeon Park

Andrew Quinn

Slides

Slides

Apr 6 Datacenter Network Scheduling Data Center TCP (DCTCP)

Efficient Coflow Scheduling with Varys

Yikai Lin

Chao-Han & Chi-Fan

Slides

Slides

Apr 8  Rack-Scale Computer Pelican: A Building Block for Exascale Cold Data Storage (No review)

R2C2: A Network Stack for Rack-Scale Computers (No review)

Network Support for Resource Disaggregation in Next-Generation Datacenters (No review)

Juncheng Gu

Youngmoon Lee

Slides

Slides

Apr 11 No Class – Work on Projects
Apr 13
Apr 15  Wrap Up  Mosharaf
Apr 18 Final Project Presentation

Prerequisites

The formal prerequisite for this course is EECS 482 (Introduction to Operating Systems), equivalent coursework from another university, or permission from the instructor. Courses in database systems, distributed systems, and networking are also helpful. The informal prerequisite is a strong interest in graduate systems research. You need (or need to acquire) strong technical skills to carry out a substantial design and implementation effort.

Undergrads must receive explicit permission from the instructor to enroll.

Policies

[hi]Honor Code[/hi] The Engineering Honor Code applies to paper summaries and exams.

[hi]Paper Reviews[/hi] Read all the papers of each day carefully and write your own paper review. Being able to critically judge others’ work is crucial for your understanding. For papers that require summaries, you must address a number of questions including:

  • What is the problem addressed by the paper, and why is this problem important?
  • What is the hypothesis of the work?
  • What is the proposed solution, and what key insight guides their solution?
  • What is one (or more) drawback or limitation of the proposal, and how will you improve it?

The paper reviews of each day must be submitted electronically at least 24 hours prior to the corresponding class. Late reviews will not be counted. You can miss up to four paper reviews during the term without penalty. Each missing review beyond the fourth one will result in 25% decrease in grade for paper reviews. Meaning, failing to turn in eight or more reviews on time will result in a zero for all paper reviews.

All reviews will be graded as “weak,” “average,” and “strong.” Allocate enough time for your reading and plan your reviews carefully.

[hi]Paper Presentation[/hi] The course will be conducted as a seminar. Each presentation should last at most 20 minutes followed by a class discussion of the designated paper(s). Each student will be assigned to present a paper at least once throughout the semester. When the number of students exceed the number of papers, some papers will be presented by a team of two students.In the presentation, you should:

  • Motivate the paper and provide background.
  • Discuss the high level idea, approach, and/or insight (using examples, whenever appropriate). Please don’t go into details because the other students are expected to have read them for paper summaries; focus only on primary contributions.
  • Explain the difference between this paper and related work (including other readings of the same day or prior days).

The slides for your presentation must be emailed to the instructor at least 24 hours prior to the corresponding class. You should ideally use this template for making your slides in powerpoint.

[hi]Participation[/hi] You are expected to attend all lectures (you may skip up to 2 lectures due to legitimate reasons), and more importantly, participate in class discussions.

[hi]Midterm Exam[/hi] There will be an open-book, take-home midterm exam based on the mandatory readings of the class up to that point in our regular meeting place. Each student must complete the exam solely by her or his own efforts. The exam must be completed within the specified time.

[hi]Final Project[/hi] The largest component of the course is a class project. You will work in groups of at most three people on an instructor-approved problem and have original contribution. Surveys are not permitted as projects; instead, each project must contain a survey of background and related work. You must meet the following milestones (unless otherwise specified in the announcements section above) to ensure a high-quality project at the end of the semester:

  • Form a group of 2-3 people as soon as possible.
  • Turn in a 2-page draft proposal (including references) by January 22. Remember to include the names and Michigan email addresses of the group members. Schedule a 15-minute meeting to pitch your idea and to get early feedback.
  • Keep revising your initial idea and get more feedback until we reach an agreement. However, your team and project proposal must be finalized and approved by me on or before February 5.
  • Each group must present mid-semester progress six weeks later during class hours on March 21 and March 23.
  • Each group must present their final results during a poster session on April 18.
  • Each group must turn in the final report and your code via email on or before 11:59PM EST on April 19. The report must be submitted as a PDF file, with formatting similar to that of the papers you’ve read in the class (revisit slide 10 from this slide deck). The self-contained (i.e., include ALL dependencies) code must be submitted as a zip file. Each zip file containing the code must include a README file with a step-by-step guide on how to compile and run the provided code.

I am always available during office hours to discuss projects, and you may also make appointments at other times. It is crucial that you start working on your project early, or you are unlikely to finish in time.

Grading

Paper Reviews 20% Written summaries of 3-4 papers per week.
Paper Presentation 15% At least one paper presentation per student.
Participation 10% Preparation and participation in classroom discussions.
Midterm Exam 15% Open-book, take-home exam.
Final Project 40% Substantial research project with a final paper and poster presentation.