Real-time data processing applications with low latency requirements have led to the increasing popularity of stream processing systems. While such systems offer convenient APIs that can be used to achieve data parallelism automatically, they offer limited support for computations that require synchronization between parallel nodes. In this paper, we propose dependency-guided synchronization (DGS), an alternative programming model for stateful streaming computations with complex synchronization requirements. In the proposed model, the input is viewed as partially ordered, and the program consists of a set of parallelization constructs which are applied to decompose the partial order and process events independently. Our programming model maps to an execution model called synchronization plans which supports synchronization between parallel nodes. Our evaluation shows that APIs offered by two widely used systems—Flink and Timely Dataflow—cannot suitably expose parallelism in some representative applications. In contrast, DGS enables implementations with scalable performance, the resulting synchronization plans offer throughput improvements when implemented manually in existing systems, and the programming overhead is small compared to writing sequential code.
Mon 4 AprDisplayed time zone: Eastern Time (US & Canada) change
10:20 - 11:20 | |||
10:20 15mTalk | CASE: A Compiler-Assisted SchEduling Framework for Multi-GPU Systems Main Conference Chao Chen Amazon Web Service, Chris Porter Georgia Institute of Technology, USA, Santosh Pande Georgia Institute of Technology | ||
10:35 15mTalk | Dopia: Online Parallelism Management for Integrated CPU/GPU Architectures Main Conference Younghyun Cho University of California, Berkeley, Jiyeon Park Seoul National University, Florian Negele ETH Zurich, Changyeon Jo Seoul National University, Thomas Gross ETH Zurich, Bernhard Egger Seoul National University | ||
10:50 15mTalk | Mashup: Making Serverless Computing Useful for HPC Workflows via Hybrid Execution Main Conference Rohan Basu Roy Northeastern University, Tirthak Patel Northeastern University, Vijay Gadepally MIT Lincoln Laboratory, Devesh Tiwari Northeastern University | ||
11:05 15mTalk | Stream Processing with Dependency-Guided Synchronization Main Conference Konstantinos Kallas University of Pennsylvania, Filip Niksic Google, Caleb Stanford University of Pennsylvania, Rajeev Alur University of Pennsylvania |