Write a Blog >>
PPoPP 2022
Sat 2 - Wed 6 April 2022
Tue 5 Apr 2022 14:25 - 14:30 - Poster Session Chair(s): Yan Gu

OpenMP has a long and successful history in parallel programming for CPUs. Since the introduction of accelerator offloading it has evolved into a promising candidate for all intra-node parallel computing needs. While this addition broke with the shared memory assumption OpenMP was initially developed with, efforts to employ OpenMP beyond shared-memory domains are practically non-existent.

In this work we show that the OpenMP accelerator offloading model is sufficient to seamlessly and efficiently utilize more than a single compute node, and its connected accelerators. Without source code or compiler modifications we run an OpenMP offload capable program on a remote CPU, or remote accelerator (e.g., GPU), as if it was a local one. For applications that support multi-device offloading, any combination of local and remote CPUs and accelerators can be utilized simultaneously, fully transparent to the user. Our low-overhead implementation of Remote OpenMP Offloading is integrated into the LLVM/OpenMP compiler infrastructure and is publicly available (in parts) with LLVM 12 and later.

To evaluate our work we provide detailed studies on micro benchmarks, as well as scaling results for two HPC proxy applications. We show perfect (weak) scaling across dozens of GPUs in multiple hosts with effectiveness that is directly proportional to the ratio of computation versus memory transfer time. Our work outlines the capabilities and limits of OpenMP 5.1 to efficiently utilize a distributed heterogeneous system without source, compiler, or language modifications.

Tue 5 Apr

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:25
Poster SessionMain Conference
Chair(s): Yan Gu UC Riverside
14:00
5m
Talk
POSTER: Automatic Synthesis of Parallel Unix Commands and Pipelines with KumQuat
Main Conference
Jiasi Shen Massachusetts Institute of Technology, Martin C. Rinard Massachusetts Institute of Technology, Nikos Vasilakis Massachusetts Institute of Technology
14:05
5m
Talk
POSTER: Towards OmpSs-2 and OpenACC Interoperation
Main Conference
Orestis Korakitis Barcelona Supercomputing Center (BSC), Simon Garcia De Gonzalo Barcelona Supercomputing Center (BSC), Nicolas Guidotti INESC-ID, Instituto Superior Técnico, University of Lisbon, João Barreto INESC-ID, José C. Monteiro INESC-ID, Instituto Superior Técnico, University of Lisbon, Antonio Pena Barcelona Supercomputing Center (BSC)
14:10
5m
Talk
POSTER: LB-HM: Load Balance-Aware Data Placement on Heterogeneous Memory for Task-Parallel HPC Applications
Main Conference
Zhen Xie University of California, Merced, Jie Liu , Sam Ma College of William & Mary, Jiajia Li William & Mary, Pacific Northwest National Laboratory, Dong Li University of California, Merced
14:15
5m
Talk
POSTER: Hardening Selective Protection across Multiple Program Inputs for HPC Applications
Main Conference
Yafan Huang University of Iowa, Shengjian Guo Baidu USA, Sheng Di Argonne National Laboratory, Guanpeng Li University of Iowa, Franck Cappello Argonne National Laboratory
14:20
5m
Talk
POSTER: A Parallel Branch-and-Bound Algorithm with History-Based Domination
Main Conference
Taspon Gonggiatgul California State University, Sacramento, Ghassan Shobaki California State University, Sacramento, Pinar Muyan-Ozcelik California State University, Sacramento
14:25
5m
Talk
POSTER: Remote OpenMP Offloading
Main Conference
Atmn Patel University of Waterloo, Johannes Doerfert Argonne National Laboratory
14:30
5m
Talk
POSTER: High Performance GPU Concurrent B+tree
Main Conference
Weihua Zhang Fudan University, Chuanlei Zhao Fudan University, Lu Peng Louisiana State University, Yuzhe Lin Fudan University, Fengzhe Zhang Fudan University, Jinhu Jiang Fudan University
14:35
5m
Talk
POSTER: The Problem-Based Benchmark Suite (PBBS), V2
Main Conference
Daniel Anderson Carnegie Mellon University, Guy E. Blelloch Carnegie Mellon University, USA, Laxman Dhulipala University of Maryland, College Park, Magdalen Dobson Carnegie Mellon University, Yihan Sun University of California, Riverside
14:40
5m
Talk
POSTER: An LLVM-based Open-Source Compiler for NVIDIA GPUs
Main Conference
Da Yan Hong Kong University of Science and Technology, Wei Wang Hong Kong University of Science and Technology, Xiaowen Chu Data Science and Analytics Thrust, HKUST(GZ)
14:45
5m
Talk
POSTER: ParGeo: A Library for Parallel Computational Geometry
Main Conference
Yiqiu Wang Massachusetts Institute of Technology, Shangdi Yu Massachusetts Institute of Technology, Laxman Dhulipala University of Maryland, College Park, Yan Gu UC Riverside, Julian Shun MIT
14:50
5m
Talk
POSTER: Parallel Algorithms for Masked Sparse Matrix-Matrix Products
Main Conference
Srđan Milaković Rice University, Oguz Selvitopi Lawrence Berkeley National Laboratory, Israt Nisa AWS AI, Zoran Budimlić Rice University, Aydin Buluc Lawrence Berkeley National Laboratory
14:55
5m
Talk
POSTER: Rethinking Graph Data Placement for Graph Neural Network Training on Multiple GPUs
Main Conference
Shihui Song The University of Iowa, Peng Jiang The University of Iowa
15:00
5m
Talk
POSTER: Optimizing Consistency for Partially Replicated Data Stores
Main Conference
Ivan Kuraj MIT CSAIL, USA, Armando Solar-Lezama Massachusetts Institute of Technology, Nadia Polikarpova University of California at San Diego
15:05
5m
Talk
POSTER: Optimizing Sparse Computations Jointly
Main Conference
Kazem Cheshmi University of Toronto, Michelle Strout University of Arizona, Maryam Mehri Dehnavi University of Toronto
15:10
5m
Talk
POSTER: wCQ: A Fast Wait-Free Queue with Bounded Memory Usage
Main Conference
Ruslan Nikolaev The Pennsylvania State University, Binoy Ravindran Virginia Tech
15:15
5m
Talk
POSTER: Automatic Differentiation of Parallel Loops with Formal Methods
Main Conference
Jan Hueckelheim Argonne National Laboratory, Laurent Hascoet Inria
15:20
5m
Talk
POSTER: A W-cycle Algorithm for Efficient Batched SVD on GPUs
Main Conference
Junmin Xiao Institute of Computing Technology of Chinese Academy of Sciences, Qing Xue Institute of Computing Technology, Chinese Academy of Sciences, Hui Ma Institute of Computing Technology, Chinese Academy of Sciences, Xiaoyang Zhang Institute of Computing Technology, Chinese Academy of Sciences, Guangming Tan Chinese Academy of Sciences(CAS)