Rethinking Thread Scheduling under Oversubscription: A User-Space Framework for Coordinating Multi-runtime and Multi-process WorkloadsBest Paper Nominee
The convergence of high-performance computing (HPC) and artificial intelligence (AI) is driving the emergence of increasingly complex parallel applications and workloads. These workloads often combine multiple parallel runtimes within the same application or across co-located jobs, creating scheduling demands that place significant stress on traditional OS schedulers. When oversubscribed (there are more ready threads than cores), OS schedulers rely on periodic preemptions to multiplex cores, often introducing interference that may degrade performance. In this paper, we present: (1) The User-space Scheduling Framework (USF), a novel seamless process scheduling framework completely implemented in user-space. USF enables users to implement their own process scheduling algorithms without requiring special permissions. We evaluate USF with its default cooperative policy, (2) SCHED_COOP, designed to reduce interference by switching threads only upon blocking. This approach mitigates well-known issues such as Lock-Holder Preemption (LHP), Lock-Waiter Preemption (LWP), and scalability collapse. We implement USF and SCHED_COOP by extending the GNU C library with the nOS-V runtime, enabling seamless coordination across multiple runtimes (e.g., OpenMP) without requiring invasive application changes. Evaluations show gains up to 2.4x in oversubscribed multi-process scenarios, including nested BLAS workloads, multi-process PyTorch inference with LLaMA-3, and Molecular Dynamics (MD) simulations.
Mon 2 FebDisplayed time zone: Hobart change
11:30 - 12:50 | |||
11:30 20mTalk | Rethinking Thread Scheduling under Oversubscription: A User-Space Framework for Coordinating Multi-runtime and Multi-process WorkloadsBest Paper Nominee Main Conference DOI | ||
11:50 20mTalk | Waste-Efficient Work Stealing Main Conference Kyle Singer Massachusetts Institute of Technology, Kunal Agrawal Washington University in St. Louis, TB Schardl Massachusetts Institute of Technology DOI | ||
12:10 20mTalk | DiggerBees: Depth First Search Leveraging Hierarchical Block-Level Stealing on GPUs Main Conference Yuyao Niu Barcelona Supercomputing Center, Yuechen Lu China University of Petroleum-Beijing, Weifeng Liu China University of Petroleum-Beijing, Marc Casas Barcelona Supercomputing Center DOI | ||
12:30 20mTalk | PANA: A Fine-Grained Runtime-Adaptive Load Balancing for Parallel SpMV on Multicore CPUs Main Conference Haodong Bian Tsinghua University, Youhui Zhang Tsinghua University, Xiang Fei Tsinghua University, Jianqiang Huang Qinghai University, Xiaoying Wang Qinghai University DOI | ||