NVIDIA Launches Topograph for Topology-Aware GPU Scheduling

NVIDIA has unveiled Topograph, an open-source toolkit designed to optimize GPU workload placement in AI factories by leveraging topology-aware scheduling. This new solution addresses inefficiencies caused by fragmented resource placement, which can throttle GPU throughput and inflate operational costs in AI training and inference workloads.

At its core, Topograph maps the physical and logical connections between GPUs, network fabrics like NVLink and InfiniBand, and other cluster components. This allows schedulers like Kubernetes, Slurm, and Slinky to make smarter placement decisions by co-locating tightly coupled jobs on GPUs with superior interconnects. NVIDIA claims this approach reduces latency, minimizes inter-GPU traffic across slower links, and improves cost efficiency by ensuring GPUs spend less time idling while waiting for data.

Why It Matters for AI Workloads

Topology-aware scheduling has become a critical focus as AI workloads grow more demanding and distributed. Large-scale training and inference tasks often involve heavy data exchange between GPUs. Without topology awareness, these workloads risk being spread across distant or poorly connected GPUs, leading to network congestion, higher latency, and underutilized accelerators.

By integrating Topograph, schedulers can optimize placement based on real-time topology data. For example, GPUs connected via NVIDIA NVLink or NVSwitch—offering up to 3.6 TB/s of bandwidth in the latest generation—can be prioritized for tightly coupled jobs, while jobs with looser requirements can utilize slower PCIe paths or Ethernet fabrics.

How It Works

Topograph discovers cluster topology through cloud APIs (e.g., Google Cloud, OCI) or on-premises systems like NVLink and Spectrum-X fabrics. It normalizes this data into a common model and exports it in scheduler-friendly formats, such as Kubernetes node labels, Slurm topology configurations, or Slinky ConfigMaps.

The tool also integrates with NVIDIA’s DSX OS, working alongside Dynamic Resource Allocation (DRA) and KAI Scheduler to enable gang scheduling—ensuring all pods in a job are placed within the same topology domain. Kubernetes v1.37, announced earlier this month, further supports this by expanding topology-aware scheduling features to handle multi-level workload hierarchies.

Broader Implications

The release of Topograph reflects NVIDIA’s ongoing push to maximize GPU efficiency across both cloud and on-premises AI infrastructure. This comes at a time when AI model training costs are skyrocketing due to ever-larger datasets and models. By improving network locality and reducing idle GPU time, Topograph could lower the total cost of ownership for AI factories, potentially making high-performance computing more accessible to smaller enterprises.

In addition, Topograph aligns with other recent NVIDIA contributions to the Kubernetes ecosystem, such as the donation of its Dynamic Resource Allocation driver and its ongoing work on KAI Scheduler, a CNCF Sandbox project. This reinforces NVIDIA’s role as a leader in AI infrastructure optimization.

What’s Next

Topograph is available as an open-source project on GitHub and can be deployed via Helm for Kubernetes users or native packages for Slurm clusters. With support for both cloud and on-premises environments, it aims to provide a unified scheduling solution across diverse deployment models.

For AI developers and infrastructure operators, adopting Topograph could unlock significant performance gains, particularly in multi-node training and inference scenarios. As Kubernetes and other schedulers continue to enhance topology-aware features, tools like Topograph will likely become essential for staying competitive in the AI race.

Image source: Shutterstock