← All resources

Making GPUs Actually Fast: A Deep Dive into Training Performance

Auto-discovered

Talk Intermediate

Corwin, Sylvain

Visit resource ↗

Description

Making GPUs Actually Fast: A Deep Dive into Training Performance is a Jane Street YouTube talk by Corwin and Sylvain on keeping GPUs busy while training neural nets in PyTorch.

It contrasts CPU latency with GPU throughput, walks a training loop, then explains launch pipelines and hidden synchronizations from .cpu copies, Boolean tensor coercion, and host-to-device transfers. Fixes include GPU-side metrics, asynchronous NaN checks, pinned memory, and non-blocking copies, plus a simplified SM, register, and shared-memory view. Nsight Systems traces show full versus gappy CUDA timelines.

ML engineers and systems programmers who train on NVIDIA GPUs with PyTorch are the audience. Basic training-loop familiarity is assumed.

Access is a free recorded talk on the Jane Street channel with diagrams and profiler screenshots. It is one session, not a multi-part course.

It helps find host synchronizations that idle expensive GPUs before you micro-optimize kernels.