← All resources

Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization

Auto-discovered

Paper Advanced

Jiading Gai, Shuai Zhang, Kaj Bostrom, Jin Huang, Vihang Patil, Haoyang Fang, Bernie Wang, Huzefa Rangwala, George Karypis

Visit resource ↗
Optimization Code

Description

Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization introduces KernelPro, a multi-agent loop that generates CUDA kernels and iterates using hardware profiler feedback.

Pluggable micro-profiling tools act as expert surrogates so the LLM receives structured signals about occupancy, memory traffic, and instruction mix instead of guessing from source alone.

Experiments on KernelBench and MoE training kernels report speedups over prior CUDA coding agents, including an energy-efficiency comparison on representative workloads.

Readers need intermediate CUDA, basic LLM agent patterns, and familiarity with NVIDIA profiling tools to reproduce the workflow.

LLM-plus-profiler GPU optimization is an active research thread; this paper is a concrete reference implementation for kernel iteration loops.

Link last verified accessible on 2026-09-28.

Related resources