Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
Researchers from Columbia University and CertiK introduced TensorLean, a Lean-based framework for formally verifying GPU kernels at the PTX level against tensor-level math specifications. The system checks properties including data-race freedom, termination, correct output within tolerance, and no stray memory writes. In its evaluation, 83 of 149 kernel instances were proved correct and 38 correctness bugs were found across projects including FlashAttention-2/3/4 and DeepGEMM, including a FlashAttention-4 data race that never appeared in 17,760 test runs. The paper and code are announced as coming soon, and the authors note limitations such as unsupported random sampling and acquire/release synchronization patterns.
Why it matters: It presents a concrete formal-verification advance for GPU kernels and documents real bugs found in widely used attention and GEMM implementations.