Research
XStack: A Benchmark for Migrating ML Repositories Between PyTorch and JAX
XStack is a new benchmark that evaluates AI agents on migrating entire machine learning repositories from PyTorch on NVIDIA GPUs to JAX on Google TPUs. The benchmark requires agents to deliver a working JAX repository that meets grading criteria based on the source repository's published results. Current agent performance is low, with the best agent (Claude Opus 5) achieving only 75% pass rate, highlighting the difficulty of cross-framework and cross-hardware migration.
XStackML repository migrationPyTorch to JAX