Latest AI updates

AutoCompress: Hardware-Guided Model Compression for LLMs

AutoCompress is a new framework that integrates hardware and workload constraints directly into differentiable architecture search for LLM pruning. It jointly optimizes global and per-layer structure using serving-engine cost models, improving accuracy over Minitron across multiple GPUs and workloads. The method also compresses a 120B model to 74B parameters while retaining 95% of benchmark performance.