Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention
Zyphra released ZHA, a sparse attention method that uses a multi-level tree indexer to select relevant KV cache entries without scanning the full context. The method adds only 81,920 trainable parameters to frozen pretra...