FOSCAR Accelerates 2- and 4-Bit KV Cache Decode Attention via Affine Folding
FOSCAR is a GPU kernel library that speeds up decode attention over 2-bit and 4-bit quantized KV caches by applying scale and offset through affine folding rather than per-value conversion. It uses bit manipulation and w...