Explore tools, prompts, skills, news and resources related to tokenization.
Prompt works related to tokenization
Latest articles, prompts, use cases and editorial content about tokenization
Amazon SageMaker AI introduced KV cache-aware routing, a routing strategy that uses cache state reported by model containers to send requests to instances holding the longest cached prompt match while weighing instance l...
Researchers propose a multi-bit LLM watermarking scheme that maps an n-bit payload to a longer u-bit codeword and embeds it during generation via token-level logit biasing. Extraction matches the received bit sequence to...
This Microsoft Research podcast roundup highlights several concrete AI advances, including a resolved 31-year-old mathematics challenge known as the KLS conjecture, a new benchmark and method for making AI coding agents...
Researchers introduce dNaViT, a Discrete Native-resolution Vision Transformer that produces discrete visual primitives analogous to language tokens, built on a frozen pretrained encoder. The method uses lightweight resid...
This paper introduces AssameseLM-E2B, a 2-billion-parameter language model adapted for the low-resource Assamese language, built on the Gemma-4 architecture with an expanded byte-level BPE vocabulary. It uses parameter-e...
SpecSlot is a pixel-space rectified-flow Transformer that expands each spatial anchor into multiple slots, each assigned a disjoint band of DCT coefficients, so anchor density is separated from spectral subdivision. Only...
Google DeepMind announced Gemini 4 Carbon, a new frontier model positioned for cybersecurity and long-horizon tasks, initially available to trusted cyber defenders through the Fairwind Program with wider developer access...
Gradium, a Paris voice AI company spun out of Kyutai, has released Phonon, a family of small text-to-speech models that run fully on-device. The English Phonon 31m ships under Apache 2.0 at 64.5 MB (q8 GGUF) with 31M par...
Microsoft Research Asia released Agent Lightning v1.0, a lightweight framework that implements a complete agentic reinforcement learning control plane in roughly 3,500 lines of code. It introduces Harnessed Agentic RL, w...
Liquid AI released LFM2.5-24B-A2B, an open-weight 24B-parameter Mixture-of-Experts reasoning model with about 2B active parameters and 131,072-token context, targeted at local agent loops. It runs via GGUF/MLX on laptops...
As the boundaries between AI, AR, and XR become increasingly blurred, the industry must first clarify a fundamental proposition: What exactly do we mean by "AI glasses"? Generally speaking, AI glasses integrate...
Discover tools, products and platforms related to tokenization