Research
PolyRoute Benchmark Tests Typed-Decision Routing Across 12 Languages and Code-Mixed Text
PolyRoute is a new benchmark of 322 native-language support messages in 12 languages and 9 writing systems, plus 156 code-mixed or romanised messages such as Hinglish, Spanglish and Arabizi, for a three-way department-routing decision. Labels were produced by Qwen3.8-27B with a blind second pass, and an independent model agreed on 97.2% of test items. A leaderboard of 13 systems reports 61.2% for the English-root checkpoint, 77.3% for the shipped multilingual checkpoint and 96.6% for the best fine-tune, while TF-IDF with logistic regression reaches 91.9%. The authors note that the split resolves differences only above about 4 points and that all texts and labels are LLM-generated.
PolyRoutemultilingual NLPbenchmark evaluation