Back to AI News
arxiv

TactiNavi: Serving and Revising an LLM Game Coach with Explicit Strategy Conditions

TactiNavi is a serving system for an LLM-based coach in a commercial open-world game that makes strategy applicability an explicit artifact. It uses offline compilation of versioned game evidence into strategy packages with prerequisites, online evaluation of three-valued conditions over player state, and a control plane that validates revisions on held-out traffic. In a retrospective evaluation of 1,122 logged turns, condition-bearing packages improved correctness from 57.9% to 82.0% and reduced strategy misapplication from 18.1% to 10.4%, with trace-based claim verification reaching 86.4% precision at 2.07s median latency.

AI Brief

Provides concrete, recent deployment insights and measurable improvements for LLM serving in dynamic environments, valuable for AI engineers and product teams.

arXiv:2611.04772v1 Announce Type: new Abstract: A deployed LLM coaching agent answers requests whose correct response depends on player state and game version, so advice that is right for one player can be wrong for the next. We describe TactiNavi, the serving system behind the LLM coach of a commercial open-world game, which makes strategy applicability an explicit artifact shared by offline compilation, online execution, failure diagnosis, and release. An offline compiler turns versioned game evidence and expert guidance into strategy packages with explicit prerequisites, the serving plane evaluates those prerequisites as three-valued conditions over structured player state and records decision traces, and a control plane uses the resulting condition-rule dependencies to validate revisions on held-out traffic before release. In a retrospective evaluation of 1,122 logged coaching turns from three deployment periods, condition-bearing packages raise correctness from 57.9% to 82.0% over atomic facts and reduce strategy misapplication under changed player state from 18.1% to 10.4%, while trace-based claim verification reaches 86.4% precision at a median reply latency of 2.07 s. The deployment also showed that most apparent fixes are generation noise, because an unchanged-system rerun recovers 38.4% of failing turns and only one of 15 shipped revision units passes the release checks retrospectively. We distill these results into lessons on treating applicability as a first-class serving object, sizing traces for later diagnosis, and building the null model into the release path.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!