ModHarness-RL treats agentic RL harnesses as learnable, evolving programs
ModHarness-RL reframes the harness around an agentic model — the scaffold controlling what the model sees of its interaction and what guidance it gets — as programs learned from data rather than hand-written and fixed. Interactions are recorded as structured State made of typed blocks, with a Compression program assembling history into context and a Steering program supplying guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories so the harness tracks shifts in model behavior. Experiments on CodeSearch, plus ALFWorld and BIRD SQL with models from 1.5B to 8B, show the Compression and Steering mechanisms improve held-out performance over a fixed-harness baseline in most settings.
It offers a concrete method for co-evolving the agent scaffold with the policy during RL, which is useful for researchers and engineers building agentic training pipelines.
Comments & Questions (0)
No comments yet
Be the first to comment!