Simple Generate-and-Select Baseline Matches Costlier Prompt Optimizers, Audit Finds
Researchers introduce Demonstration-Conditioned Prompt Search (DCPS), a deliberately simple control that freezes a generator, conditions it on randomly sampled demonstrations, scores candidates on validation data, and keeps the best candidate without learned updates, reflection, or evolution. Across two optimizer families, DCPS reportedly matches StablePrompt's PPO-based classification accuracy within 0.5 percentage points at roughly one-third the measured cost, and matches GEPA's macro accuracy on GPT-4.1-mini in DSPy compound systems using 55% fewer rollouts at 32% lower measured API cost on average. The audit finds gains are task-dependent: proposal-and-selection recovers much of heavier optimizers' performance on structured and instruction-following tasks, while extra mechanisms still help in some open-ended and reasoning-heavy settings. The authors recommend benchmarking prompt optimizers against generate-and-select controls and reporting rollout counts, token usage, and dollar costs alongside accuracy.
It gives practitioners a cheap, reproducible baseline for judging whether elaborate prompt-optimization pipelines actually earn their extra compute and API spend.
Comments & Questions (0)
No comments yet
Be the first to comment!