Research
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
EngineAgent is an agentic system that automates enabling new LLM models in on-device inference engines by separating reusable engine and hardware knowledge from model-specific implementation. It builds a hierarchy-aware engine index, pre-validates backend workflows, and uses a builder-auditor loop to coordinate cross-layer changes and device-side debugging. Across 12 tasks covering three mobile LLMs and four engine-backend pairs on CPUs, GPUs, and NPUs, it achieved 75% end-to-end success versus 33.3% for the strongest coding-agent baseline, while reducing average per-task time and model-API cost.
on-device LLM inferenceAI agentsmobile hardware