Live intelligence

Latest AI updates

1 updates
Research

LEDGE: A Simulator for Distributed LLM Serving on Heterogeneous Edge Devices

The paper introduces LEDGE, a lightweight simulator for evaluating distributed LLM serving on heterogeneous edge devices without executing tensor operations. It uses event graphs supporting non-uniform tensor and pipeline parallelism, calibrated reusable performance models, and shared-link contention simulation. On a 10-device Jetson and Raspberry Pi testbed covering 17 model-precision combinations across four LLM families, LEDGE reports request-level MAPE below 3.6% on Qwen deployments and a 135.3x aggregate speedup over physical inference. A model calibrated on one LLM family achieved 7.44% zero-shot end-to-end latency MAPE across three other families, and the source code is publicly available.

LEDGEdistributed LLM servingedge devices