Back to AI News
arxiv

Joint Quantization and Dynamic Partitioning for Edge Vision Transformers

This arXiv paper proposes an edge orchestration framework for deploying vision transformers across connected and automated vehicles. It jointly selects model partition points, per-layer quantization bitwidths, and execution placement, formulated as a MILP that minimizes compute and communication energy under latency and task-quality constraints. Evaluated on Jetson Orin Nano, NX, and AGX Orin nodes with OPV2V multi-stream workloads, it reports 33.3–36.0% lower V2V communication energy than a Uniform INT8 split and up to 34.0% lower end-to-end latency at K=8, with 3.1–4.1 percentage-point mAP50 reduction versus FP16.

AI Brief

It offers a concrete optimization method and measured trade-offs for running vision transformers across bandwidth-constrained edge devices.

arXiv:2611.14196v1 Announce Type: new Abstract: Deploying vision transformer models across connected and automated vehicles (CAVs) requires joint management of computation and Vehicle-to-Vehicle (V2V) communication. Static split-inference methods use fixed partition points and cannot adapt to time-varying link bandwidth, processing load, and concurrent multi-stream traffic. This paper proposes an edge orchestration framework that jointly selects model partition points, per-layer quantization bitwidths, and execution placement across heterogeneous vehicular compute nodes. The joint partitioning and mixed-precision allocation problem is formulated as a Mixed-Integer Linear Program (MILP) that minimizes compute and communication energy subject to end-to-end latency and task-quality constraints. The framework is evaluated on Jetson Orin Nano (8~GB), Jetson Orin NX (16~GB), and Jetson AGX Orin (32~GB) nodes using OPV2V-driven multi-stream workloads with $K \in \{2,4,6,8\}$. Across the evaluated configurations, the proposed formulation reduces V2V communication energy by $33.3$--$36.0\%$ relative to a Uniform INT8 split. At $K=8$, end-to-end task completion latency is reduced by $34.0\%$ on the Orin Nano and $33.1\%$ on the Orin NX relative to the same baseline, while limiting the reduction in 3D detection accuracy to $3.1$--$4.1$ percentage points in $\mathrm{mAP}_{50}$ relative to FP16 execution.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!