Frozen ASL Pose Encoder Transfers to Low-Resource Sign Language Retrieval
Researchers evaluate a pose encoder trained with metric learning on American Sign Language, kept frozen, on six low-resource sign language dictionary datasets. With one enrolled clip per sign, the encoder ranks the correct sign first in 36.4–80.7% of queries, compared with 16.0–69.6% for angle-based dynamic time warping and a six-target mean of 39.3% for an ASL-trained OpenHands graph network. An ablation finds that a per-pair InfoNCE objective and blocking attention between body and hands improve accuracy, with the best configuration reaching 68.0% one-shot accuracy versus 59.0% for the reference encoder. On the 4,765-word CISLR Indian Sign Language dictionary, top-1 accuracy is 17.3–19.0% versus 16.8% for the RGB baseline, while embedding geometry has only a minor effect.
It quantifies zero-shot cross-lingual transfer for sign language dictionary retrieval and identifies training and architecture choices that improve low-resource performance.
Comments & Questions (0)
No comments yet
Be the first to comment!