Back to AI News
arxiv

Representational Atoms in Neural Networks

The paper formally defines fundamental representational units (FRUs) and introduces fidelity and stability as evaluation criteria. It proves that threshold-activated sparse autoencoders can identify FRUs with high fidelity and stability under appropriate conditions, and reports large-scale experiments supporting the result. Further analyses indicate that higher-stability FRUs are more monosemantic and can be composed across layers to recover semantic organization corresponding to human conceptual structure. The work offers a unified framework for defining, evaluating, and identifying representational units in neural networks.

AI Brief

It gives interpretability researchers a formal framework and a sparse-autoencoder method for identifying stable, monosemantic units in model representations.

arXiv:2611.14168v1 Announce Type: cross Abstract: Understanding the internal representations of neural networks and characterizing their relationship with human conceptual structure are two critical questions for elucidating model mechanisms. The former reveals how learned knowledge is organized in internal representations, whereas the latter provides a basis for interpreting these representations through human conceptual structure. Here, we formally define fundamental representational units (FRUs) and establish two evaluation criteria: fidelity and stability. We further prove that threshold-activated sparse autoencoders (TSAEs) can identify FRUs with high fidelity and stability under appropriate conditions, and validate this result through large-scale experiments. Further analyses reveal that FRUs with higher stability exhibit greater monosemanticity and can be composed across layers to recover semantic organization corresponding to human conceptual structure. These results provide a unified framework for defining, evaluating and identifying FRUs in neural networks, and establish a foundation for characterizing the relationship between neural network representations and human conceptual structure.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!