Representational Atoms in Neural Networks
The paper formally defines fundamental representational units (FRUs) and introduces fidelity and stability as evaluation criteria. It proves that threshold-activated sparse autoencoders can identify FRUs with high fidelity and stability under appropriate conditions, and reports large-scale experiments supporting the result. Further analyses indicate that higher-stability FRUs are more monosemantic and can be composed across layers to recover semantic organization corresponding to human conceptual structure. The work offers a unified framework for defining, evaluating, and identifying representational units in neural networks.
It gives interpretability researchers a formal framework and a sparse-autoencoder method for identifying stable, monosemantic units in model representations.
Comments & Questions (0)
No comments yet
Be the first to comment!