Check out our ICML 2025 paper on making interpretable sparse wide neurl networks possible leveraging sparsity and mixture-of-experts tricks.