论文标题
使用统计推断和交互式可视化在学习表示中发现概念
Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization
论文作者
论文摘要
概念发现是可解释性文献中的开放问题之一，对于弥合非深度学习专家与模型最终用户之间的差距很重要。在当前的配方中，概念将它们定义为学习的表示空间中的方向。该定义使得评估特定概念是否显着影响兴趣类别的分类决策。但是，找到相关概念是乏味的，因为表示空间是高度的，而且很难浏览。当前的方法包括手工制作概念数据集，然后将其转换为潜在空间方向；另外，可以通过聚类潜在空间来自动化该过程。在这项研究中，我们提供了另一种方法来指导用户发现有意义的概念，一种基于多个假设检验，另一种基于交互式可视化。我们通过仿真实验以及对真实数据的演示视觉界面探索这些方法的潜在价值和局限性。总体而言，我们发现这些技术提供了一种有希望的策略，可以在用户没有对其进行预定义的描述但没有完全自动化过程的情况下发现相关概念。
Concept discovery is one of the open problems in the interpretability literature that is important for bridging the gap between non-deep learning experts and model end-users. Among current formulations, concepts defines them by as a direction in a learned representation space. This definition makes it possible to evaluate whether a particular concept significantly influences classification decisions for classes of interest. However, finding relevant concepts is tedious, as representation spaces are high-dimensional and hard to navigate. Current approaches include hand-crafting concept datasets and then converting them to latent space directions; alternatively, the process can be automated by clustering the latent space. In this study, we offer another two approaches to guide user discovery of meaningful concepts, one based on multiple hypothesis testing, and another on interactive visualization. We explore the potential value and limitations of these approaches through simulation experiments and an demo visual interface to real data. Overall, we find that these techniques offer a promising strategy for discovering relevant concepts in settings where users do not have predefined descriptions of them, but without completely automating the process.
