论文标题
在语义传播转换下对神经程序分析仪的普遍性评估
Evaluation of Generalizability of Neural Program Analyzers under Semantic-Preserving Transformations
论文作者
论文摘要
大量可用的源代码存储库以及神经网络中的进步已经实现了数据驱动的程序分析方法。这些方法称为神经程序分析仪，使用神经网络来提取计划中的模式，以进行从开发生产力到程序推理的任务。尽管神经程序分析仪的流行日益普及，但其结果的推广程度尚不清楚。 在本文中，我们使用七个语义等效的程序转换对两个流行的神经程序分析仪的普遍性进行了大规模评估。我们的结果警告说，在许多情况下，神经程序分析仪无法很好地概括，有时会引起文本差异忽略的程序。结果为量化神经程序分析仪的鲁棒性提供了最初的踏板结石。
The abundance of publicly available source code repositories, in conjunction with the advances in neural networks, has enabled data-driven approaches to program analysis. These approaches, called neural program analyzers, use neural networks to extract patterns in the programs for tasks ranging from development productivity to program reasoning. Despite the growing popularity of neural program analyzers, the extent to which their results are generalizable is unknown. In this paper, we perform a large-scale evaluation of the generalizability of two popular neural program analyzers using seven semantically-equivalent transformations of programs. Our results caution that in many cases the neural program analyzers fail to generalize well, sometimes to programs with negligible textual differences. The results provide the initial stepping stones for quantifying robustness in neural program analyzers.
