论文标题
组装自动代码摘要的基础模型
Assemble Foundation Models for Automatic Code Summarization
论文作者
论文摘要
自动代码摘要对日常软件开发有益，因为它可以帮助减少手动编写的要求。目前，人工智能正在经历范式转变。根据大量数据预估计的基础模型，并针对下游任务进行了填充，超过了专门定制的模型。这种趋势激发了我们考虑重复使用基础模型而不是从头开始学习的趋势。因此，我们基于神经模型提出了一种灵活而强大的方法来自动代码摘要。我们将可用的基础模型（例如Codebert和gpt-2）组装成名为Adamo的单个神经模型。此外，我们利用高斯噪声作为上下文信息的仿真来优化潜在表示。此外，我们从知识转移的角度介绍了两种自适应方案，即连续预处理和中间填充，并为一般序列到序列学习设计中间阶段任务。最后，我们通过将ADAMO与最新模型进行比较来评估ADAMO用于代码摘要的基准数据集。
Automatic code summarization is beneficial to daily software development since it could help reduce the requirement of manual writing. Currently, artificial intelligence is undergoing a paradigm shift. The foundation models pretrained on massive data and finetuned to downstream tasks surpass specially customized models. This trend inspired us to consider reusing foundation models instead of learning from scratch. Thereby, we propose a flexible and robust approach for automatic code summarization, based on neural models. We assemble available foundation models, such as CodeBERT and GPT-2, into a single neural model named AdaMo. Moreover, we utilize Gaussian noise as the simulation of contextual information to optimize the latent representation. Furthermore, we introduce two adaptive schemes from the perspective of knowledge transfer, namely continuous pretraining and intermediate finetuning, and design intermediate stage tasks for general sequence-to-sequence learning. Finally, we evaluate AdaMo against a benchmark dataset for code summarization, by comparing it with state-of-the-art models.
