论文标题
参数有效的抽象问题回答表或文本
Parameter-Efficient Abstractive Question Answering over Tables or Text
论文作者
论文摘要
寻求质量检查系统的信息的长期野心是在多模式上下文上进行推理，并为用户查询产生自然答案。如今，内存密集的预训练的语言模型可以通过在特定模式（如非结构化文本或结构化表）中对质量图数据进行微调，以适应下游任务，例如QA。为了避免训练这种记忆模型，同时利用每种模式的统一体系结构，参数有效的适配器在变压器层之间添加和训练小型任务特定于特定于任务的瓶盖层。在这项工作中，我们在结构化表格数据和非结构化的文本数据上研究了参数效率高效的抽象质量质量QA，每种模式仅使用1.5％的附加参数。我们还在编码器和解码器模块中的适配器层上消融，以研究效率 - 性能权衡取舍，并证明将额外的可训练参数降低到0.7％-1.0％会导致相当的结果。我们的模型在表格QA数据集（例如TableSum和Fetaqa）上超过当前最新模型，并在诸如NordaiteDQA（例如NordaiteDQA）上实现可比的性能，而不是训练参数明显较小，而不是微调参数。
A long-term ambition of information seeking QA systems is to reason over multi-modal contexts and generate natural answers to user queries. Today, memory intensive pre-trained language models are adapted to downstream tasks such as QA by fine-tuning the model on QA data in a specific modality like unstructured text or structured tables. To avoid training such memory-hungry models while utilizing a uniform architecture for each modality, parameter-efficient adapters add and train small task-specific bottle-neck layers between transformer layers. In this work, we study parameter-efficient abstractive QA in encoder-decoder models over structured tabular data and unstructured textual data using only 1.5% additional parameters for each modality. We also ablate over adapter layers in both encoder and decoder modules to study the efficiency-performance trade-off and demonstrate that reducing additional trainable parameters down to 0.7%-1.0% leads to comparable results. Our models out-perform current state-of-the-art models on tabular QA datasets such as Tablesum and FeTaQA, and achieve comparable performance on a textual QA dataset such as NarrativeQA using significantly less trainable parameters than fine-tuning.
