论文标题
用于瑞典语实践的不断发展的对话代理中生成对话模型的质量保证
Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice
论文作者
论文摘要
由于迁移大趋势，因此有效而有效的第二语言获取至关重要。一个提出的解决方案涉及以人为本的以人为本的互动语言实践的对话代理。我们提出了正在进行的行动研究的结果，该研究针对专有生成对话模型的质量保证，该模型接受了虚拟工作面试的培训。行动团队引起了一组38个要求，我们为15个特别感兴趣的自动化测试用例设计了不断发展的解决方案。我们的结果表明，六个测试案例设计可以检测到候选模型之间有意义的差异。尽管自然语言处理应用程序的质量保证很复杂，但我们在不断发展的对话代理的背景下为机器学习模型选择提供了初始步骤。未来的工作将集中于MLOPS设置中的模型选择。
Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.
