论文标题
什么是一个好的预测？评估代理商知识的挑战
What's a Good Prediction? Challenges in evaluating an agent's knowledge
论文作者
论文摘要
通过学习与世界无关的世界模型来构建一般知识可以帮助代理解决具有挑战性的问题。但是，构建和评估此类模型都是一个开放的挑战。评估模型的最常见方法是评估其相对于可观察值的准确性。但是，主要依赖估算器准确性作为知识实用性的代理，有可能使我们误入歧途。我们通过一系列说明性示例（包括Minecraft中的思想实验和经验示例），使用一般价值函数框架（GVF）来证明准确性和实用性之间的冲突。在确定了评估代理商知识的挑战之后，我们提出了一种替代评估方法，该方法在在线持续学习环境中不断出现，建议通过检查内部学习过程，特别是GVF特征与手头预测任务的相关性。本文通过使用它们的使用，这是对预测的评估，这是预测知识的不可或缺的组成部分，但尚未探索。
Constructing general knowledge by learning task-independent models of the world can help agents solve challenging problems. However, both constructing and evaluating such models remains an open challenge. The most common approaches to evaluating models is to assess their accuracy with respect to observable values. However, the prevailing reliance on estimator accuracy as a proxy for the usefulness of the knowledge has the potential to lead us astray. We demonstrate the conflict between accuracy and usefulness through a series of illustrative examples including both a thought experiment and empirical example in MineCraft, using the General Value Function framework (GVF). Having identified challenges in assessing an agent's knowledge, we propose an alternate evaluation approach that arises continually in the online continual learning setting we recommend evaluation by examining internal learning processes, specifically the relevance of a GVF's features to the prediction task at hand. This paper contributes a first look into evaluation of predictions through their use, an integral component of predictive knowledge which is as of yet unexplored.
