论文标题
聚类草图：一种嵌入表压缩的新方法
Clustering the Sketch: A Novel Approach to Embedding Table Compression
论文作者
论文摘要
机器学习系统使用嵌入表来处理分类功能。在现代推荐系统中，这些表可能很大，也需要开发新方法，即使在训练期间，也需要开发它们。我们建议将基于聚类的压缩（例如量化与代码簿）结合使用的组成嵌入（CCE），并使用动态方法（例如哈希技巧和组成嵌入）（Shi等，2020）。在实验上，CCE实现了两全其美的最好的：基于代码本的量化的高压率，但 *动态 *类似于基于哈希的方法，因此可以在训练过程中使用。从理论上讲，我们证明CCE可以保证将其收敛到最佳代码簿，并为所需的迭代次数限制。
Embedding tables are used by machine learning systems to work with categorical features. In modern Recommendation Systems, these tables can be very large, necessitating the development of new methods for fitting them in memory, even during training. We suggest Clustered Compositional Embeddings (CCE) which combines clustering-based compression like quantization to codebooks with dynamic methods like The Hashing Trick and Compositional Embeddings (Shi et al., 2020). Experimentally CCE achieves the best of both worlds: The high compression rate of codebook-based quantization, but *dynamically* like hashing-based methods, so it can be used during training. Theoretically, we prove that CCE is guaranteed to converge to the optimal codebook and give a tight bound for the number of iterations required.
