论文标题
经验相对熵的有限样本浓度围绕其平均值
Finite-sample concentration of the empirical relative entropy around its mean
论文作者
论文摘要
在本说明中，我们表明，相对于真实的基础分布，$ n $样品的经验分布的相对熵指数围绕其预期，而中央力矩生成函数则构成了gamma分布的形状$ 2K $ $ 2K $和速率$ n/2 $。这改善了Bhatt和Pensia（Arxiv 2021）在同一问题上的工作，他们的形状与$ K $的额外的聚类因子相似，并且还证实了Mardia等人最近的猜想。 （信息和推论2020）。通过将多项式分布的$ k> 3 $减少到二项式的$ k = 2 $的情况下，证明是通过将所需的绑定符合二项式浓度的标准结果遵循的。
In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponentially concentrated around its expectation, with central moment generating function bounded by that of a gamma distribution with shape $2k$ and rate $n/2$. This improves on recent work of Bhatt and Pensia (arXiv 2021) on the same problem, who showed such a similar bound with an additional polylogarithmic factor of $k$ in the shape, and also confirms a recent conjecture of Mardia et al. (Information and Inference 2020). The proof proceeds by reducing the case $k>3$ of the multinomial distribution to the simpler case $k=2$ of the binomial, for which the desired bound follows from standard results on the concentration of the binomial.
