Low Complexity Estimation Method of Renyi Entropy for Ergodic Sources

Since the entropy is a popular randomness measure, there are many studies for the estimation of entropies for given random samples. In this paper, we propose an estimation method of the Renyi entropy of order . Since the Renyi entropy of order is a generalized entropy measure including the Shannon entropy as a special case, the proposed estimation method for Renyi entropy can detect any significant deviation of an ergodic stationary random source's output. It is shown that the expected test value of the proposed scheme is equivalent to the Renyi entropy of order . After deriving a general representation of parameters of the proposed estimator, we discuss on the particular orders of Renyi entropy such as 1, =1/2, and =2. Because the Renyi entropy of order 2 is the most popular one, we present an iterative estimation method for the application with stringent resource restrictions.