summ_entropy() computes entropy of single distribution while
summ_entropy2() - for a pair of distributions. For "discrete"
pdqr-functions a classic formula
-sum(p * log(p)) (in nats) is used. In
"continuous" case a differential entropy is computed.
summ_entropy(f) summ_entropy2(f, g, method = "relative", clip = exp(-20))
A pdqr-function representing distribution.
A pdqr-function. Should be the same type as
Entropy method for pair of distributions. One of "relative" (Kullback–Leibler divergence) or "cross" (for cross-entropy).
Value to be used instead of 0 during
A single number representing entropy. If
clip is strictly positive,
then it will be finite.
Note that due to pdqr approximation error there can be a rather big error in entropy estimation in case original density goes to infinity.
Other summary functions:
#>  1.418913summ_entropy2(d_norm, d_norm_2)#>  9.006174summ_entropy2(d_norm, d_norm_2, method = "cross")#>  10.42509# Increasing `clip` leads to decreasing maximum output value d_1 <- new_d(1:10, "discrete") d_2 <- new_d(20:21, "discrete") # Formally, output isn't clearly defined because functions don't have the # same support. Direct use of entropy formulas gives infinity output, but # here maximum value is `-log(clip)`. summ_entropy2(d_1, d_2, method = "cross")#>  20summ_entropy2(d_1, d_2, method = "cross", clip = exp(-10))#>  10summ_entropy2(d_1, d_2, method = "cross", clip = 0)#>  Inf