Geek asks and answers

Jensen Shannon's divergence in R

I am new to R and tried to find a function that calculates JS divergence in R. I see that R has KLdiv for calculating the KL divergence, but is there anything for the JS divergence?

+6
source share
3 answers

I was looking for a simple implementation of the JS divergence , not the R library. Since I did not see any of the answers, I suggested the following.

Assuming we have the following input distributions:

# p & q are distributions so their elements should sum up to 1
p <- c(0.00029421, 0.42837957, 0.1371827, 0.00029419, 0.00029419,
       0.40526004, 0.02741252, 0.00029422, 0.00029417, 0.00029418)

q <- c(0.00476199, 0.004762, 0.004762, 0.00476202, 0.95714168,
       0.00476213, 0.00476212, 0.00476202, 0.00476202, 0.00476202)

The Jensen-Shannon divergence will be as follows:

n <- 0.5 * (p + q)
JS <- 0.5 * (sum(p * log(p / n)) + sum(q * log(q / n)))
> JS
[1] 0.6457538

For more than two distributions (which have already been discussed here ), we need a function to calculate the entropy :

H <- function(v) {
      v <- v[v > 0]
      return(sum(-v * log(v)))
}

Then the divergence of JS will be:

JSD <- function(w, m) {
  return(H(m %*% w) - apply(m, 2, H) %*% w)
}

> JSD(w = c(1/3, 1/3, 1/3), m = cbind(p, q, n))
          [,1]
[1,] 0.4305025

w - , 1, m - .

+10

According to Jensen-Shannon's Wikipedia, the discrepancy is a transformation of the KL discrepancy. Applying the formula from the definition should give you a JS discrepancy, then ...

See: http://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence

0
source

More articles:


All Articles