[Scipy-tickets] [SciPy] #497: Relative entropy calculation has sign inverted

SciPy scipy-tickets@scipy....
Sun Sep 9 21:36:14 CDT 2007

#497: Relative entropy calculation has sign inverted
 Reporter:  lars    |       Owner:  somebody
     Type:  defect  |      Status:  new     
 Priority:  low     |   Milestone:  0.6     
Component:  Other   |     Version:  0.5.2   
 Severity:  minor   |    Keywords:          

 The scipy.stats.entropy(distA, distB) method has an incorrect formula, in
 that it returns the correct value, but negative instead of positive. This
 is minor, but annoying.

 The current implemented formula is:

 S = -sum(pk * log(pk / qk), axis=0)

 The actual formula should be:

 S = sum(pk * log(pk / qk), axis=0)

 Indeed, the returned value should never be negative, as this method is

 See http://en.wikipedia.org/wiki/Kullback-Leibler_divergence for

Ticket URL: <http://scipy.org/scipy/scipy/ticket/497>
SciPy <http://www.scipy.org/>
SciPy is open-source software for mathematics, science, and engineering.

More information about the Scipy-tickets mailing list