[Numpy-discussion] Log Arrays

Robert Kern robert.kern@gmail....
Thu May 8 11:56:00 CDT 2008


On Thu, May 8, 2008 at 11:25 AM, Charles R Harris
<charlesr.harris@gmail.com> wrote:
>
> On Thu, May 8, 2008 at 10:11 AM, Anne Archibald <peridot.faceted@gmail.com>
> wrote:
>>
>> 2008/5/8 Charles R Harris <charlesr.harris@gmail.com>:
>> >
>> > What realistic probability is in the range exp(-1000) ?
>>
>> Well, I ran into it while doing a maximum-likelihood fit - my early
>> guesses had exceedingly low probabilities, but I needed to know which
>> way the probabilities were increasing.
>
> The number of bosons in the universe is only on the order of 1e-42.
> Exp(-1000) may be convenient, but as a probability it is a delusion. The
> hypothesis "none of the above" would have a much larger prior.

When you're running an optimizer over a PDF, you will be stuck in the
region of exp(-1000) for a substantial amount of time before you get
to the peak. If you don't use the log representation, you will never
get to the peak because all of the gradient information is lost to
floating point error. You can consult any book on computational
statistics for many more examples. This is a long-established best
practice in statistics.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco


More information about the Numpy-discussion mailing list