[Numpy-discussion] Precision difference between dot and sum

Joon groups.and.lists@gmail....
Mon Nov 1 21:39:16 CDT 2010


Thanks for the replies.

I tried several stuff like changing dot into sum in the gradient  
calculations just to see how they change the results, but it seems that  
part of the code is the only place where the results get affected by the  
choice of dot/sum.

I am using 64bit machine and EPD python (I think it uses Intel MKL) so  
that could have affected the calculation. I will use dot whenever possible  
 from now on. :)

-Joon



On Mon, 01 Nov 2010 21:27:07 -0500, David <david@silveregg.co.jp> wrote:

> On 11/02/2010 08:30 AM, Joon wrote:
>> Hi,
>>
>> I just found that using dot instead of sum in numpy gives me better
>> results in terms of precision loss. For example, I optimized a function
>> with scipy.optimize.fmin_bfgs. For the return value for the function, I
>> tried the following two things:
>>
>> sum(Xb) - sum(denominator)
>>
>> and
>>
>> dot(ones(Xb.shape), Xb) - dot(ones(denominator.shape), denominator)
>>
>> Both of them are supposed to yield the same thing. But the first one
>> gave me -589112.30492110562 and the second one gave me 
>> -589112.30492110678.
>
> Those are basically the same number: the minimal spacing between two
> double floats at this amplitude is ~ 1e-10 (given by the function
> np.spacing(the_number)), which is essentially the order of magnitude of
> the difference between your two numbers.
>
>> I was wondering if this is well-known fact and I'm supposed to use dot
>> instead of sum whenever possible.
>
> You should use dot instead of sum when application, but for speed
> reasons, essentially.
>
>>
>> It would be great if someone could let me know why this happens.
>
> They don't use the same implementation, so such tiny differences are
> expected - having exactly the same solution would have been surprising,
> actually. You may be surprised about the difference for such a trivial
> operation, but keep in mind that dot is implemented with highly
> optimized CPU instructions (that is if you use ATLAS or similar library).
>
> cheers,
>
> David
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20101101/d426054f/attachment.html 


More information about the NumPy-Discussion mailing list