[SciPy-User] understanding machine precision

Dag Sverre Seljebotn dagss@student.matnat.uio...
Sat Dec 18 03:10:05 CST 2010


Francesc Alted wrote:
> A Wednesday 15 December 2010 13:36:44 josef.pktd@gmail.com escrigué:
>   
>> On Tue, Dec 14, 2010 at 2:59 PM, Bruce Southey <bsouthey@gmail.com> 
>>     
> wrote:
>   
>>> On 12/14/2010 01:29 PM, Francesc Alted wrote:
>>>       
>>>> A Tuesday 14 December 2010 20:17:10 Nils Wagner escrigué:
>>>>         
>>>>> Hi all,
>>>>>
>>>>> I am using ATLAS
>>>>>
>>>>> python -i try_deterministic.py
>>>>>
>>>>> lstsq
>>>>> 0.0
>>>>> 0.0
>>>>> 0.0
>>>>> 0.0
>>>>> 0.0
>>>>>           
>>>> That's interesting.  Maybe Josef is using a threaded ATLAS?  I
>>>> positively know that threading introduces variability in the order
>>>> that the computations are done.  However, I'm not sure on why
>>>> ATLAS has decided to use several threads for so small matrices
>>>> ((100, 10)) :-/
>>>>         
>>> Does this 'issue' occur with numpy's lstsq?
>>>       
>> good question, I didn't think about it.
>>
>> same problem and my numpy is from an official installer (1.4.0) and
>> not compiled against the old ATLAS, that I'm still using.
>>     
>
> With the slightly modified script that depends only on numpy, I'm having 
> problems of reproducibility only with pinv:
>
> C:\Users\francesc>python x:\try_deterministic_numpy.py
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Python version:    2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 
> 32 bit (Intel)]
> NumPy version:     1.5.0
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>
> lstsq
> 0.0
> 0.0
> 0.0
> 0.0
> 0.0
>
> pinv
> 0.0
> 5.55111512313e-17
> 0.0
> 0.0
> 0.0
>
> And, curiously enough, that pattern is always the same for every run.  
> So, at least, the issue is more deterministic that we initially thought.
>
> But, in the same machine, using Python 2.7, problem disappears 
> completely:
>
> C:\Users\francesc>python x:\try_deterministic_numpy.py
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Python version:    2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 
> bit (Intel)]
> NumPy version:     1.5.0
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>
> lstsq
> 0.0
> 0.0
> 0.0
> 0.0
> 0.0
>
> pinv
> 0.0
> 0.0
> 0.0
> 0.0
> 0.0
>
> Well, at least we have narrowed the possibilities significantly: this 
> seems to happen only (up to now) with Win32, Python 2.6 and NumPy 
> (without ATLAS in the loop).  It is worth noting that the compilers used 
> for Python 2.6 and 2.7 should be the same (MSVC 1500).  I think I 
> installed NumPy from sourceforge repository, so the compiler used here 
> should also be the same.
>
> And also interesting is the fact that the 'lstsq' method is always 
> reproducible on my machine.  However, Josef also reported non-
> deterministic behaviour with 'lstsq' :-/
>   

How about memory alignment? It may be that
a) the matrix has to be copied to new memory before passing it to LAPACK,
b) you get different results depending on whether the array is allocated 
on 128-bit boundary or not

One way to test is to stick "aligned16" (or was that "align16"?) into 
generic_flapack.pyf and rebuild SciPy (there's a couple of aligned8 in 
there you can look at). Or, see if it matters if you make your array 
Fortran contiguous.

Dag Sverre


More information about the SciPy-User mailing list