[SciPy-User] scipy.linalg.solve()'s overwrite option does not work

braingateway braingateway@gmail....
Sat Nov 6 17:18:15 CDT 2010


josef.pktd@gmail.com :
> On Sat, Nov 6, 2010 at 5:46 PM, braingateway <braingateway@gmail.com> wrote:
>   
>> Joe Kington :
>>     
>>> On Sat, Nov 6, 2010 at 12:13 PM, braingateway <braingateway@gmail.com
>>> <mailto:braingateway@gmail.com>> wrote:
>>>
>>>     David Warde-Farley:
>>>     > On 2010-11-05, at 9:21 PM, braingateway wrote:
>>>     >
>>>     >
>>>     >> Hi everyone,
>>>     >> I believe the overwrite option is used for reduce memory usage.
>>>     But I
>>>     >> did following test, and find out it does not work at all. Maybe I
>>>     >> misunderstood the purpose of overwrite option. If anybody could
>>>     explain
>>>     >> this, I shall highly appreciate your help.
>>>     >>
>>>     >
>>>     > First of all, this is a SciPy issue, so please don't crosspost
>>>     to NumPy-discussion.
>>>     >
>>>     >
>>>     >>>>> a=npy.random.randn(20,20)
>>>     >>>>> x=npy.random.randn(20,4)
>>>     >>>>> a=npy.matrix(a)
>>>     >>>>> x=npy.matrix(x)
>>>     >>>>> b=a*x
>>>     >>>>> import scipy.linalg as sla
>>>     >>>>> a0=npy.matrix(a)
>>>     >>>>> a is a0
>>>     >>>>>
>>>     >> False
>>>     >>
>>>     >>>>> b0=npy.matrix(b)
>>>     >>>>> b is b0
>>>     >>>>>
>>>     >> False
>>>     >>
>>>     >
>>>     > You shouldn't use 'is' to compare arrays unless you mean to
>>>     compare them by object identity. Use all(b == b0) to compare by value.
>>>     >
>>>     > David
>>>     >
>>>     >
>>>     Thanks for reply, but I have to say u did not understand my post
>>>     at all.
>>>     I did this 'is' comparison on purpose, because I wanna know if the
>>>     overwrite flag is work or not.
>>>     See following example:
>>>      >>> a=numpy.matrix([0,0,1])
>>>      >>> a
>>>     matrix([[0, 0, 1]])
>>>      >>> a0=a
>>>      >>> a0 is a
>>>     True
>>>
>>>
>>> Just because two ndarray objects aren't the same doesn't mean that
>>> they don't share the same memory...
>>>
>>> Consider this:
>>> import numpy as np
>>> x = np.arange(10)
>>> y = x.T
>>> x is y # --> Yields False
>>> Nonetheless, x and y share the same data, and storing y doesn't double
>>> the amount of memory used, as it's effectively just a pointer to the
>>> same memory as x
>>>
>>> Instead of using "is", you should use "numpy.may_share_memory(x, y)"
>>>       
>> Thanks a lot for pointing this out! I were struggling to figure out
>> whether the different objects share memory or not. And good to know
>> a0=numpy.matrix(a) actually did not share the memory.
>>  >>> print 'a0 shares memory with a?', npy.may_share_memory(a,a0)
>> a0 shares memory with a? False
>>  >>> print 'b0 shares memory with b?', npy.may_share_memory(b,b0)
>> b0 shares memory with b? False
>> I also heard that even may_share_memory is 'True', does not necessarily
>> mean they share any element. Maybe, is 'a0.base is a' usually more
>> suitable for this purpose?
>>
>> Back to the original question: is there anyone actually saw the
>> overwrite_a or overwrite_b really showed its effect?
>> If you could show me a repeatable example, not only for
>> scipy.linalg.solve(), it can also be other functions, who provide this
>> option, such as eig(). If it does not show any advantage in memory
>> usage, I might still using numpy.linalg.
>>     
>
>
> import numpy as np
>
> a=np.random.randn(20,20)
> abak = a.copy()
> x=np.random.randn(20,4)
> xbak = x.copy()
> a=np.matrix(a)
> x=np.matrix(x)
> b=a*x
> b = np.array(b)
> bbak = b.copy()
>
> import scipy.linalg as sla
> a0=np.matrix(a)
> print a is a0
> #False
> b0=np.matrix(b)
> print b is b0
> #False
> X=sla.solve(a,b,overwrite_a=True,debug=True)
> print X is b
> #False
> print (X==b).all()
> #False
> print 'a:', (a0==a).all(), (abak==a).all()
> print 'b:', (b0==b).all(), (bbak==b).all()
> #
> Y = sla.solve(a,b,overwrite_a=True,overwrite_b=True,debug=True)
> print 'a:', (a0==a).all(), (abak==a).all()
> print 'b:', (b0==b).all(), (bbak==b).all()
>
> print (X==Y).all()
>
> printout
> -----------
> False
> False
> solve:overwrite_a= True
> solve:overwrite_b= False
> False
> False
> a: False False
> b: True True
> solve:overwrite_a= True
> solve:overwrite_b= True
> a: False False
> b: True True
> False
>
>   
Thanks a lot! So right now I see it might be some bugs in my Scipy 
version!, After running your code, I got following result:
 >>> sla.__version__
'0.4.9'
 >>>
False
False
solve:overwrite_a= True
solve:overwrite_b= False
False
False
a: True True
b: True True
solve:overwrite_a= True
solve:overwrite_b= True
a: True True
b: True True
True
> The first solve overwrites a, the second solve solves a different
> problem and the solutions X and Y are not the same.
> (if the first solve allows overwriting of be instead, then X==Y )
>
> I never got a case with overwritten b, and as you said there was no
> sharing memory with the original array, the copy has always the same
> result as your original.
>
> This was for quick playing with your example, no guarantee on no mistakes.
>
> Josef
>
>
>   
>>>     This means a0 and a is actually point to a same object. Then a0 act
>>>     similar to the C pointer of a.
>>>     I compared a0/b0 and a/b by 'is' first to show I did create a new
>>>     object
>>>     from the original matrix, so the following (a0==a).all()
>>>     comparison can
>>>     actually prove the values inside the a and b were not overwritten.
>>>
>>>     Sincerely,
>>>     LittleBigBrain
>>>     > _______________________________________________
>>>     > SciPy-User mailing list
>>>     > SciPy-User@scipy.org <mailto:SciPy-User@scipy.org>
>>>     > http://mail.scipy.org/mailman/listinfo/scipy-user
>>>     >
>>>
>>>     _______________________________________________
>>>     SciPy-User mailing list
>>>     SciPy-User@scipy.org <mailto:SciPy-User@scipy.org>
>>>     http://mail.scipy.org/mailman/listinfo/scipy-user
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> SciPy-User@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>
>>>       
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>     
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>   



More information about the SciPy-User mailing list