[Numpy-discussion] ValueError: objects are not aligned
Bruce Southey
bsouthey@gmail....
Wed Sep 8 08:39:56 CDT 2010
On 09/08/2010 08:30 AM, josef.pktd@gmail.com wrote:
> On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta<ruidc@yahoo.com> wrote:
>> Thanks again,
>>
>> I was trying to
>> follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> but with the key difference being, that instead of the data for the three
>> arrays being constructed from literals, that i'd be sourcing from a
>> relational database. The data is quite simple, in that I have 199 recorded
>> days for 8 different variables, but am just trying to do a proof-of-concept
>> first with three, when I had this problem.
>> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
>> =CORREL(rf1,rf1)
>> =CORREL(rf1,rf2)
>> =CORREL(rf1,rf3)
>> =CORREL(rf2,rf1)
>> =CORREL(rf2,rf2)
>> =CORREL(rf2,rf3)
>> =CORREL(rf3,rf1)
>> =CORREL(rf3,rf2)
>> =CORREL(rf3,rf3)
>> I tried to simplify and isolate the problem, and hence the confusion.
>> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
>> of the 8), each 199 rows in length.
>> From reading the lists and blogs, the way to put these into numpy seemed to
>> be:
>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
>> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
>> Then to try and produce a correlation matrix:
>> print (np.corrcoef([rf1, rf2, rf3]))
>> But this is where I get the original error.
>> From looking at column_stack, this doesn't seem like what I want to achieve,
>> or am I looking at this the wrong way somehow?.
> column_stack and rowvar=0 should work, but it might be easier to get
> rid of the extra dimension that the cursor gives you
>
> e.g.
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
>
> depends on whether you want to work with 2d or 1d data arrays
>
> Josef
>
>> Thanks again,
>> Rui
>>
>> ________________________________
>> From: "josef.pktd@gmail.com"<josef.pktd@gmail.com>
>> To: Discussion of Numerical Python<numpy-discussion@scipy.org>
>> Sent: Wed, September 8, 2010 2:34:42 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta<ruidc@yahoo.com> wrote:
>>> Thanks for your assistance,
>>> I was following this example:
>>>
>>> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> I didn't know,
>> If arrays are 1d, then numpy does the stacking for you., but it
>> doesn't work with column-vectors or 2d arrays.
>>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
>> array([[ 1., 1.],
>> [ 1., 1.]])
>>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
>> array([[ NaN, NaN, NaN],
>> [ NaN, NaN, NaN],
>> [ NaN, NaN, NaN]])
>>> I intend to use different arrays once I have resolved this problem.
>>> Trying your suggestion:
>>> ...
>>> print (rf1)
>>> print (np.corrcoef(rf1, rf1, rowvar=0))
>>> results in:
>>> [[-0.00641625]
>>> [-0.00498411]
>>> [-0.0038015 ]]
>>> [[ 1. 1.]
>>> [ 1. 1.]]
>>>
>>> which should have been a 3x3 matrix
>> I'm not sure what you want here, you only have three values
>> variance would also be zero, and corrcoef wouldn't be defined if you
>> only have 1 observation per variable
>>
>>>>> np.corrcoef(np.arange(3)[:,None])
>> array([[ NaN, NaN, NaN],
>> [ NaN, NaN, NaN],
>> [ NaN, NaN, NaN]])
>>
>>>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
>>>>> np.arange(5)**3]), rowvar=0)
>> array([[ 1. , 0.9589266 , 0.90588235],
>> [ 0.9589266 , 1. , 0.98713033],
>> [ 0.90588235, 0.98713033, 1. ]])
>>
>> examples are in docstring for np.cov
>>
>>
>>
>>
>>> as for "stacking rf1 rf1 into a single array first" how would I do this?
>>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>> [1, 1],
>> [2, 2]])
>>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required
>>>>> 2d
>> array([[0, 0],
>> [1, 1],
>> [2, 2]])
>>
>>>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
>> array([[0, 0],
>> [1, 1],
>> [2, 2]])
>>>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>> [1, 1],
>> [2, 2]])
>>
>>
>> Josef
>>
>>> Regards,
>>> Rui
>>> ________________________________
>>> From: "josef.pktd@gmail.com"<josef.pktd@gmail.com>
>>> To: Discussion of Numerical Python<numpy-discussion@scipy.org>
>>> Sent: Wed, September 8, 2010 1:09:46 PM
>>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>>
>>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC<ruidc@yahoo.com> wrote:
>>>> I'm getting this error, which must be a simple error in getting the
>>>> result
>>>> from the cursor in the right shape, how do I get the cursor into the
>>>> array
>>>> as a single dimension?:
>>>>
>>>> import numpy as np
>>>> import sqlite3
>>>> conn = sqlite3.connect("simpledb")
>>>> cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>>> print (rf1)
>>>> print (np.corrcoef([rf1, rf1]))
>>> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>>>
>>> try
>>> np.corrcoef(rf1, rf1, rowvar=0) no [ ]
>>> or
>>> stacking rf1 rf1 into a single array first
>>>
>>> Josef
>>>
>>>> result:
>>>> [[-0.00641625]
>>>> [-0.00498411]
>>>> [-0.0038015 ]]
>>>> Traceback (most recent call last):
>>>> File "C:\Documents and
>>>> Settings\rui\workspace\numpytest\src\numpytest.py",
>>>> line 38, in<module>
>>>> print (np.corrcoef([rf1, rf1]))
>>>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>>> 2003, in corrcoef
>>>> c = cov(x, y, rowvar, bias, ddof)
>>>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>>> 1953, in cov
>>>> return (dot(X, X.T.conj()) / fact).squeeze()
>>>> --
>>>> View this message in context:
>>>>
>>>> http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>>>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion@scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Your errors suggest that you are not getting what expect from the
database query. Please check the dtype and shape of your arrays created
from the database query. Then check the contents to ensure the values
make sense especially for unexpected values. You should be able to find
the mean and variance of each of your arrays.
Of course, providing the actual data from the query and/or the arrays is
also a plus.
Bruce
More information about the NumPy-Discussion
mailing list