[Numpy-discussion] genfromtxt converter question

Bruce Southey bsouthey@gmail....
Fri Jun 17 09:31:37 CDT 2011


On 06/17/2011 08:51 AM, Olivier Delalleau wrote:
>
>
> 2011/6/17 Bruce Southey <bsouthey@gmail.com <mailto:bsouthey@gmail.com>>
>
>     On 06/17/2011 08:22 AM, gary ruben wrote:
>     > Thanks Olivier,
>     > Your suggestion gets me a little closer to what I want, but doesn't
>     > quite work. Replacing the conversion with
>     >
>     > c = lambda x:np.cast[np.complex64](complex(*eval(x)))
>     > b = np.genfromtxt(a,converters={0:c, 1:c, 2:c,
>     > 3:c},dtype=None,delimiter=18,usecols=range(4))
>     >
>     > produces
>     >
>     > [[(-3.97000002861-5.03999996185j) (-1.1318000555-2.56929993629j)
>     >   (-4.60270023346-0.142599999905j) (-1.42490005493+1.73300004005j)]
>     >   [(-5.4797000885+0j) (1.85850000381-1.5501999855j)
>     >   (4.41450023651-0.763800024986j) (-0.480500012636-1.19760000706j)]
>     >   [0j (6.26730012894+0j) (-0.45039999485-0.0289999991655j)
>     >   (-1.34669995308+1.65789997578j)]
>     >   [0j 0j (-3.5+0j) (2.56189990044-3.37080001831j)]]
>     >
>     > which is not yet an array of complex numbers. It seems close to the
>     > solution though.
>     >
>     > Gary
>     >
>     > On Fri, Jun 17, 2011 at 8:40 PM, Olivier Delalleau<shish@keba.be
>     <mailto:shish@keba.be>>  wrote:
>     >> If I understand correctly, your error is that you convert only
>     the second
>     >> column, because your converters dictionary contains a single
>     key (1).
>     >> If you have it contain keys from 0 to 3 associated to the same
>     function, it
>     >> should work.
>     >>
>     >> -=- Olivier
>     >>
>     >> 2011/6/17 gary ruben<gruben@bigpond.net.au
>     <mailto:gruben@bigpond.net.au>>
>     >>> I'm trying to read a file containing data formatted as in the
>     >>> following example using genfromtxt and I'm doing something
>     wrong. It
>     >>> almost works. Can someone point out my error, or suggest a simpler
>     >>> solution to the ugly converter function? I thought I'd leave
>     in the
>     >>> commented-out line for future reference, which I thought was a
>     neat
>     >>> way to get genfromtxt to show what it is trying to pass to the
>     >>> converter.
>     >>>
>     >>> import numpy as np
>     >>> from StringIO import StringIO
>     >>>
>     >>> a = StringIO('''\
>     >>>   (-3.9700,-5.0400) (-1.1318,-2.5693) (-4.6027,-0.1426)
>     (-1.4249, 1.7330)
>     >>>   (-5.4797, 0.0000) ( 1.8585,-1.5502) ( 4.4145,-0.7638)
>     (-0.4805,-1.1976)
>     >>>   ( 0.0000, 0.0000) ( 6.2673, 0.0000) (-0.4504,-0.0290)
>     (-1.3467, 1.6579)
>     >>>   ( 0.0000, 0.0000) ( 0.0000, 0.0000) (-3.5000, 0.0000) (
>     2.5619,-3.3708)
>     >>> ''')
>     >>>
>     >>> #~ b = np.genfromtxt(a,converters={1:lambda
>     >>> x:str(x)},dtype=object,delimiter=18)
>     >>> b = np.genfromtxt(a,converters={1:lambda
>     >>> x:complex(*eval(x))},dtype=None,delimiter=18,usecols=range(4))
>     >>>
>     >>> print b
>     >>>
>     >>> --
>     >>>
>     >>> This produces
>     >>> [ (' (-3.9700,-5.0400)', (-1.1318-2.5693j), '
>     (-4.6027,-0.1426)', '
>     >>> (-1.4249, 1.7330)')
>     >>>   (' (-5.4797, 0.0000)', (1.8585-1.5502j), ' ( 4.4145,-0.7638)', '
>     >>> (-0.4805,-1.1976)')
>     >>>   (' ( 0.0000, 0.0000)', (6.2673+0j), ' (-0.4504,-0.0290)', '
>     (-1.3467,
>     >>> 1.6579)')
>     >>>   (' ( 0.0000, 0.0000)', 0j, ' (-3.5000, 0.0000)', ' (
>     2.5619,-3.3708)')]
>     >>>
>     >>> which I just need to unpack into a 4x4 array, but I get an
>     error if I
>     >>> try to apply a different view.
>     >>>
>     >>> thanks,
>     >>> Gary
>     >>> _______________________________________________
>     >>> NumPy-Discussion mailing list
>     >>> NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>     >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>     >>
>     >> _______________________________________________
>     >> NumPy-Discussion mailing list
>     >> NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>     >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>     >>
>     >>
>     > _______________________________________________
>     > NumPy-Discussion mailing list
>     > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>     > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>     Just an observation for the StringIO object, you have multiple spaces
>     within the parentheses but, by default, you are using whitespace
>     delimiters in genfromtxt. So, yes, genfromtxt is going have issues.
>
>     If you can rewrite the input, then you need a non-space and non-comma
>     delimiter then specify that delimiter to genfromtxt. Otherwise you are
>     probably going to have to write you own parser - for each line,
>     split on
>     ' (' etc.
>
>     Bruce
>
>
> It's funny though because that part (the parsing) actually seems to work.
>
> However I've been playing a bit with his example and indeed I can't 
> get numpy to return a complex array. It keeps resulting in an "object" 
> dtype.
> The only way I found was to convert it temporarily into a list to 
> recast it in complex64, adding the line:
>   b = np.array(map(list, b), dtype=np.complex64)
>
> -=- Olivier
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Yes, my mistake as I did not see that the delimiter was set to 18.
The problem is about the conversion into complex which I do not think is 
the converter per se. So a brute force way is:

b= np.genfromtxt(a, dtype=str,delimiter=18)
nrow, ncol=b.shape
print b
d=np.empty((nrow, ncol), dtype=np.complex)
for row in range(nrow):
     for col in range(ncol):
         d[row][col]=complex(*eval(b[row][col]))
print d


Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110617/5921c505/attachment.html 


More information about the NumPy-Discussion mailing list