# [Numpy-discussion] fromiter shape argument -- was Re: For loop tips

Torgil Svensson torgil.svensson at gmail.com
Thu Aug 31 13:21:50 CDT 2006

```> Yes. fromiter(iterable, dtype, count) works.

Oh. Thanks. I probably had too old documentation to see this (15
at least until 1.0 is released :)

> Regardless, L is only iterated over once.

How can this be true? If no size is given, mustn't numpy either loop
over L twice or build an internal representation on which it'll
iterate or copy in chunks?

I just found out that this works
>>> import numpy,itertools
>>> rec_dt=numpy.dtype(">i4,S10,f8")
>>> rec_iter=itertools.cycle([(1,'s',4.0),(5,'y',190.0),(2,'h',-8)])
>>> numpy.fromiter(rec_iter,rec_dt,10).view(recarray)
recarray([(1, 's', 4.0), (5, 'y', 190.0), (2, 'h', -8.0), (1, 's', 4.0),
(5, 'y', 190.0), (2, 'h', -8.0), (1, 's', 4.0), (5, 'y', 190.0),
(2, 'h', -8.0), (1, 's', 4.0)],
dtype=[('f0', '>i4'), ('f1', '|S10'), ('f2', '<f8')])

but what's wrong with this?

>>> d2_dt=numpy.dtype("4f8")
>>> d2_iter=itertools.cycle([(1.0,numpy.nan,-1e10,14.0)])
>>> numpy.fromiter(d2_iter,d2_dt,10)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: a float is required
>>> numpy.__version__
'1.0b4'

//Torgil

On 8/30/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> Torgil Svensson wrote:
> >>    return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))
> >>
> >
> > Is it possible for fromiter to take an optional shape (or count)
> > argument in addition to the dtype argument?
> Yes. fromiter(iterable, dtype, count) works.
>
> > If both is given it could
> > preallocate memory and we only have to iterate over L once.
> >
> Regardless, L is only iterated over once. In general you can't rewind
> iterators, so that's a requirement. This is accomplished by doing
> successive overallocation similar to the way appending to a list is
> handled. By specifying the count up front you save a bunch of reallocs,
> but no iteration.
>
> -tim
>
>
>
> > //Torgil
> >
> > On 8/29/06, Keith Goodman <kwgoodman at gmail.com> wrote:
> >
> >> On 8/29/06, Torgil Svensson <torgil.svensson at gmail.com> wrote:
> >>
> >>> something like this?
> >>>
> >>> def list2index(L):
> >>>    uL=sorted(set(L))
> >>>    idx=dict((y,x) for x,y in enumerate(uL))
> >>>    return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))
> >>>
> >> Wow. That's amazing. Thank you.
> >>
> >> -------------------------------------------------------------------------
> >> Using Tomcat but need to do more? Need to support web services, security?
> >> Get stuff done quickly with pre-integrated technology to make your job easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >> _______________________________________________
> >> Numpy-discussion mailing list
> >> Numpy-discussion at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>
> >>
> >
> > -------------------------------------------------------------------------
> > Using Tomcat but need to do more? Need to support web services, security?
> > Get stuff done quickly with pre-integrated technology to make your job easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
> >
> >
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier