[Numpy-discussion] newbie question - large dataset
Steve Staneff
staneff@constructiondatares....
Sat Apr 7 11:31:37 CDT 2007
Hi,
I'm looking for a better solution to managing a very large calculation.
Set A is composed of tuples a, each of the form a = [float, string]; set B
is composed of tuples of similar structure (b = [float, string]). For
each possible combination of a and b I'm calculating c, of the form c =
f(a,b) = [g(a[0], b[0]), h(a[1], b[1])] where g() and h() are non-trivial
functions.
There are now 15,000 or more tuples in A, and 100,000 or more tuples in B.
B is expected to grow with time as the source database grows. In
addition, there are many more elements in a and b than I've stated (and
many more functions operating on them). I'm currently using python to
loop through each a in A and each b in B, which takes days.
If anyone can point me to a better approach via numpy ( or anything
else!), I'd be very appreciative.
Thanks in advance,
Steve
More information about the Numpy-discussion
mailing list