[SciPy-dev] Thoughts on weave improvements
prabhu at aero.iitm.ernet.in
Mon Feb 11 11:46:40 CST 2002
>>>>> "PM" == Pat Miller <pnmiller at pacbell.net> writes:
PM> Prabhu Ramachandran wrote:
>> All this sounds very promising. I just hope it becomes part of
PM> Guess what.... It works (some restrictions and caveats of
PM> course :-)
PM> # Get one of those nice weave module builders M =
PM> # Define some handy python function def f(x): "doc strings get
PM> propagated" return x*x
PM> # Here's where we define the accelerated function # Note that
PM> you need to give the input signature # of the function
PM> E.add_function( weaver.Python2CXX( f, [FloatType] ) )
I guess you need to do E = W somewhere. Anyway, I get the idea.
PM> # OK... It isn't everything, but it can make a lot of simple
PM> stuff run # about 2 to 5 times faster (real speed mostly eaten
PM> up with function call # overhead for these really small
Yes, this is very cool!
PM> Here are things that don't work...
PM> Still, not too shabby for a day and a half development time.
Thats real impressive indeed!
PM> I'll be patching up holes and writing some more extensive
PM> tests for this. There are three main enhancements that will
PM> get this kicking butt...
PM> 4) Build an interface to the struct module so that you can
PM> build and use real C structs
Does this mean that you can actually accelerate a full fledged Python
PM> My goal here, in case it isn't obvious, is to allow users to
PM> write modules completely in Python, debug and develop in a
PM> great language, and then recover the speed through
PM> compilation. This has a side benefit in that Jython and
PM> PalmPilot-thon and Embedded-processor-thon can all use the
PM> Python variant of your code and if you run on a "real" machine
PM> you get the speed.
Yes, it sure would be a *very* nice goal. One thing I cant but help
noticing is that it looks like this is all heading towards something
like Dylan. Its like Python + type info + magical parser that parses
functions and produces c/c++ extensions.
I ask a naive question, what is the feasibility of actually doing
something like Dylan? OK forget truly generic functions. I'm just
talking of optional typing to speed things up. I guess psyco, weave,
pyCOD etc. all put together and taken to their logical end will give
us Python + optional typing + heaven? :D Of course I could also be
*completely* mistaken. Looking at Eric's subsequent post it looks
like even optional typing can be magically handled. This is looking
How far can we go? What would be possible and what is simply out of
question?? If only a small subset of Python features are used by high
perf. Python code, what subset are we talking about?? Wrapping a
function is fine. What about classes? Would that be possible. I
know maybe I'm wishing for too much but I think its important to
atleast understand what is possible.
PM> Hope this is useful stuff. If you want the code, drop me a
PM> line and I'll pass it along. I'm hoping that Eric will let me
PM> integrate this directly with weave. I think it is good enough
I'm sure Eric will be very happy to let you integrate this with weave
as will a lot of other users!
PM> to use out of the box (since it doesn't barf at every
PM> opportunity), but it will take some shaking out to get it to
PM> really work and this can only happen with users! Let me know
PM> who you are...
Well, count me in but I am busy with too many things these days but I
will try to experiment with it when I have the time.
More information about the Scipy-dev