[SciPy-user] Making faster statistical distributions

Christopher Fonnesbeck chris at fonnesbeck.org
Thu Jan 29 20:05:41 CST 2004


On Jan 29, 2004, at 8:17 PM, Travis Oliphant wrote:

> Robert Kern wrote:
>
>> On Thu, Jan 29, 2004 at 03:53:10PM -0500, Christopher Fonnesbeck 
>> wrote:
>>
>> [snip]
>>
>>
>>> Well, the binomial and normal distributions, for sure, off the top 
>>> of my head. Using the scipy distributions slows my MCMC code down 
>>> significantly (they were the bottleneck, according to the profiling 
>>> module). Using Fortran via f2py sped things up a lot. I'm not 
>>> talking about the generation of random deviates, necessarily, but 
>>> rather the pdf's, which are used for calculating likelihoods.
>>>
>>
>> I'm willing to bet that the speed is lost by the generality of the
>> framework. There's a lot of argument manipulation and not a lot of
>> actual computation. It might be useful to add a fastdists module that
>> computes pdfs and cdfs in a more specific (but fast) way for the more
>> common distributions. And log-pdfs, too, please! In fact, log-pdfs
>> first since in many instances like MCMC, they're much more suitable.
>>
>> One could then override pdf() and cdf() (not _pdf() and _cdf()) in the
>> specific classes. But that's a design that appears to have been 
>> decided
>> against.
>>
>
> I wouldn't say anything has been decided against at this point.   We 
> could pretty easily over-ride pdf and cdf for whatevern distribution 
> you like.  Perhaps it would be a good idea to do so for the most 
> common pdfs.
> -Travis O.

Perhaps I will take a shot at this, then, since I already have several 
Fortran extensions written for log-likelihoods.

C.

--
Christopher J. Fonnesbeck ( c h r i s @ f o n n e s b e c k . o r g )
Georgia Cooperative Fish & Wildlife Research Unit, University of Georgia



More information about the SciPy-user mailing list