[SciPy-User] Least-squares fittings with bounds: why is scipy not up to the task?

Matthew Newville matt.newville@gmail....
Fri Mar 9 15:13:35 CST 2012


Hi Gael,

On Friday, March 9, 2012 10:54:43 AM UTC-6, Gael Varoquaux wrote:
>
> Hi Matt,
>
> I am not going to answer to the core of your message. I partly agree with
> it and partly disagree. I think that it is fair to have different points
> of view. In addition, I do share the opinion that the situation of
> developers in open source scientific software is not ideal. I've suffered
> from it personally.
>
> I just want to react to a couple of minor points
>
> >    At no point has anyone from the scipy team expressed an interest in
> >    putting this into scipy.
>
> Who is the scipy team? What is the scipy team? Who could or should
> express such an interest? These are people struggling to maintain a
> massive package on their free time. It actually takes a lot of time to
> monitor the mailing lists and pick up offers like yours to turn them into
> something that can be integrated.
>
OK, that's all fair.  I am certainly not a scipy dev.   But you do seem to 
want it both ways: chastise people for asking for features instead of 
writing those features themselves, and also saying you're far too busy to 
read and act on submissions of code unless it conforms exactly to your 
needs (this year, with github!).   I would guess that this is not likely to 
work very well for you.

For myself, I might be willing to contribute to scipy, but I have my own 
projects and mailing lists to worry about.   Lmfit was something of a fun 
side-project for me -- I'm willing to support it, and can see that it 
*could* go into scipy.   Since you originally brought it up, I did 
essentially everything in Fernando Perez's slides pleaded for how to 
contribute (except submit a pull request, but I did ask whether this was 
viewed as something that should be a scikit, or go into scipy, or retained 
as standalone). 
 

> Had you submitted a pull request, with code ready to be merged, i.e. with
> no extra work in terms of documentation, API or tests, I think that it
> would be legitimate to blame the scipy developers for lack of interest.
> That said, I can easily understand how such a pull request would fall
> between the cracks. It's unfortunate, not excusable, but it does happen.
> Indeed, in the projects I maintain, I am kept busy full time with pure
> maintenance work (bug fixing, answering emails, improving documentation).
> When I review and merge pull requests, a lot of the time they are for
> features that I do not need, and I spend full week ends adding tests,
> fixing numerical instabilities, completing the docs so that they can be
> merged. You have to realize that most contributions to open source
> projects actually add up to the workload of the core developers.
> Thankfully, not all of them. Teams do build upon people unexpectedly
> fixing bugs, contributing flawless code that can be merged in without any
> additional work.
>
Well, I understand you don't necessarily consider yourself to be a scipy 
dev, but I'll ask anyway:  How many pull requests have been made, and how 
many have been accepted?  Perhaps I am not reading github's Pull Request 
link correctly, but that seems to indicate that the numbers are 17 and 0.  
Surely, those cannot be correct.      Unless I am counting wrong, 5 of the 
17 (total? outstanding?) pull requests listed for scipy/scipy
involve optimization. 
  

> I personally have seen my time invested in maintenance of open source
> project go up and up for the last few years, until it was to a point
> where I was spending a major part of my free time on it. It ended up
> giving me a nasty back pain, and I started not answering bug reports,
> pull requests and support emails to preserve my health: it is not sane to
> spend all onces time in front of a computer.
>
> >    There are many kinds of skills.  Sometimes, not insulting your 
> customers,
> >    colleagues, and potential collaborators is the most important one.
>
> Maybe I went over the top. I didn't want to sound insulting. I felt
> insulted, as an open source develop (even thought I am not a scipy
> developer). I am sorry that I ignited a flame. Getting worked out about
> email is never a good thing, and discussion pushing blame certainly don't
> help building a community. Maybe I shouldn't have sent this email, or I
> should have worded it differently. I apologize for the harsh tone. I
> certainly did feel bad when I received the original email, and I wanted
> to express it.
>
It is easy to feel like one's hard work on an open source project is 
under-appreciated.  I understand that feeling, and have felt that way 
myself in the past.   I am definitely quite appreciative of the work done 
on scipy and friends.
 

> >    For myself, I find it quite discouraging that the scipy team is so
> >    insular.
>
> Firstly, I would like to stress that I cannot consider myself as part of
> the scipy team. I contribute very little code to scipy. As a consequence
> I do not feel that I have much legitimacy in making decisions or
> comments on the codebase. Thus you shouldn't take my reaction as a
> reaction coming from the scipy team, but rather as coming from myself.
>
> Second, can I ask you what makes you think that the scipy team is
> insular? Scipy is a big project with a lot of history. As such it is
> harder to contribute to it than a small and light project. But I don't
> feel any dogmatism or clique attitude from the developers. And, by the
> way, if we are going to talk about the scipy developers, 
>

Well, there was a discussion "Alternative to scipy.optimize" in the past 
two weeks on scipy-users that mentioned lmfit, and several over the past 
several months, and apparently requests about mpfit in the more distant 
past.  And yet two scipy contributers (according to github's list, I am 
counting you) responded to the original request for features **exactly like 
lmfit** with something reading an awful lot like "Well, you'll have to 
write one".    Perhaps insular is not a fair characterization -- how would 
you characterize that?

I encourage everybody to find out who they are, i.e. who has been 
> contributing lately
> [1]. I don't think that the handful of people that come on top of the
> list have an insular behavior. I do think that they are on an island, in
> the sens that they are pretty much left alone to do the grunt work. None
> of these people reacted badly to any mail on this mailing list about the
> state of scipy. I raise my hat to them!
>
And so do I.
  
--Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20120309/fbd36d8b/attachment.html 


More information about the SciPy-User mailing list