[SciPy-Dev] Question about subpackage/submodule API
Tue Mar 8 23:17:58 CST 2011
On Sun, Feb 20, 2011 at 3:33 PM, Ralf Gommers
> On Tue, Feb 15, 2011 at 7:53 AM, Warren Weckesser
> <firstname.lastname@example.org> wrote:
>> On Sat, Feb 12, 2011 at 7:28 PM, Ralf Gommers <email@example.com>
>>> On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <firstname.lastname@example.org> wrote:
>>>> One wild idea to make this clearer could be to prefix all internal sub-
>>>> package names with the usual '_'. In the long run, it probably wouldn't
>>>> be as bad as it initially sounds like.
>>> This is not a wild idea at all, I think it should be done. I considered
>>> all modules without '_' prefix public API.
>> Agreed (despite what I said in my initial post).
>> To actually do this, we'll need to check which packages have modules that
>> should be private. These can be renamed in 0.10 to have an underscore, and
>> new public versions created that contain a deprecation warning and that
>> import everything from the private version. The deprecated public modules
>> can be removed in 0.11.
>> Some modules will require almost no changes. For example, scipy.cluster
>> *only* exposes two modules, vq and hierarchy, so no changes are needed.
>> (Well, there is also the module info.py that all packages have. That should
>> become _info.py--there's no need for that to be public, is there?)
> Agreed, rename to _info.py
This can't actually be done very easily, because the info.py name is
hardcoded in PackageLoader in numpy._import_tools.py. So if desired,
it first has to be done in numpy.
>> Other packages will probably require some discussion about what modules should be
>> Consider the above a proposed change for 0.10 and 0.11--what do you think?
> Sounds good. Attached is a file that goes through scipy sub-packages
> and checks their __all__ for modules. Those are public by definition
> (but this doesn't give you the whole API). It's pretty messy, for
> That should be cleaned up. Then there are also public modules that
> don't show up of course (for example odr.models).
> How about doing the following? :
> 1. Start a doc, perhaps on the wiki, with a full list of public modules.
> 2. Put that doc at the beginning of the reference guide, as well as
> the relevant part in the docstring for each sub-package.
> 3. Clean up existing __all__, and add __all__ to sub-packages that
> don't have them yet.
> 4. Rename private modules, with suitable deprecation warning.
I've done the uncontroversial part (3):
This cleans up the sub-package namespaces quite a bit, which will also
make for example tab-completion in IPython easier to use. For example,
sp.signal.<TAB> gives now 147 results instead of 262.
There is one thing I wasn't sure about: should
arccos/arccosh/arcsinh/... stay exposed in the scipy.special
namespace, even though they are numpy functions?
Here is a complete list of modules that I think are part (or should be
part) of the public API. I added modules because they are documented
as being public in docs, or contain useful functions/objects that are
not exposed one level up, or because the sub-package namespace is very
large and could benefit from a subdivision.
<deprecate this module>
<add more after refactoring, Josef wants to break up module. I agree
it's too large and slow to import.>
<didn't look at weave yet>
More information about the SciPy-Dev