[SciPy-user] Docstring standards for NumPy and SciPy

Edward Loper edloper at gradient.cis.upenn.edu
Wed Jan 17 10:46:26 CST 2007

>> First, I'd pretty strongly recommend against inventing your own   
>> markup language.  [...]
> I'm not really convinced by this argument.   I agree we shouldn't  
> be flippant about introducing new markup and that is not what I've  
> proposed.   But, already we must learn multiple markup.  For  
> example,  Moin Moin uses one way to describe tables and  
> restructured text.

> Basically, I've said that the existing markups do not seem geared  
> toward mathematical description (as latex must be hacked on to them).

This is certainly true -- the only markup language I know of for  
concisely representing math as text *is* latex.  But it doesn't sound  
like you're proposing doing anything different -- you just want to  
use $expr$ to hack latex into the markup, rather than  
using :math:`expr` or `expr`.

> In addition, I don't like the look of existing markup docstrings  
> --- especially in the parameter section.  That's where my biggest  
> problem really lies.   What should be clear ends up with all kinds  
> of un-necessary back-ticks.

As I mentioned in my previous email, that is *not* part of the  
definition of rst as a markup language; it's just a detail about how  
that markup's output (a document tree) gets processed by epydoc.  It  
would be fairly simple to change epydoc to accept parameter sections  
where the variables don't have to be marked by backticks; and based  
on the feedback here, I intend to do just that.

Responding to Alan Isaac's comment on the same topic, this change  
would *not* result in any ability to cross reference from the  
parameters.  But there's a separate issue of what meaning should be  
associated with the expression `foo`.  There's two possibilities: it  
can be used to mark cross-references; or it can be used to mark  
mathematical expressions.  Whichever of those it is *not* used for  
will need to be marked with some more explicit markup.  My preference  
is for `foo` to be used for cross-references, since they're used more  
often in general.  But I understand there's a push from scipy to make  
math easy to express.  (Although, as I said before, anything more  
than simple expressions will look like line-noise to the uninitiated,  
whether it's surrounded by $...$ or `...` or :math:`...`.)

> I also don't really like the extra ":" at the beginning of a  
> paragraph to denote a section.  I could live with the underline  
> though.

I don't think the ":" at the beginning detracts from readability, and  
it certainly doesn't take much work to write, so I guess this is  
primarily an aesthetic reaction..  My guess is that it wouldn't take  
you too long to get used to.  Does anyone other than Travis consider  
the use of eg ":Parameters:" instead of "Parameters:" to be a show- 
stopper for using rst?

> Basically, scipy docstrings have been developed already and the  
> follow a sort of markup.  Why should they all be changed (in  
> essentially unnecessary ways because a computer program could be  
> written to change them and they will look "messier" in the end)  
> just to satisfy a particular markup language that was designed  
> without inquiring about our needs.

This is a somewhat more persuasive argument than the ones I've seen  
you give before.  If the docs are already formatted in a particular  
style, and that style is widely known & used etc., then I can  
understand not wanting to make sudden radical departures from it.   
(Keep in mind that I don't have first-hand experience with what the  
current state fo the sci-py docs is.)  But if you would need to make  
substantial changes in order for your automatic transformation to  
work correctly anyway, and those changes are on a scale comparable to  
the changes you'd need to make to switch to rst, then rst seems to me  
to be far preferable.  E.g., there's already an emacs mode to  
correctly colorize rst text, and there's been some work on making a  
combined emacs/rst colorizing mode.

> This is not a case of Not Invented Here.  I'm very happy to use an  
> existing markup standard.  In fact, I basically like restructured  
> Text.  There are only a few issues I have with it.  Ideally, we can  
> get those issues resolved in epydoc itself.  So that we can use a  
> slightly modified form of restructured Text in the docstrings.

I'm happy to discuss changes in how epydoc interprets the document  
tree generated by restructuredtext.  What I'm much less likely to  
entertain, at least for incorporation into epydoc itself, is changes  
to the markup language itself.  (I realize that the difference  
between those can seem very arbitrary from the outside, but I don't  
want epydoc to require some modified version of rst to run -- and I'd  
like to discourage you from branching/patching epydoc if it's  
possible to make the changes you need in the official version.)

> I'm all for simplifying things, but there is a limit to how much  
> I'm willing to give up when we can easily automate the conversions  
> to what epydoc currently expects.

I think you may underestimate how difficult this type of conversion  
will be, at least if you want it to be technically correct (as  
opposed to working "most of the time.") -- there are a *lot* of  
corner cases to consider.  Of course, this may just reflect my  
personal aversion to "messy" markup languages that don't have any  
precise definition..  As just a simple example, would your  
translation render the following as math or not:?

     If you pay me $12, I'll make this face: :$

>> [...] the authors of ReST have done a *very* good job at  keeping  
>> it to a minimum.
> Well that is debatable in the specific instances of parameter  
> descriptions.  The extra back-ticks are annoying to say the least,  
> and un-necessary.

That's not part of how ReST is defined; it's part of how epydoc is  
defined.  And as I said, it can be changed.

>>   __docformat__ = 'restructuredtext'
> SciPy is big enough that I see no reason we cannot define a  
> slightly modified form of restructured Text (i.e. it uses MoinMoin  
> tables, gets rid of back-ticks in parameter lists, understands math  
> ($ $), and has specific layout for certain sections.

Well, the final decisions are of course up to the scipy developers.   
I think I've made my position fairly clear.  But if you *do* use your  
own slightly-modified-rst, then please give it a new name specify  
that name in __docformat__ variables in all your modules.

>> If ReST table support isn't expressive enough for you, then you  
>> must  be using some pretty complex tables. :)
> Moin Moin uses a different way to describe tables.   :-(

Is there a reason you're attached to MoinMoin's syntax instead of  
rst's?  MoinMoin's doesn't seem particularly more readable to me.  If  
you're using rst for most of your markup, why not use it for tables?

> In summary, my biggest issues with just straight restructured Text are
> 1) back-ticks in parameter lists.
Easy to fix.
> 2) the way math is included
If you can't live with :math:`x=12`, then this may be an issue.
> 3) doesn't understand Moin Moin tables
I don't see why you're wedded to MoinMoin table sytnax
> 4) doesn't seem to have special processing for standard section  
> headers (I may be wrong about this as I'm not an reST expert.
As long as you mark the section headers with ":foo:" instead of  
"foo:", this shouldn't be a problem.  You could even define a custom  
rst transform to replace paragraphs of the form "foo:" with section  
headers if you *really* object to that extra ":"

> I also don't really like the way bold, italics, and courier are  
> handled.  My favorite is now *bold* /italics/ and  `fixed-width`.

Well, rst defines *...* to be used for "emph," so you could use  
stylesheets to translate that to bold instead of italics if you  
want.  `...` already gets rendered as fixed-width, although ``...``  
would be the more technically correct thing to use -- `...` is  
supposed to be used for crossreferences, but will end up rendered as  
fixed width w/ no xref link if the no target is found.  You won't  
get /italics/, though -- could you live with **italics**?  (**...**  
is defined in rst as "strong", which again could be rendered however  
you like w/ stylesheets.)

> It seems like we are going to have to customize (hack) epydoc to do  
> what we want anyway.  Why then can we not "tweak" reST a little-bit  
> too.

Well, if it's possible, I'd like to allow you to do everything you  
want with standard epydoc -- that way, your users don't have to worry  
about compiling & maintaining some variant version.  Failing that,  
I'd like to add enough hooks to epydoc that you can do what you want  
without using patches, but by just having a custom interface.


More information about the SciPy-user mailing list