[SciPy-user] Docstring standards for NumPy and SciPy
edloper at gradient.cis.upenn.edu
Wed Jan 17 10:46:26 CST 2007
>> First, I'd pretty strongly recommend against inventing your own
>> markup language. [...]
> I'm not really convinced by this argument. I agree we shouldn't
> be flippant about introducing new markup and that is not what I've
> proposed. But, already we must learn multiple markup. For
> example, Moin Moin uses one way to describe tables and
> restructured text.
> Basically, I've said that the existing markups do not seem geared
> toward mathematical description (as latex must be hacked on to them).
This is certainly true -- the only markup language I know of for
concisely representing math as text *is* latex. But it doesn't sound
like you're proposing doing anything different -- you just want to
use $expr$ to hack latex into the markup, rather than
using :math:`expr` or `expr`.
> In addition, I don't like the look of existing markup docstrings
> --- especially in the parameter section. That's where my biggest
> problem really lies. What should be clear ends up with all kinds
> of un-necessary back-ticks.
As I mentioned in my previous email, that is *not* part of the
definition of rst as a markup language; it's just a detail about how
that markup's output (a document tree) gets processed by epydoc. It
would be fairly simple to change epydoc to accept parameter sections
where the variables don't have to be marked by backticks; and based
on the feedback here, I intend to do just that.
Responding to Alan Isaac's comment on the same topic, this change
would *not* result in any ability to cross reference from the
parameters. But there's a separate issue of what meaning should be
associated with the expression `foo`. There's two possibilities: it
can be used to mark cross-references; or it can be used to mark
mathematical expressions. Whichever of those it is *not* used for
will need to be marked with some more explicit markup. My preference
is for `foo` to be used for cross-references, since they're used more
often in general. But I understand there's a push from scipy to make
math easy to express. (Although, as I said before, anything more
than simple expressions will look like line-noise to the uninitiated,
whether it's surrounded by $...$ or `...` or :math:`...`.)
> I also don't really like the extra ":" at the beginning of a
> paragraph to denote a section. I could live with the underline
I don't think the ":" at the beginning detracts from readability, and
it certainly doesn't take much work to write, so I guess this is
primarily an aesthetic reaction.. My guess is that it wouldn't take
you too long to get used to. Does anyone other than Travis consider
the use of eg ":Parameters:" instead of "Parameters:" to be a show-
stopper for using rst?
> Basically, scipy docstrings have been developed already and the
> follow a sort of markup. Why should they all be changed (in
> essentially unnecessary ways because a computer program could be
> written to change them and they will look "messier" in the end)
> just to satisfy a particular markup language that was designed
> without inquiring about our needs.
This is a somewhat more persuasive argument than the ones I've seen
you give before. If the docs are already formatted in a particular
style, and that style is widely known & used etc., then I can
understand not wanting to make sudden radical departures from it.
(Keep in mind that I don't have first-hand experience with what the
current state fo the sci-py docs is.) But if you would need to make
substantial changes in order for your automatic transformation to
work correctly anyway, and those changes are on a scale comparable to
the changes you'd need to make to switch to rst, then rst seems to me
to be far preferable. E.g., there's already an emacs mode to
correctly colorize rst text, and there's been some work on making a
combined emacs/rst colorizing mode.
> This is not a case of Not Invented Here. I'm very happy to use an
> existing markup standard. In fact, I basically like restructured
> Text. There are only a few issues I have with it. Ideally, we can
> get those issues resolved in epydoc itself. So that we can use a
> slightly modified form of restructured Text in the docstrings.
I'm happy to discuss changes in how epydoc interprets the document
tree generated by restructuredtext. What I'm much less likely to
entertain, at least for incorporation into epydoc itself, is changes
to the markup language itself. (I realize that the difference
between those can seem very arbitrary from the outside, but I don't
want epydoc to require some modified version of rst to run -- and I'd
like to discourage you from branching/patching epydoc if it's
possible to make the changes you need in the official version.)
> I'm all for simplifying things, but there is a limit to how much
> I'm willing to give up when we can easily automate the conversions
> to what epydoc currently expects.
I think you may underestimate how difficult this type of conversion
will be, at least if you want it to be technically correct (as
opposed to working "most of the time.") -- there are a *lot* of
corner cases to consider. Of course, this may just reflect my
personal aversion to "messy" markup languages that don't have any
precise definition.. As just a simple example, would your
translation render the following as math or not:?
If you pay me $12, I'll make this face: :$
>> [...] the authors of ReST have done a *very* good job at keeping
>> it to a minimum.
> Well that is debatable in the specific instances of parameter
> descriptions. The extra back-ticks are annoying to say the least,
> and un-necessary.
That's not part of how ReST is defined; it's part of how epydoc is
defined. And as I said, it can be changed.
>> __docformat__ = 'restructuredtext'
> SciPy is big enough that I see no reason we cannot define a
> slightly modified form of restructured Text (i.e. it uses MoinMoin
> tables, gets rid of back-ticks in parameter lists, understands math
> ($ $), and has specific layout for certain sections.
Well, the final decisions are of course up to the scipy developers.
I think I've made my position fairly clear. But if you *do* use your
own slightly-modified-rst, then please give it a new name specify
that name in __docformat__ variables in all your modules.
>> If ReST table support isn't expressive enough for you, then you
>> must be using some pretty complex tables. :)
> Moin Moin uses a different way to describe tables. :-(
Is there a reason you're attached to MoinMoin's syntax instead of
rst's? MoinMoin's doesn't seem particularly more readable to me. If
you're using rst for most of your markup, why not use it for tables?
> In summary, my biggest issues with just straight restructured Text are
> 1) back-ticks in parameter lists.
Easy to fix.
> 2) the way math is included
If you can't live with :math:`x=12`, then this may be an issue.
> 3) doesn't understand Moin Moin tables
I don't see why you're wedded to MoinMoin table sytnax
> 4) doesn't seem to have special processing for standard section
> headers (I may be wrong about this as I'm not an reST expert.
As long as you mark the section headers with ":foo:" instead of
"foo:", this shouldn't be a problem. You could even define a custom
rst transform to replace paragraphs of the form "foo:" with section
headers if you *really* object to that extra ":"
> I also don't really like the way bold, italics, and courier are
> handled. My favorite is now *bold* /italics/ and `fixed-width`.
Well, rst defines *...* to be used for "emph," so you could use
stylesheets to translate that to bold instead of italics if you
want. `...` already gets rendered as fixed-width, although ``...``
would be the more technically correct thing to use -- `...` is
supposed to be used for crossreferences, but will end up rendered as
fixed width w/ no xref link if the no target is found. You won't
get /italics/, though -- could you live with **italics**? (**...**
is defined in rst as "strong", which again could be rendered however
you like w/ stylesheets.)
> It seems like we are going to have to customize (hack) epydoc to do
> what we want anyway. Why then can we not "tweak" reST a little-bit
Well, if it's possible, I'd like to allow you to do everything you
want with standard epydoc -- that way, your users don't have to worry
about compiling & maintaining some variant version. Failing that,
I'd like to add enough hooks to epydoc that you can do what you want
without using patches, but by just having a custom interface.
More information about the SciPy-user