[Numpy-discussion] var bias reason?
Wed Oct 15 10:58:24 CDT 2008
On Wednesday 15 October 2008 11:31:44 am Paul Barrett wrote:
> I'm behind Travis on this one.
> -- Paul
> On Wed, Oct 15, 2008 at 11:19 AM, David Cournapeau
> > On Wed, Oct 15, 2008 at 11:45 PM, Travis E. Oliphant
> > <email@example.com> wrote:
> >> Gabriel Gellner wrote:
> >>> Some colleagues noticed that var uses biased formula's by default
> >>> in numpy, searching for the reason only brought up:
> >>> http://article.gmane.org/gmane.comp.python.numeric.general/12438/
> >>> which I totally agree with, but there was no response? Any reason
> >>> for this?
> >> I will try to respond to this as it was me who made the change. I
> >> think there have been responses, but I think I've preferred to
> >> stay quiet rather than feed a flame war. Ultimately, it is a
> >> matter of preference and I don't think there would be equal
> >> weights given to all the arguments surrounding the decision by
> >> everybody.
> >> I will attempt to articulate my reasons: dividing by n is the
> >> maximum likelihood estimator of variance and I prefer that
> >> justification more than the "un-biased" justification for a
> >> default (especially given that bias is just one part of the
> >> "error" in an estimator). Having every package that computes
> >> the mean return the "un-biased" estimate gives it more cultural
> >> weight than than the concept deserves, I think. Any surprise that
> >> is created by the different default should be mitigated by the
> >> fact that it's an opportunity to learn something about what you
> >> are doing. Here is a paper I wrote on the subject that you
> >> might find useful:
> >> https://contentdm.lib.byu.edu/cdm4/item_viewer.php?CISOROOT=/EER&C
> >>ISOPTR=134&CISOBOX=1&REC=1 (Hopefully, they will resolve a link
> >> problem at the above site soon, but you can read the abstract).
> > Yes, I hope too, I would be happy to read the article.
> > On the limit of unbiasdness, the following document mentions an
> > example (in a different context than variance estimation):
> > http://www.stat.columbia.edu/~gelman/research/published/badbayesres
> > AFAIK, even statisticians who consider themselves as "mostly
> > frequentist" (if that makes any sense) do not advocate unbiasdness
> > as such an important concept anymore (Larry Wasserman mentions it
> > in his "all of statistics").
> > cheers,
> > David
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpyfirstname.lastname@example.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> Numpy-discussion mailing list
Scott M. Ransom Address: NRAO
Phone: (434) 296-0320 520 Edgemont Rd.
email: email@example.com Charlottesville, VA 22903 USA
GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
More information about the Numpy-discussion