[SciPy-User] stats.ranksums vs. stats.mannwhitneyu

josef.pktd@gmai... josef.pktd@gmai...
Thu Oct 11 08:13:30 CDT 2012

On Thu, Oct 11, 2012 at 5:20 AM, Sturla Molden <sturla@molden.no> wrote:
> On 10.10.2012 17:18, josef.pktd@gmail.com wrote:
>> (but I don't think it should make much difference in our conclusions
>> if we have 0.051 or 0.045.)
> I think it should make all the difference in the world. The
> Neuman-Pearson error rates comes from the fixed a priori decision rule.
> That is why group sizes and stopping rules need to be fixed in advance
> when doing classical statistics. We should NOT be tempted to "add more
> data" in case of p=0.051. A sharp null hypothesis is known to be false
> in advance, so you can always reject it by adding more data. Once you
> start using the p-value as a "subjective measure of evidence" (which by
> the way violates the likelihood principle), you should do Bayesian
> analysis instead.

Depends on your purpose, I wouldn't bet my money on the difference,
or it wouldn't change much the odds for a bet.
Maybe it's necessary to get *more* data in both cases.

There is still a lot of uncertainty about the p-values because the assumptions
of these tests might not be satisfied. For example

Even permutation tests rely on additional assumptions on
the distributions of the two samples, and I doubt they are
exactly satisfied.

If we want to get a few more decimals in the small sample case
(less than 20 observations), then we could add the tables that
are available for these cases.

classical statistics and bayesian decision theory

> Sturla
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user

More information about the SciPy-User mailing list