[SciPy-User] stats.ranksums vs. stats.mannwhitneyu

josef.pktd@gmai... josef.pktd@gmai...
Thu Oct 11 08:13:30 CDT 2012


On Thu, Oct 11, 2012 at 5:20 AM, Sturla Molden <sturla@molden.no> wrote:
> On 10.10.2012 17:18, josef.pktd@gmail.com wrote:
>>
>> (but I don't think it should make much difference in our conclusions
>> if we have 0.051 or 0.045.)
>
> I think it should make all the difference in the world. The
> Neuman-Pearson error rates comes from the fixed a priori decision rule.
> That is why group sizes and stopping rules need to be fixed in advance
> when doing classical statistics. We should NOT be tempted to "add more
> data" in case of p=0.051. A sharp null hypothesis is known to be false
> in advance, so you can always reject it by adding more data. Once you
> start using the p-value as a "subjective measure of evidence" (which by
> the way violates the likelihood principle), you should do Bayesian
> analysis instead.


Depends on your purpose, I wouldn't bet my money on the difference,
or it wouldn't change much the odds for a bet.
Maybe it's necessary to get *more* data in both cases.

There is still a lot of uncertainty about the p-values because the assumptions
of these tests might not be satisfied. For example
http://onlinelibrary.wiley.com/doi/10.1002/sim.3561/abstract

Even permutation tests rely on additional assumptions on
the distributions of the two samples, and I doubt they are
exactly satisfied.

If we want to get a few more decimals in the small sample case
(less than 20 observations), then we could add the tables that
are available for these cases.

Josef
classical statistics and bayesian decision theory

>
> Sturla
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user


More information about the SciPy-User mailing list