[SciPy-dev] Two new test failures

josef.pktd@gmai... josef.pktd@gmai...
Thu Nov 13 14:45:44 CST 2008


On Thu, Nov 13, 2008 at 3:03 PM, Pauli Virtanen <pav@iki.fi> wrote:
> Thu, 13 Nov 2008 14:28:33 -0500, josef.pktd wrote:
> [clip]
>> I have problems making my nose tests more complex. All my tests are
>> based on test generators with yield.
>>
>>
>> I did not manage to get the test generators to work inside a TestCase
>> class and using fixtures but I managed to work around that. I added a
>> seed for the random number generator to my test function so now I get
>> deterministic results.
>
> Nose indeed has some problems with test generators in classes inherited
> from TestCase. This is probably a bug in Nose.
>
> If you don't need the self.* methods available in a 'TestCase' class, you
> can base the class on 'object' instead. Nose still finds the tests in it,
> and the generators start to work.
>
>> The problem, I still didn't manage to figure out, is how and whether the
>> knownfailureif decorator works with test generators, e.g.
>>
>> this is my test:
>>
>> @dec.knownfailureif(True, "This test is known to fail") def
>> test_discrete_rvs_cdf_fail():
> [clip]
>>         yield check_discrete_chisquare, distname, arg
> [clip]
>> I get an error instead of a known failure which is not counted towards
>> failures and errors.
>>
>> Is there a trick to get the decorators to work with generators or is
>> this not possible?
>
> I think it will work if the decorators are applied to the _check methods
> instead. But this seems like a bug in Nose -- it should catch SkipTest
> raised already in the generator method, not only in functions yielded by
> it.
>
> --
> Pauli Virtanen
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>

applying the decorator to the check method doesn't seem to work
either, nose still doesn't capture the error:

@dec.knownfailureif(True, "This test is known to fail")
def check_discrete_chisquare(distname, arg, alpha = 0.01):

gives me:
<cut>
KnownFailureTest: This test is known to fail

------------------------------------------------
Ran 12 tests in 0.000s

FAILED (errors=12)


The second problem I have with this is that I did not find any
explanation what I can put into the ifcondition of the decorator,
e.g. when I try to refer to one of the function arguments, I get an
undefined name. Unconditional knownfailure are not very
useful when using test generators.

@dec.knownfailureif(distname=='nbinom', "This test is known to fail")
def check_discrete_chisquare(distname, arg, alpha = 0.01):

gives me:

    @dec.knownfailureif(distname=='nbinom', "This test is known to fail")
NameError: name 'distname' is not defined

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Thanks, I will skip the failing test for now, since I need my time to
finish up some missing tests.
We can do stylistic changes, once I'm pretty sure the basic stuff is ok.
So far, test coverage for distributions.py has increased from 54% to 86%

One more question, recently there was the discussion, not to use the
plain python assert in tests.
I didn't find an assert_ in numpy.testing, but I would prefer to use
an assert without having
to construct a testcase class.
What's the recommended way now with numpy 1.2.1? Maybe I missed
something, otherwise I will work around it.

Josef


More information about the Scipy-dev mailing list