TODO

When you don’t want to see the failing tests any more.

What if a bug was reported. You wrote a test to reproduce it, but due to other priorities you don’t have time to fix the code.You don’t want to throw away the test case, but also you should not add a failing test. That would make the CI fail. People would have to learn that this failure is ok. However if at this time some other tests start to fail you’d have a good chance of not noticing it.

The CI should never keep failing. If it fails it should be fixed immediately or it will quickly lose its value.

What can we do?

We can mark the test to be a TODO-test which is expected to fail. (Pytest calls it xfail.)

use strict;
use warnings;

use lib 'lib';
use MySimpleCalc qw(sum);

use Test::More tests => 3;

diag "Add two numbers";
is(sum(1, 1),    2,     '1+1');
is(sum(2, 2),    4,     '2+2');

diag "Add 3 numbers";
TODO: {
    local $TODO = "fix bug summing more than 2 values #173";
    is(sum(2, 2, 2), 6,  '2+2+2');
}

Running it with prove it will fail the whole test.

$ prove -l t/34.t

# Add two numbers
# Add 3 numbers
t/34.t .. ok
All tests successful.
Files=1, Tests=3,  1 wallclock secs ( 0.01 usr  0.00 sys +  0.02 cusr  0.01 csys =  0.04 CPU)
Result: PASS