headers
Jeremy Harris | 1 Jun 2012 23:01

dnssec

I'm guessing from the message, due to 1f4a55daf :

dnsdb/2202 dns_again_means_nonexist
===============f test-stderr-munged with stderr/2202 failed
Line 48 of "test-stderr-munged" occurs before line 48 of "stderr/2202".
----------
Reverse DNS security status: unverified
===============
1 difference found.
"test-stderr-munged" contains 74 lines; "stderr/2202" contains 73 lines.

--

-- 
Jeremy
Permalink | Reply |
headers
Phil Pennock | 2 Jun 2012 09:19
Favicon
Gravatar

Re: dnssec

On 2012-06-01 at 22:01 +0100, Jeremy Harris wrote:
> I'm guessing from the message, due to 1f4a55daf :

Yes.  For now, just accept the update.  We're going to have to figure
out flag-controllable munging, so that we can munge this away *unless*
the test is about DNSSEC in particular.

-Phil
Permalink | Reply |
headers
Jeremy Harris | 3 Jun 2012 19:00

Re: dnssec

On 2012-06-02 08:19, Phil Pennock wrote:
> On 2012-06-01 at 22:01 +0100, Jeremy Harris wrote:
>> I'm guessing from the message, due to 1f4a55daf :
>
> Yes.  For now, just accept the update.  We're going to have to figure
> out flag-controllable munging, so that we can munge this away *unless*
> the test is about DNSSEC in particular.

I kinda think we need per-testcase-custom munge filters, in addition
to (and possibly subsuming parts of) the wedge of string-translations
done by munge() in runtest.  A layer of indirection would provide for
common cases - the present case would have all of
0002, 0094, 0381, 2202
all pointing to a single script doing
'grep -v "Reverse DNS security status"' on stderr.

Would this do?
--

-- 
Jeremy
Permalink | Reply |
headers
Phil Pennock | 3 Jun 2012 19:14
Favicon
Gravatar

Re: dnssec

On 2012-06-03 at 18:00 +0100, Jeremy Harris wrote:
> On 2012-06-02 08:19, Phil Pennock wrote:
> > On 2012-06-01 at 22:01 +0100, Jeremy Harris wrote:
> >> I'm guessing from the message, due to 1f4a55daf :
> >
> > Yes.  For now, just accept the update.  We're going to have to figure
> > out flag-controllable munging, so that we can munge this away *unless*
> > the test is about DNSSEC in particular.
> 
> I kinda think we need per-testcase-custom munge filters, in addition
> to (and possibly subsuming parts of) the wedge of string-translations
> done by munge() in runtest.  A layer of indirection would provide for
> common cases - the present case would have all of
> 0002, 0094, 0381, 2202
> all pointing to a single script doing
> 'grep -v "Reverse DNS security status"' on stderr.
> 
> Would this do?

I'm thinking that a hash used as a set, where the keys are "features",
could be used in the existing munge, and hash is emptied at the start of
each test, and reading the test driver could add some more keywords
(where we currently have "need_ipv4" or whatever, for instance) which
will toggle on the hash entry.

So the test case for DNSSEC would include "tests dnssec" as an entry
near the start, and munge would have:
  unless exists $test_features{'dnssec'};
after the substitution to be suppressed.
(Continue reading)

Permalink | Reply |
headers
Jeremy Harris | 3 Jun 2012 20:17

Re: dnssec

On 2012-06-03 18:14, Phil Pennock wrote:
> I'm thinking that a hash used as a set, where the keys are "features",
> could be used in the existing munge, and hash is emptied at the start of
> each test, and reading the test driver could add some more keywords
> (where we currently have "need_ipv4" or whatever, for instance) which
> will toggle on the hash entry.
>
> So the test case for DNSSEC would include "tests dnssec" as an entry
> near the start, and munge would have:
>    unless exists $test_features{'dnssec'};
> after the substitution to be suppressed.
>
> Using /^tests\s+(\S+)\s*$/ for the test reading, we can then just take
> $1, shove it in as the key into $test_features and so the work to add a
> new test is "invent a string, use it as 'tests foo' in the driver
> script, check for $test_features{'foo'} on the bit of the rig we want to
> adjust".  That's the minimum of fuss for each new change.

I can see how it works, assuming that the substitutions can be
made versatile enough to do whatever's needed.  This particular case
requires the removal of an entire line; is that possible without
forcing perl to slurp the entire file rather than working line-by-line?

However, I'm concerned slightly that it's a negative test.  While
less work, it is more risky in that the new munge added to avoid
byproducts of $new-feature (in cases not specifically testing $new-feature)
get applied everywhere and could mistakenly mask some other problem.
I'd be happier with a positive test and having to deliberately apply
the munge only where it is identified as being needed, even though
that's more work.
(Continue reading)

Permalink | Reply |
headers
Phil Pennock | 3 Jun 2012 23:38
Favicon
Gravatar

Re: dnssec

On 2012-06-03 at 19:17 +0100, Jeremy Harris wrote:
> I can see how it works, assuming that the substitutions can be
> made versatile enough to do whatever's needed.  This particular case
> requires the removal of an entire line; is that possible without
> forcing perl to slurp the entire file rather than working line-by-line?

  unless (exists $test_feature{'foo'}) {
    next if /^line to be skipped$/;
  }

> However, I'm concerned slightly that it's a negative test.  While
> less work, it is more risky in that the new munge added to avoid
> byproducts of $new-feature (in cases not specifically testing $new-feature)
> get applied everywhere and could mistakenly mask some other problem.
> I'd be happier with a positive test and having to deliberately apply
> the munge only where it is identified as being needed, even though
> that's more work.

Invert the sense of the test.  Trivial enough.
Permalink | Reply |

Gmane