Subject: [tap] Passing Grammar

From: Michael G Schwern <[email-masked]>
Date: Tue, 20 Jan 2009 19:17:43 -0800

Generating some TAP streams from it, it seems kosher.  The plan is nonsense,
though, and I can't think of a way we can encode a plan or test sequence
violation in the grammar.  For example...

ok 9238
not ok 5 # TODO
1..9085934126779

Each individual test passes, but they're A) out of sequence B) missing tests
and C) didn't run enough tests.

Fortunately, those are relatively easy for the parser to keep track of.  So I
think there's value in a grammar which does a good chunk of the pass checks.
Even if it's just enough to determine if a single line is passing or failing,
like so:

TAP-Line        = (Passing-Line / Failing-Line)
Passing-Line    = (Simple-OK / Passing-TODO / Failing-TODO / Passing-SKIP) EOL
Failing-Line    = (Simple-Fail / Failing-SKIP) EOL

Simple-OK       = "ok" [SP Test-Number] [SP Description]
Simple-Fail     = "not ok" [SP Test-Number] [SP Description]
Passing-TODO    = Simple-OK   SP TODO-Directive
Failing-TODO    = Simple-FAIL SP TODO-Directive
Passing-SKIP    = Simple-OK   SP SKIP-Directive
Failing-SKIP    = Simple-FAIL SP SKIP-Directive
TODO-Directive  = "# TODO" [SP Reason]
SKIP-Directive  = "# SKIP" [SP Reason]

Test-Number     = %x31-39 *DIGIT        ; Starting at 1

Description     = Safe-String           ; Description of this test.
Reason          = String                ; Reason for a directive

EOL             = LF             ; Specific to the system producing the stream
Safe-String     = 1*VCHAR                  ; UTF8 without EOL or "#"
String          = 1*(Safe-String / "#")    ; UTF8 without EOL

I think that would be useful.

-- 
"Clutter and overload are not an attribute of information,
 they are failures of design"
    -- Edward Tufte
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Ovid <[email-masked]>
Date: Thu, 22 Jan 2009 13:18:01 -0800 (PST)

> From: Michael G Schwern <[email-masked]>

> violation in the grammar.  For example...

On an unrelated note:

Why do we have "1.." in the plan at all?  It implies that "17..23" is allowed.  And why not?  It might be nice to concatenate TAP streams and we could have embedded plans.  xUnit style tests can work nicely, some nested TAP issues might go away and we have a new, possibly useful feature.

The only reason I can think of for having "1.." is because it's legacy.

Cheers,
Ovid
--
Buy the book         - http://www.oreilly.com/catalog/perlhks/
Tech blog            - http://use.perl.org/~Ovid/journal/
Twitter              - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6

---

From: Aristotle Pagaltzis <[email-masked]>
Date: Fri, 23 Jan 2009 01:19:02 +0100

Why do we have "1.." in the plan at all?

Because it’s 2 characters shorter than `"plan "`. ;-)

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Michael G Schwern <[email-masked]>
Date: Thu, 22 Jan 2009 17:10:12 -0800

> Because it’s 2 characters shorter than `"plan "`. ;-)

Yeah, the grammar needs something to latch on to otherwise you'll just have a
number.  And, as Ovid pointed out, it opens up the opportunity for things like
18..23 but I'm not sure what the practical utility of a formal way to
concatenate TAP streams is.

In reality, I suspect because it mirrors the Perl range operator.

-- 
Life is like a sewer - what you get out of it depends on what you put into it.
    - Tom Lehrer
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap