Subject: [tap] Complete TAP v13 ABNF grammar. Now run it!

From: Michael G Schwern <[email-masked]>
Date: Sun, 18 Jan 2009 20:41:30 -0800

I would like to have a meaningful TAP v13 grammar ready before the Birmingham
QA Conference end of March.  To that end, I've picked up Gaurav's ball and
completed the TAP grammar on the wiki.  By "complete" I mean it covers all the
syntax of TAP v13 which is normal TAP, none of the proposed extensions.  I
used TAP::Parser::Grammar for reference.
http://testanything.org/wiki/index.php/TAP_at_IETF:_Draft_Standard

My knowledge of ABNF sucks, I was going from the Wikipedia article.  There's
probably mistakes.  There's probably things I'm forgetting.  There's a small
pile of unresolved issues in the discussions page.  But it's something.

Now what would be interesting is for someone to take that ABNF, make a simple
parser out of it and see what happens.  There is the TAP test suite around
somewhere, it would be interesting to run the grammar through that.

-- 
I have a date with some giant cartoon robots and booze.

_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 19 Jan 2009 06:18:45 +0100

>http://testanything.org/wiki/index.php/TAP_at_IETF:_Draft_Standard

I'm familiar with Internet Draft authoring and ABNF, not so much
familiar with TAP itself, please understand my comments in that light.

>pile of unresolved issues in the discussions page.  But it's something.

There is an error in the comment production,

  Comment = # String EOL

Presumably the '#' is meant to be quoted.

I am not sure why

  Version = "T" "A" "P" SP "version" SP Version-Number EOL

Is written like this, that's just the same as

  Version = "TAP version " Version-Number EOL

The specification should be explicit about case-sensitivity, normally
ABNF quoted strings imply ascii-insensitivity. If that is intended, it
should be spelled out somewhere. If it's not intended, %... sequences
would have to be used.

The relationship between TAP-line and TAP-stream is unclear. I would
have expected something more like

  TAP-document = header plan body [comments]
               / header body plan [comments]

  header       = [comments] [version]

  body         = *(comment / TAP-line)

  TAP-line     = Test-Result / bail-out

This would actually use the TAP-line production and make clear that you
have to have exactly one plan, in addition to being somewhat easier to
read, in my opinion.

Rules like EOL and String should probably be in their own section, at
least putting them alongside Plan is not a good idea.

>somewhere, it would be interesting to run the grammar through that.

My Parse::ABNF (on CPAN) module turns an ABNF grammar into a array/hash
reference structure that should be easy to print out in whatever form is
desired for a given parsing toolkit. That might give a quick and easy
way to do that.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/ 
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Michael G Schwern <[email-masked]>
Date: Sun, 18 Jan 2009 22:54:31 -0800

> familiar with TAP itself, please understand my comments in that light.

Thanks for your fast comments.

> Presumably the '#' is meant to be quoted.

Yep, fixed.

> should be spelled out somewhere. If it's not intended, %... sequences

You're right, I should have used % sequences.  I specifically wanted to
require TAP in all-caps (this may or may not be a good idea, but that's what I
intended).  The rest of the string literals in the spec can be case sensitive.

Fixed.

> have to have exactly one plan, in addition to being somewhat easier to

TAP-Line was really just a way to define Significant-TAP-Line.  You're right,
it's stilted.  Your way is much better.  Fixed.

> least putting them alongside Plan is not a good idea.

I've put them in a section called "Values" along with some discussion.  Just
using String for Description is not entirely correct.  Is there a way to
specify "any character in this set EXCEPT this one"?  Description should be
String without "#".

Is there a way to specify in the grammar that the document has to have a
consistent EOL?  That is, it has to use LF or CRLF but not both?

> desired for a given parsing toolkit. That might give a quick and easy

Excellent, I'll look into it.

-- 
package Outer::Space;  use Test::More tests => 9;
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 19 Jan 2009 08:51:52 +0100

>specify "any character in this set EXCEPT this one"?  Description should be

There is not, other than a prose value like

  foo = <a String without any "#" characters>

That's fine for drafts but ultimately makes automatically generating
code from the grammar more difficult as the specification matures.

A better place for prose values would be defining a symbol like

  Extension = <string without any EOL and not matching TAP-line> EOL

>consistent EOL?  That is, it has to use LF or CRLF but not both?

In ABNF there is not. I would not recommend specifying that either. My
personal preference in handling this is doing what the XML specification
does, require processing as if there was a pre-processing step turning
CRLF and bare CR into LF, then use only LF in the grammar as needed.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/ 
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Ovid <[email-masked]>
Date: Mon, 19 Jan 2009 02:30:05 -0800 (PST)

----- Original Message ----

> From: Michael G Schwern <[email-masked]>

> http://testanything.org/wiki/index.php/TAP_at_IETF:_Draft_Standard

Out of curiosity, what was wrong with my grammar?

Cheers,
Ovid
--
Buy the book         - http://www.oreilly.com/catalog/perlhks/
Tech blog            - http://use.perl.org/~Ovid/journal/
Twitter              - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6

---

From: Michael G Schwern <[email-masked]>
Date: Mon, 19 Jan 2009 14:10:19 -0800

> Out of curiosity, what was wrong with my grammar?

Heh, I never even thought to look at the TAP::Parser::Grammar docs for a
grammar.  I went straight into the code to see how it's actually implemented.

Aside from the technical issue of not being ABNF, it will be worthwhile to
compare the differences between the two to reveal edge cases and strict vs
pragmatic concerns.

-- 
There will be snacks.