Subject: [tap] Parse error vs failure

From: Michael G Schwern <[email-masked]>
Date: Sat, 31 Jan 2009 12:18:08 -0800

In Oslo, when we came across an edge case that was considered nonsense, the
tendency was to say that its a parse error.  For example, "not ok 4 # SKIP",
because why would you fail a test that didn't run?  Also "1..0" without a
skip, because why would you not run any tests and yet not skip the test?

I think I brought this up earlier, but I want to make sure I have consensus.
I've been calling them failures (with a warning) rather than parse errors.
Why?  Because a parse error means the parser can not extract any information
and can't really tell the user anything but "syntax error at line X".  If your
parser starts guessing at what sort of syntax error it is, well, then it might
as well be grammar at that point.

Worse, a TAP document parser, parsing the full document at once rather than a
traditional line-by-line parser, might invalidate the whole stream and thus
can not extract any information.

If it's just a failure, the parser can get information out of it and has
flexibility about what to do and what action it can suggest the user take.

It complicates the grammar to allow "ok 4 # SKIP" but make "not ok 4 # SKIP" a
syntax error and adds in another special case for the parser.  With "not ok #
SKIP" as normal syntax the skip directive has no effect on the truthiness of
the test.

Consensus?  Generally prefer failure-with-warning over parse error for valid
but sort of nonsense output?

-- 
184. When operating a military vehicle I may *not* attempt something
     "I saw in a cartoon".
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
           http://skippyslist.com/list/

_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: Aristotle Pagaltzis <[email-masked]>
Date: Sat, 31 Jan 2009 22:37:20 +0100

* Michael G Schwern <[email-masked]> [2009-01-31 21:20]:
> Generally prefer failure-with-warning over parse error for
> valid but sort of nonsense output?

Basically. I think we should differentiate between syntactical
and semantic errors.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap

---

From: "David E. Wheeler" <[email-masked]>
Date: Sat, 31 Jan 2009 14:30:10 -0800

> but sort of nonsense output?

+1

David

---

From: Michael Peters <[email-masked]>
Date: Sat, 31 Jan 2009 19:46:28 -0500

> Basically. I think we should differentiate between syntactical

I agree. Syntax errors should be parse errors, but semantic errors should be failure-with-warning.

-- 
Michael Peters
Plus Three, LP

---

From: Steffen Schwigon <[email-masked]>
Date: Sun, 01 Feb 2009 01:58:32 +0100

> I agree. Syntax errors should be parse errors, but semantic errors should be failure-with-warning.

+1 Me too.

Steffen
-- 
Steffen Schwigon <[email-masked]>
Dresden Perl Mongers <http://dresden-pm.org/>
German Perl-Workshop 2009 <http://www.perl-workshop.de/en/2009>
_______________________________________________
tap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tap