Subject: [tap] Rethinking the TAP version

From: Michael G Schwern <[email-masked]>
Date: Thu, 19 Feb 2009 15:57:10 -0800

After I was the one who pushed for a simple integer TAP version, I now want to
argue for an X.Y format.  Why?

In Oslo we went nuts trying to figure out a syntax for sub-plans that was
elegant and backwards compatible.  By "backwards compatible" I mean that the
suite passes and fails in the same way.  Diagnostic information may be lost,
and even individual test results may be lost, but you still get that critical
pass/fail across all versions.

I see this becoming more and more of a problem as time goes on.  At some
point, we're either going to want to break compatibility or the protocol will
stagnate and die.

I propose that the version number change to X.Y where X indicates the
compatibility series and Y indicates an incremental, but still compatible,
revision.  Everything that can read protocol 2.0 can read 2.1 and 2.13 and
2.99 but if a 2.x parser sees 3.1 it knows it can't handle it and can act
appropriately.  Similarly, a 2.x parser is not guaranteed to be able to read 1.x.

This gives us a lot more latitude for future change.  It enables a sane
deprecation cycle.  It allows readers to explicitly know what it can and can
not handle.  It lets us correct some of the mistakes of the past and build on
our experience for the future.

Can I get some +1s?

PS  My objection to X.Y versions has always been that what X and Y mean
changes from person to person [1], so you can't reliably convey any meaning,
and humans aren't going to check to see what it means for YOUR particular
project.  But we can define what X and Y mean in the protocol, and we have a
program to interpret it.  No ugly flesh bags need be consulted.

[1] http://use.perl.org/~schwern/journal/35127

-- 
I do have a cause though. It's obscenity. I'm for it.
    - Tom Lehrer

---

From: Aristotle Pagaltzis <[email-masked]>
Date: Fri, 20 Feb 2009 12:49:47 +0100

* Michael G Schwern <[email-masked]> [2009-02-20 01:00]:
> I propose that the version number change to X.Y where X
> indicates the compatibility series and Y indicates an
> incremental, but still compatible, revision. Everything that
> can read protocol 2.0 can read 2.1 and 2.13 and 2.99 but if a
> 2.x parser sees 3.1 it knows it can't handle it and can act

It sounds good in theory, but doesn’t work out very well in
practice. It’s only a win if you want to break backcompat often,
which is not a smart idea. But if you revv that major version
only once in every decade, you effectively create a new protocol
each time, not a revision to the old one. Going from `TAP version
1.13` to `TAP version 2.0` is effectively indistinguishable from
going to `TAP2 version 1`, and you don’t need any version check
code to make consumers fail to parse future TAP versions.

(That’s how we handled it in the Atom WG too. There’s no format
version in Atom; if backwards-incompatible changes are ever
necessary, the XML namespace will change, so old consumers will
fail to process the document at all.)

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: Michael G Schwern <[email-masked]>
Date: Fri, 20 Feb 2009 15:24:19 -0800

> fail to process the document at all.)

I get what you're saying.  The problem is we have no namespace.  Technically,
because we ignore anything we can't parse, everything is a TAP document.
There's currently no way for a TAP parser to say "I can not parse this".

TAP parsers will ignore any line they don't recognize.  Upon failing to see a
TAP version they will assume it's TAP version 12.  At best, they'll see a
suite with no tests and fail.  At worst they'll only see the passing tests,
miss new-style failures, and pass what should have failed.

So we need some way to indicate a backwards incompatible change.

-- 
<Schwern> What we learned was if you get confused, grab someone and swing
          them around a few times
        -- Life's lessons from square dancing

---

From: Eric Wilhelm <[email-masked]>
Date: Fri, 20 Feb 2009 22:45:19 -0800

# from Michael G Schwern
# on Friday 20 February 2009 15:24:

>I get what you're saying.  The problem is we have no namespace.

That's exactly what I was thinking.  Yes a major version bump is 
basically a new protocol, but you still need some way to indicate it.

I'm in favor of a $major.$minor version scheme.  You could choose to 
name the major (incompatible) bumps after cartoon characters or types 
of fish, but numbers have a nice sequence.

--Eric
-- 
"It is impossible to make anything foolproof because fools are so 
ingenious."
--Murphy's Second Corollary
---------------------------------------------------
    http://scratchcomputing.com
---------------------------------------------------

---

From: Aristotle Pagaltzis <[email-masked]>
Date: Mon, 23 Feb 2009 10:30:35 +0100

* Michael G Schwern <[email-masked]> [2009-02-21 00:25]:
> Technically, because we ignore anything we can't parse,
> everything is a TAP document. There's currently no way for a
> TAP parser to say "I can not parse this".

Ugh. Yeah.

A corollary is that pre-v13 consumers are a problem no matter
what, as well.

> So we need some way to indicate a backwards incompatible

How about `TAP 1 version 12` then? Because the reason I brought
this up in the first place is that I’d like the notation to
convey that we expect the basic protocol to almost never change.
Using a $major.$minor versioning scheme seems to suggest that we
expect the major version to change relatively frequently. The
notation should be designed for psychological friction against
the idea of willy-nilly change and over-generalisation in
consumers.

Regards,