Subject: [NNTP] wildmat-exact

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 4 May 2005 17:15:39 GMT

In draft26.pre-1.txt I find

     wildmat = wildmat-pattern *("," ["!"] wildmat-pattern)
     wildmat-pattern = 1*wildmat-item
       ; must not begin with "!" if not immediately preceded by "!"
     wildmat-item = wildmat-exact / wildmat-wild
     wildmat-exact = %x21-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E /
          UTF8-non-ascii ; exclude * , ? [ \ ]
     wildmat-wild = "*" / "?"
and
     newsgroup-name = 1*wildmat-exact

So it seems to me that a wildmat-exact (and hence a newsgroup-name) can
contain (and in particular start with) a '!'. That seems to me to make
wildmat ambiguous.

Thanks to Pete Forman for pointing this out to me, in connection with the
proposed news uri revision, to which I had copied the bug.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: [email-masked] (Peter Robinson)
Date: Wed, 4 May 2005 22:53:40 +0100

> contain (and in particular start with) a '!'.

Yes, I remember puzzling over this some time ago.

> That seems to me to make wildmat ambiguous.

I could have sworn that the comment about "!" was reflected in the
normative text last time I looked at it.  That would disambiguate it,
but it's still very far from clear, and it leaves odd gaps in how you
can use wildmat with newsgroup names starting with "!".  Is there any
good reason to allow group names containing "!"?

Regards,

Peter

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 5 May 2005 09:50:35 +0100

> contain (and in particular start with) a '!'.

Yes.

We wanted wildmats to be as general as possible so that they could be used
in other commands in the future. That's why they are allowed to contain
characters like " and !. When we started there was a command that used
wildmats against headers.

On 1st May 2001 you proposed removing it and I said that was "too minimal".
I can't find any further comments in the archives. However, if we're going
to change this from the then-consensus, such as it was, we need to discuss
it first.

> That seems to me to make wildmat ambiguous.

Mu. The point of that comment is to point out how to resolve the ambiguity.
Trying to make the grammar unambiguous just isn't worth the bother. When I
tried to in March 2001 you wrote:

>*You* know that there is a Van Wijngaarden Grammar hiding in there, and
>*I* know that there is a Van Wijngaarden Grammar hiding in there, but for
>the unfortunate masses who do not know, this is just too complicated and
>counterproductive

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 6 May 2005 11:48:57 GMT

>>        ; must not begin with "!" if not immediately preceded by "!"

Ah! I had missed the significance of that disambiguating sentence.

>wildmats against headers.

OK, I had forgotten all the previous exchanges on this (it was a long time
ago). However, I am still unhappy, especially as wildmats are now used
only for filtering newsgroup-names.

Suppose, for the sake of argument, that '!' were a permitted character in
newsgroup-names (despite prohibitions to the contrary in RFC 1036 and
Usefor). Then the following newsgroup-names could legitimately exist.

    !foo.bar
    foo.bar
    foo.baz
    bar.foo

So the wildmat "!foo.bar" matches "!foo.bar", "foo.baz" and "bar.foo", and
"!!foo.bar" matches "foo.bar", "foo.baz" and "bar.foo". You can have great
fun working out other interesting effects.

However, I can see no way to write a wildmat that matches "!foo.bar" and
nothing else.

So it seems that I cannot call NEWNEWS on just that group or, more
importantly, I cannot write a news or nntp URI that opens just that one
group, since <news:!foo.bar> will give me access to just about every group
available on the server :-( .

Life would be simpler if '!' was simply excluded from wildmat-exact.

>to change this from the then-consensus, such as it was, we need to discuss

OK. Discussion please.

---

From: [email-masked] (Peter Robinson)
Date: Sat, 7 May 2005 13:11:35 +0100

Charles Lindsey said:
> In <[email-masked]>
> "Clive D.W. Feather" <[email-masked]> writes:
> 
> >Charles Lindsey said:

As will any implementor coming across this for the first time.  In any
case, it's (now) just a comment in the ABNF so it has no power to
disambiguate anything.

> fun working out other interesting effects.

Actually, assuming the disambiguating comment was normative, neither of
those are valid wildmats according to this spec.  You'd have to have
something like "a,!foo.bar" and "a,!!foo.bar".  Is there a good reason
why we don't have the following: ?

        wildmat = ["!"] wildmat-pattern *("," ["!"] wildmat-pattern)
                  ^^^^^

As things stand (wildmat as it is, the comment just a comment) I'd say
both your examples are legal and unambiguous but don't mean what you
wanted them to mean (they match exactly 1 and 0 groups respectively).

> group,

> Life would be simpler if '!' was simply excluded from wildmat-exact.

I agree.

> >to change this from the then-consensus, such as it was, we need to discuss

There are two possibilities.  One is to enforce (and clarify) the
disambiguating comment, and live with the strange hole in wildmat
functionality for newsgroup names starting with "!".

The other possibility is to exclude ! from wildmat-exact.  Any future
extension that needs to use wildmats to match strings containing ! will
probably need to be able to match the other excluded characters (*?,\[])
as well as "!".  Presumably it will do that by permitting them and using
\ as an escape character.  In that case, handling ! is no different than
any of the others.

I favour the latter possibility.

These are both independent of whether we change the overall wildmat ABNF
to allow it to start with a "!", meaning not.

Regards,

Peter

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 9 May 2005 19:08:03 GMT

>        wildmat = ["!"] wildmat-pattern *("," ["!"] wildmat-pattern)

I believe the disambigating comment was intended to be normative (and if
it wasn't and if it stays, then that would need to be fixed).

You are right about my examples being invalid, but similar absurdities
would arise with the forms "a,!foo.bar" and "a,!!foo.bar", and it is still
not possible to write a wildmat that would accept just the single group
"!foo.bar".

As to allowing a '!' before the first comma, the intention was evidently
to establish the universe within which the exclusions were then to be
made. It fails to do so, of course, because if you want the effect of "all
non-comp.* groups", you can still write "*,!comp.*", even if you cannot
write "!comp.*". However, the former makes it clearer to the human reader
what you are trying to do, so I have no wish to change it.

>I agree.

OK, that makes two of us. But we need some more players ...

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 10 May 2005 07:25:07 +0100

> Usefor). Then the following newsgroup-names could legitimately exist.

Right.

> So the wildmat "!foo.bar" matches "!foo.bar", "foo.baz" and "bar.foo",

No: that violates the syntax (the first wildmat-pattern can't have a !
prefix and a wildmat-pattern can't begin with ! if it doesn't have such a
prefix).

> and
> "!!foo.bar" matches "foo.bar", "foo.baz" and "bar.foo".

That violates it as well, because the first wildmat-pattern can't have a !
prefix.

The intent of the comment is that the syntax:

is equivalent to:

    wildmat = wildmat-pattern *("," ["!"] wildmat-pattern)
    wildmat-pattern = wildmat-init-item *wildmat-item
    wildmat-init-item = wildmat-init-exact / wildmat-wild
    wildmat-item = wildmat-exact / wildmat-wild
    wildmat-init-exact = %x22-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E /
         UTF8-non-ascii ; exclude ! * , ? [ \ ]
    wildmat-exact = "!" / wildmat-init-exact
    wildmat-wild = "*" / "?"

> However, I can see no way to write a wildmat that matches "!foo.bar" and

That would appear to be so.

>> to change this from the then-consensus, such as it was, we need to discuss

In the absence of [] and \ metacharacters, I've come to agree with you.
Other opinions?

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 10 May 2005 07:30:48 +0100

> case, it's (now) just a comment in the ABNF so it has no power to

I thought I'd seen - elsewhere - comments being used to provide normative
but hard-to-describe-syntactically restrictions. However, I can't find
anything in RFC2234 on the topic.

See also Charles's comment about Van Wijngaarden grammars.

> Is there a good reason

A wildmat of the form "!a" would mean, according to our semantics, either
the same as "*,!a" or would match nothing. The former is better expressed
explicitly and the latter is nugatory.

> \ as an escape character.

Which is something we looked out but left out, for now, because of problems
with the corner cases.

> to allow it to start with a "!", meaning not.

Which I am against.

---

From: "Ruud H.G. van Tol" <[email-masked]>
Date: Tue, 10 May 2005 09:27:27 +0200

Clive D.W. Feather:
> Charles Lindsey:

>> However, I can see no way to write a wildmat that matches "!foo.bar"
>> and nothing else.

> That would appear to be so.

A workaround:  *!,!foo.bar 

where "*!" means: empty, nothing.

-- 
Grtz, Ruud

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 10 May 2005 08:43:27 +0100

> where "*!" means: empty, nothing.

"*!" means "any string ending in !"
"!foo.bar" means "except foo.bar" and, in this context, is redundant.

---

From: [email-masked] (Peter Robinson)
Date: Tue, 10 May 2005 23:33:40 +0100

|      wildmat = wildmat-pattern *("," ["!"] wildmat-pattern)
|      wildmat-pattern = 1*wildmat-item
|        ; must not begin with "!" if not immediately preceded by "!"
|      wildmat-item = wildmat-exact / wildmat-wild
|      wildmat-exact = %x21-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E /
|           UTF8-non-ascii ; exclude * , ? [ \ ]
|      wildmat-wild = "*" / "?"
| and
|      newsgroup-name = 1*wildmat-exact

> >> and "!!foo.bar" matches "foo.bar", "foo.baz" and "bar.foo". You can
> >> have great fun working out other interesting effects.
> 
> would arise with the forms "a,!foo.bar" and "a,!!foo.bar",

Yes they would.

[...]

> made.

I see.

> It fails to do so, of course, because if you want the effect of "all
> write "!comp.*".

Yes you can.  But worse, I'd say it "fails to do so" because you can
prefix any valid wildmat with 'not-the-universe,!*,' without altering
its meaning.

> However, the former makes it clearer to the human reader

OK.  Leaving it as it is doesn't restrict functionality in any way, so
I'll drop it.

Regards,

Peter

---

From: [email-masked] (Peter Robinson)
Date: Tue, 10 May 2005 23:33:40 +0100

> explicitly and the latter is nugatory.

OK, that's good enough for me.

> Which is something we looked out but left out, for now, because of problems

I certainly don't think we should try to do all that now.  For clarity,
all I'm saying is that forbidding ! is no worse than any of the other
special characters, and because of the excellent way this document
handles that, the transition to the obvious future extension (if it's
ever needed) should be smooth.

Regards,

Peter

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Wed, 11 May 2005 08:45:51 +0100

Peter Robinson said:
> handles that,

Crawling won't help :-)

Okay, given the responses so far, unless I hear violent objection from
anyone in the next day I will remove ! from the characters legal in
wildmat-exact.

---

From: [email-masked] (Peter Robinson)
Date: Wed, 11 May 2005 23:01:17 +0100

> > special characters, and because of the excellent way this document

Heh, it can't hurt!

> anyone in the next day I will remove ! from the characters legal in

OK.

Peter