Subject: APC-1711, 1712 review

From: Abdallah Rayhan <[email-masked]>
Date: Thu, 14 Oct 1999 11:01:59 -0400

I took a pass through APC-1711 and APC-1712 and I would like to post my analysis

on the mailing list. I don't claim my analysis to be complete, however I tried
to point
out the main problems I came across while reading APC-1711 ABNF. My primary
goal is the final ABNF and not the ASN.1 for two reasons. First, I am not an
ASN.1
expert, so I would leave critiquing it to the ASN.1 experts. Second, it would
be a big distraction to the IETF having the MEGACO ABNF at an advanced
stage with a set of testbenches (ANNEX G). Hence, I will try to focus on
APC-1711 ABNF
which I call ABNF1711 and MEGACO/H.248 ABNF which I call ABNF248.

To start with I have noted that the ABNF1711 does not take the ABNF248 as is
but modifies it to suite ASN.1 encoding ( is ASN.1 so restrictive that it does
not
allow ABNF248 syntax ? ) These discrepancies are found in: (I started noting
the different  PRs but then found myself marking almost every one.)

- Tagging every value.
ABNF1711:
PkgdName = LBRKT  PkgItemID [ PkgName ]  RBRKT
PkgName = LBRKT  "/"  ( ( "PE" EQUAL Name )  / "*" ) RBRKT
PkgItemID = ( ( "PI" EQUAL Name )  /   "*"  )

ABNF248:
pkgdName  = [ (PackageName | "*")  SLASH ] (ItemID | "*" )
PackageName  = NAME
ItemID = NAME

You need "PE=" before package name and "PI=" before ItemID.
These differences appear in many production rules.

- Extra brackets as illustrated in the previous example. In ABNF1711,
there is a curly bracket before and after PkgdName while ABNF248 does not
have that. This is all over ABNF1711.

- Extensions do not appear any where in ABNF1711 while they are a major
feature of ABNF248.

-ABNF1711 changes the syntax of the MEGACO constructs.

ABNF1711:
 RequestedActions = LBRKT RequestedAction [ "E" EQUAL SecondEventsDescriptor ]
    [ "SG" EQUAL SignalsDescriptor ]  [ "IC" ]   RBRKT

ABNF248:
requestedActions     = requestedAction LWSP [COLON LWSP  embeddedSignalEvents ]
[COLON LWSP InterceptToken ]
embeddedSignalEvents = firstEmbedding [COMMA firstEmbedding]

as you can see the first level of embedding (firstEmbedding) is chopped in
ABNF1711.
As a matter of fact, it changes the syntax into something different.

-ABNF1711 enforces ordering of descriptors and parameters while the ABNF248
does not. This issue was raised before and the resolution was made to enforce no
order
since there is no relationship between descriptors, e.g., modems or muxes,
whichever comes
first does not affect the other.

- ABNF248 uses COMMA as a separator for commands, descriptors
and parameters. This concept is gone in ABNF1711.

- ABNF1711 changes the syntax and the semantics of MEGACO, e.g.,
ABNF1711:
NotifyReply = LBRKT TerminationID  [ "ER" EQUAL ErrorDescriptor ]  RBRKT
You always have TerminationID regardless of whether it is NotifyReply error or
not,
and ErrorDescriptor is tagged.

ABNF248
notifyReply       = NotifyToken ( commandError | [EQUAL TerminationID])
commandError = EQUAL TerminationID LBRKT errorDescriptor RBRKT
TerminationID is optional in pure notify reply and mandatory in notify error.
ErrorDescriptor is not tagged.

Also, ABNF1711:
SecurityParmIndex = ( 1( BASE64 ) BASE64END )
SequenceNum = ( 1( BASE64 ) BASE64END )
AuthData = ( 5( BASE64 ) BASE64END )
BASE64    = 4( BASE64CHAR )
BASE64END  = 4( BASE64CHAR ) / ( 3( BASE64CHAR ) "?" ) / ( 2( BASE64CHAR ) "??"
)
Where  did "?" and "??" come from ?

ABNF248:
SecurityParmIndex    = 8(HEXDIG)
SequenceNum          = 8(HEXDIG)
AuthData             = 32*64(HEXDIG)

These are some of the differences. If we review each PR, no PR in ABNF1711
will be saved. So I will stop here.

I dont know if the kind of constraints ABNF1711 enforces on the ABNF are due to
to the ASN.1 encoding constraints or because of ASN.12ABNF.
Whatever the reasoning is, APC1711 should focus on completing the ASN.1
description and generate similar testbenches as in the MEGACO I-D either
binary or text, or both. The binary will be definitely different than the text
because of
tagging, however the text should be same as the test benches written so far.
In theory, bin and text should reflect the same protocol and should introduce
the least
entropy into the syntax.

My recommendation is that rather than writing ASN.12ABNF tool, APC1711
should write examples that use the ASN.1 and bin2text and text2bin tools in
order to
test the ASN.1 encoding. Once that is out of the way and we know that the ASN.1
is ok,
we could look into tools like ASN.12ABNF presuming that it would generate ABNF
that is equivalent to ABNF248 syntactical and semantical wise. Otherwise,
ABNF1711
approach is unproductive and waste of effort trying to verify every PR of
ABNF1711.

In addition to that, if we assume that the ASN.1 is ok and the ASN.12ABNF is
ok, from what I saw both are not, is there any guarantee that when they change
the ASN.1, that will not introduce problems into the ABNF ? My feeling is that
it would be a source of confusion and we would need to review the ABNF and
probably hand craft some PRs to make it consistent and ambiguous free.
If that is that is the case, why should we throw out ABNF248 which
we have tested and have a high degree of confidence in. We are very familiar
with ABNF248; we know the ins and outs of it; and we know how to fix any problem

without disturbing the rest of the grammar. How would ABNF1711 do that ?

As for ASN.1, it is not compatible with the standard. This means that ASN.1
compilers
can not produce a correct code.

regards
Abdallah Rayhan

---

From: John Segers <[email-masked]>
Date: Fri, 15 Oct 1999 20:19:39 -0400

Abdallah,

It's good to see that someone had a good look at APC-1711 and 1712.
Below you find some responses to your observations.

I'm x-posting to SG16 list, as the contributions will be discussed in
the Red Bank meeting.

John

Abdallah Rayhan wrote:
> on the mailing list. I don't claim my analysis to be complete, however
> I tried to point
> goal is the final ABNF and not the ASN.1 for two reasons. First, I am
> not an ASN.1
> stage with a set of testbenches (ANNEX G). Hence, I will try to focus
> on APC-1711 ABNF
> but modifies it to suite ASN.1 encoding ( is ASN.1 so restrictive that
> it does not
> allow ABNF248 syntax ? )

In developing an ASN.1 to ABNF translation mechanism, you have to take
into account that the ABNF that is generated must be unambiguous.  One
of the dangers of omitting tags (field names) is introducing ambiguity,
for instance because a string value may happen to be the same as a tag.
If you do an ABNF spec by hand, you can verify that this does not lead
to ambiguity _for that ABNF specification_ but in a mechanical
translation process this is much harder.  That's why the mechanism
proposed in APC-1712 puts restrictions on the use of "AS ?"

>                          These discrepancies are found in: (I started
> noting

Well, one grammar can be specified by different sets of PRs.  So
different sets of PRs I don't find worrying.  Differences in the grammar
is what's important.

> These differences appear in many production rules.

Yes, this is true.  But I don't think it is a major change because the
structure isn't changed very much.

> have that. This is all over ABNF1711.

There are indeed many instances where the generated ABNF has more
braces.  This is because braces are used to delimit sequences of
parameters, consistently throughout the entire syntax.  A packaged name
is a sequence of two parameters: package name and item name.  Hence it
is surrounded by braces.

> - Extensions do not appear any where in ABNF1711 while they are a

The ASN.1 has extension markers (the ellipses).  These are not
translated into ABNF, because the indicate that optional parameters may
be added.  If it is important, every extension marker in the ASN.1 of
APC-1711 can be preceded by a sequence to produce the extension fields
in the ABNF.

> requestedActions     = requestedAction LWSP [COLON LWSP
> embeddedSignalEvents ] [COLON LWSP InterceptToken ]
> as you can see the first level of embedding (firstEmbedding) is
> chopped in ABNF1711.

There's no more production for firstEmbedding, but in ABNF248, the
syntax continues:
firstEmbedding = (secondEvent | signalsDescriptor)
secondEvent = EventsDescriptorToken EQUAL [...]
signalsDescriptor = SignalsDescriptorToken [...]

Noting that the EventsDescriptorToken is an "E" and that the
SignalsDescriptorToken is the string "SG", you see that the differences
are very minor.  ABNF1711 does not use colons for separators, but I
think that's not important.

> does not. This issue was raised before and the resolution was made to
> enforce no order
> since there is no relationship between descriptors, e.g., modems or
> muxes, whichever comes

True.  This is not a problem of the approach per se, but my personal
stubborness.  ABNF1711 enforces ordering, but the syntax specifies that
every descriptor can only occurs once in a list, while ABNF248 puts that
in a comment (i.e., specifies it as a semantic constraint).  It's a
small operation to write ASN.1 that will produce ABNF that contains
"descriptor *(COMMA descriptor)" constructs instead of the
"[descr1][descr2][descr3]" constructs.

> and parameters. This concept is gone in ABNF1711.

If we rewrite as explained above, you'll have your commas back.

> You always have TerminationID regardless of whether it is NotifyReply
> error or not,

I explained about the tagging above, so let that go.  The optional
TerminationID presented a bit of a problem for the ASN.1.  Because I
couldn't for the life of me remember why it's important to be able to
leave TerminationIDs out in NotifyReply, I made them mandatory.

> BASE64END  = 4( BASE64CHAR ) / ( 3( BASE64CHAR ) "?" ) / ( 2(
> BASE64CHAR ) "??" )

They ensure that the number of characters in the BASE64 encoded octet
string is a multiple of 4.  I was told that BASE64 works that way.

> will be saved. So I will stop here.

Once again, the PRs aren't the most important, the differences in
grammar are.

> binary or text, or both. The binary will be definitely different than
> the text because of
> In theory, bin and text should reflect the same protocol and should
> introduce the least
> entropy into the syntax.
>
> should write examples that use the ASN.1 and bin2text and text2bin
> tools in order to
> test the ASN.1 encoding. Once that is out of the way and we know that
> the ASN.1 is ok,
> that is equivalent to ABNF248 syntactical and semantical wise.
> Otherwise, ABNF1711
> approach is unproductive and waste of effort trying to verify every PR
> of ABNF1711.

I agree that examples are useful, but this comment is counter
productive.

> ok, from what I saw both are not,

Well, if you define ABNF248 as being right and anything that's different
as being wrong, then obviously your right, right?  Otherwise, your're
wrong, right?

As far as I can tell, the ASN.1 is ok, but it definitely needs review.
I'm glad you took the effort to be the first to comment. And I'm glad
that you didn't find any semantic errors (barring the mandatory
TerminationID, which I don't consider an error but a difference).

>                                   is there any guarantee that when
> they change

Using the mechanical translation mechanism gives me a higher degree of
confidence.  I don't have to worry anymore about having accidentally
omitted a closing brace, or being inconsistent with the use of
separators.

>
> As for ASN.1, it is not compatible with the standard. This means that
> ASN.1 compilers

APC-1712 addresses this.  The extensions that were added to deal with
the different types of separators and brackets in ABNF248 can be put
into special comments in the ASN.1.  They take the form "--& <extension>
--"  A tool that does the text encoding as described, recognizes these
comments, a normal ASN.1 compiler sees them as comments, ignores them
and produces code (correctness is up to the compiler writer :-).

--
John Segers                                  email: [email-masked]
Lucent Technologies                                        Room HE 306
Dept. Forward Looking Work                      phone: +31 35 687 4724
P.O. Box 18, 1270 AA  Huizen                      fax: +31 35 687 5954