Subject: Re: generic token framing in gssapi-cfx-02

From: Luke Howard <[email-masked]>
Date: Wed, 8 Oct 2003 00:17:09 +1000

>I just noticed that from gssapi-cfx-01 to gssapi-cfx-02 the
>generic token framing of rfc-1964 was dropped on all but the
>initial context token.  Did I miss the discussion of this change?

I believe it was discussed at the interim WG.

>Dropping that framing on the protected message tokens (wrap and mic tokens)
>would save a little network bandwidth, however it does add significant
>implementation complexity, i.e. using that framing must be supported
>for rfc-1964 interoperability and also for prot-ready on a
>rfc-1964bis connections where the initiator cannot yet know whether
>the acceptor is rfc-1964 or rfc-1964bis.

Is this not addressed by the following language in section 1:

|   The new token formats specified in this memo MUST be used with all 
|   "newer" encryption types [KRBCLAR] and MAY be used with "older" 
|   encryption types, provided that the initiator and acceptor know, 
|   from the context establishment, that they can both process these new 
|   token formats. 
|    
|   "Newer" encryption types are those which have been specified along 
|   with or since the new Kerberos cryptosystem specification [KCRYPTO], 
|   as defined in section 3.1.3 of [KRBCLAR]. 

-- Luke

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Tue, 7 Oct 2003 10:43:47 -0700

Martin Rex wrote:
>> My primary interoperability concern is between implementations of
>> the new protocol and installed base of rfc-1964!

The draft spelled out how to handle backward-compatibility issues. In
addition, we have run quite extensive tests between 1964 implementations
and the new protocols at this point (with only MS implementations) and
we will certainly do more in the near future, perhaps with other
implementations as well.  Do you have anything more specific in your
mind?

Larry

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Martin Rex
Sent: Tuesday, October 07, 2003 10:00 AM
To: Sam Hartman
Cc: [email-masked]; [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

Sam Hartman wrote:
> 
>     Martin> When doing the change on the protected messages, then I
>     Martin> hope that implementors will perform serious
>     Martin> interoperability testing on this *BEFORE* shipping any
>     Martin> code!
> 
> I hope so too.  I believe Microsoft and Heimdal are already close to
> successful interoperability testing on this spec; MIT is a bit behind,
> but we would be shocked if we have not tested interoperability before
> IETF last call.

My primary interoperability concern is between implementations of
the new protocol and installed base of rfc-1964!

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Tue, 7 Oct 2003 12:16:11 -0700

Martin Rex wrote:
>> GSS-API is two standards in one, an API and a wire-format.

>> By changing a mechanism spec you may break two distinct areas:
>> on-the-wire interoperability and application interoperability.
>> The current spec doesn't sufficiently address application
>> interoperability and W2K3 seems to break it badly...

we have tested against w2k3, with application tests and protocol tests.

I suspect you know something I do not. Please spell it out.

Larry

-----Original Message-----
From: Martin Rex [mailto:[email-masked]] 
Sent: Tuesday, October 07, 2003 12:11 PM
To: Liqiang(Larry) Zhu
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

> implementations as well.  Do you have anything more specific in your

GSS-API is two standards in one, an API and a wire-format.

By changing a mechanism spec you may break two distinct areas:
on-the-wire interoperability and application interoperability.
The current spec doesn't sufficiently address application
interoperability and W2K3 seems to break it badly...

-Martin

---

From: Martin Rex <[email-masked]>
Date: Wed, 8 Oct 2003 19:32:20 +0200 (MET DST)

Sam Hartman wrote:
> 
> To clarify.  Would you consider it acceptable to drop per-message oid
> framing but retain it for context level tokens?

Yes,

but I still see the added complexity and potential interoperability
problems, so I really would recommend some more explicit guidance
and warnings for implementors what the change really means.

since rfc1964 will never change and the crypto-algs of rfc1964 are few,
they should be explicitly listed in rfc1964bis.

The general rule of IETF protcols is "be liberal in what you accept,
be conservative what you send out", therefore more care must be used to
describe the new behaviour of gss_wrap() and gss_get_mic() than that
of gss_unwrap() and gss_verify_mic().

Add the following:
  all Token IDs 0x60 0x00 through 0x60 0xff are reserved and
  must never be assigned by this or a future rfc1964 successor.

Describe exactly how an implementation should decide how to process
a message token in gss_unwrap() or gss_verify_mic().  Example:
  look at the first byte, if it is 0x60 then the token must carry
  the generic gssapi framing.  if the first byte is not 0x60,
  then the parser should try to interpret it as an inner token.

Describe exactly how an implementation should decide in gss_wrap()
and gss_get_mic() whether to add or to leave off the generic framing.
Make sure the description *fully* covers Kerberos implementations
that provide PROT_READY today and may want to continue providing
that feature.  -- this part is harder.  Whoever wants the change
should help coming up with a sufficiently clear spec.

-Martin

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Wed, 8 Oct 2003 18:57:12 -0700

Martin Rex	wrote:			
>> EncryptMessage() in Microsoft's SSPI takes three seperate
>> application-supplied buffers, DecryptMessage() takes a
>> single application-supplied buffer -- and I'm not aware
 >> of any alignment requirements for those buffers...

In DCE_RPC, we need the header size to be constant, having the pseudo
ASN.1 header makes it have variable-size for a given context using AES
enctypes, and this is not desirable.

>> If the members of the kerbers working group really think that
>> they're the only ones who will implement that standard and therefore
 >>whatever is fine with them implementors must be fine for everyone
>> else, then this work should no longer be continued within
>> the IETF.  The purpose of the IETF is explicitly to produce standards
>> that others who didn't participate in the standardization will
>> be able to implement easily and correctly.

1> the implementers of new algorithms do not need to support 1964,
therefore they do not have two code paths.
2> the two code paths already exist regardless if we drop the pseudo
ASN.1 header or not.
3> the experiments we did indicated that other implementers will be able
to implement it easily and correctly.  What else can we do to ensure
that is the case?
4> dropping the ASN.1 header will improve the efficient of per-message
tokens, if nothing else.

Larry
 
-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Martin Rex
Sent: Wednesday, October 08, 2003 10:12 AM
To: Sam Hartman
Cc: [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

Sam Hartman wrote:
> 
> >>>>> "Martin" == Martin Rex <[email-masked]> writes:
> 
>     Martin> Sam Hartman wrote:
>     >>  No, oops we should have called it out on the list.  The draft
>     >> authors and current implementers of the spec seem to believe it
>     >> is a good idea.
> 
>     Martin> Could you elaborate more on the "believe it is a good
>     Martin> idea".
> 
> Sure.  We believe that having the header size of per-message tokens be
> non-constant is non-ideal.  For Microsoft it creates some memory
> alignment issues.

memory alignment issues for Microsoft?  I don't buy that right away!

EncryptMessage() in Microsoft's SSPI takes three seperate
application-supplied buffers, DecryptMessage() takes a
single application-supplied buffer -- and I'm not aware
of any alignment requirements for those buffers...

> 
> for the rest of us, we don't see any benefit in having the DER header
> present.

This is not about taste.  Taste may have been an issue when
rfc-1964 was discussed.  We're discussing technical necessities
for a change from what rfc-1964, and I haven't seen any good
reason so far -- size was a reason that had been discussed on
CAT when thinking about gss-api protected telnet sessions,
but you indicated yourself that size wasn't really an issue for you.

> 
> I continue to not buy the complexity argument very much, especially
> for per-message tokens.

Wait a minute -- I don't buy that changing this characteristic
of rfc1964 is sufficiently justified so far.  Those who want to
change it have to come up with a good reason for it, not the
other way round.

You have to consider all those implementors who never participate
in the IETF process, and for them it will be undoubtedly easier
to implement if there's only a single codepath.

17 of 20 Implementors of gssapi mechanisms that have certified
for interoperability with our application have never participated
the IETF, they only took the final RFC and tried to implement
what was specified.  And that is the real purpose of a standard.

If the members of the kerbers working group really think that
they're the only ones who will implement that standard and therefore
whatever is fine with them implementors must be fine for everyone
else, then this work should no longer be continued within
the IETF.  The purpose of the IETF is explicitly to produce standards
that others who didn't participate in the standardization will
be able to implement easily and correctly. 

-Martin

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Thu, 9 Oct 2003 14:06:53 -0700

Martin Rex wrote:
>>
>> Saving some bytes on the network is the only real value that
>> I can see, but Sam pointed out that this wasn't a reason to
>> come up with the proposal.

>> The alignment issue that you raised could be sufficiently addressed
>> with my proposal above retaining full interoperability with the
>> installed base and one single code path.

you were happy with the comprise we reached yesterday(retaining the OIDs
in all context level tokens), right? Your new proposal is not compatible
with the encoding in 2743.  Dropping the pseudo ASN.1 header in
per-message tokens is better for both efficiency and alignments. If you
still have objections, can you tell us what kind of enhanced error
handling you can get out from the pseudo ASN.1 headers in per-message
tokens? In other words, is there anything broken in your gsstest if we
do not make protocol changes in the current draft? And if so, can that
be fixed?  Your proposed textual changes looked great to us.

Here I quote the encoding specific from 2743.

      2. Token length octets, specifying length of subsequent data
      (i.e., the summed lengths of elements 3-5 in this list, and of the
      mechanism-defined token object following the tag).  This element
      comprises a variable number of octets:

         2a. If the indicated value is less than 128, it shall be
         represented in a single octet with bit 8 (high order) set to
         "0" and the remaining bits representing the value.

         2b. If the indicated value is 128 or more, it shall be
         represented in two or more octets, with bit 8 of the first
         octet set to "1" and the remaining bits of the first octet
         specifying the number of additional octets.  The subsequent
         octets carry the value, 8 bits per octet, most significant
         digit first.  The minimum number of octets shall be used to
         encode the length (i.e., no octets representing leading zeros
         shall be included within the length encoding).

>> In case you're reusing the rfc1964 mechanism OID then all mandatory
>> to implement algorithms of rfc1964 should be retained in the
>> successor spec as mandatory.

>> They don't exist in Kerberos implementations that are still at
rfc-1964
>> and they don't need to exist in newer, see above.

this does not conflict with the current draft, older algorithms are
recommended to continue to use the old token formats.  We use new token
IDs and the new token formats for newer enctypes.  

If you decide to pursue it further along this direction, you have to be
more specific about what capability issues you can think of. So far we
are talking about code complexity and it did not seem to be inline with
practice.

Thanks,

Larry

-----Original Message-----
From: Martin Rex [mailto:[email-masked]] 
Sent: Thursday, October 09, 2003 3:11 AM
To: Liqiang(Larry) Zhu
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

> enctypes, and this is not desirable.

The only variable length part in the generic framing of rfc2743 section
3.1
is the token length field.  2a+b spells out ASN.1 DER encoding, but
I would say it should NOT break interoperability if an implementation
inserted leading zeros to the length field to achieve a constant
header length, i.e. 0x84 0xYY 0xYY 0xYY 0xYY to provide enough
room for inner message tokens up to 2 Gigabytes (we better don't
make assumptions whether an implementation used signed or unsigned
length fields internally.  

This should be fully interoperable with the installed base of rfc-1964
kerberos gssapi mechanisms and with the rfc-1964 crypto algorithms.

> therefore they do not have two code paths.

I violently disagree.

In case you're reusing the rfc1964 mechanism OID then all mandatory
to implement algorithms of rfc1964 should be retained in the
successor spec as mandatory.

> 2> the two code paths already exist regardless if we drop the pseudo

They don't exist in Kerberos implementations that are still at rfc-1964
and they don't need to exist in newer, see above.

> to implement it easily and correctly.  What else can we do to ensure

Personally I consider that a flawed assumption.

> 4> dropping the ASN.1 header will improve the efficient of per-message

Saving some bytes on the network is the only real value that
I can see, but Sam pointed out that this wasn't a reason to
come up with the proposal.

The alignment issue that you raised could be sufficiently addressed
with my proposal above retaining full interoperability with the
installed base and one single code path.

-Martin

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Thu, 9 Oct 2003 15:43:38 -0700

Martin Rex wrote:

>> The wording in rfc2743 2a+b only describes the ASN.1 DER encoding
>> which is a canonical encoding.  We don't need the encoding to be
>> canonical here, so it is really not necessary to take this part
>> of the spec literally, REALLY.

You implicitly assume that the encoding is in ASN.1 (thus the
canonization talk). But my understanding is that only the notation is
ASN.1 alike, not the encoding.

>> Full backwards compatibility and one consistent behaviour for all
crypto->> algs.

We are going circles here; I still do not see specifics on what is the
backward capability issue you refer to.

We are talking about carrying 10+ extra bytes completely useless here
and these constant bytes were repeated for every single per-message
tokens! Is your GSS-API wrapper for SSPI using these OIDs currently?
Please help us to understand the concern here.

Larry

-----Original Message-----
From: Martin Rex [mailto:[email-masked]] 
Sent: Thursday, October 09, 2003 2:41 PM
To: Liqiang(Larry) Zhu
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

Liqiang\ wrote:
> with the encoding in 2743.

Yes it is against the exact wording of rfc2743 section 3.1 2a+b.
I carefully read that section of the spec before I wrote my Email
and I did EXPLICITLY mention that my proposal is slightly
bending section 3.1 2a+b!
But although it is not be conforming to the exact wording of rfc2743,
it is perfectly compatible and interoperable with every implementation.

The wording in rfc2743 2a+b only describes the ASN.1 DER encoding
which is a canonical encoding.  We don't need the encoding to be
canonical here, so it is really not necessary to take this part
of the spec literally, REALLY.

If you talk about length fields inside data entities that are
digitally signed or integrity protected, then tearing them apart
and reassembling them according to ASN.1 DER would not reproduce
the same data if the original encoding wasn't correct ASN.1 DER
(i.e. canonical).

That's similar to the non-canonical encodings of Unicode characters
in UTF8.  Every software will understand non-canonical encodings,
but they're not generally forbidden.  For some specific areas
non-canonical encodings are a problem, and only where that is
the case you need to prohibit them.

> Dropping the pseudo ASN.1 header in
> per-message tokens is better for both efficiency and alignments.

Only efficiency, alignment can be achieved with my proposal.

> If you
> tokens?

Full backwards compatibility and one consistent behaviour for all
crypto-algs.

> are talking about code complexity and it did not seem to be inline
with

No, we were discussing whether a change of that part of rfc1964
was really necessary and worth the added complexity.

I do NOT think the change is sufficiently justified.

-Martin

---

From: "Liqiang(Larry) Zhu" <[email-masked]>
Date: Sat, 11 Oct 2003 13:41:02 -0700

Martin Rex wrote:
>> "Taste" was the most plausible explanation I got so far, but I
consider 
>> that entirely insufficient to justify why an extremely useful and
well-
>> engineered characteristic should be ripped from the successor
Kerberos5 
>> gssapi mechanism.

In my experience, the layouts and sizes of the SECBUFFER_TOKEN buffer
for message protection purpose are so different for different GSS-API
mechanisms that I would never need to look for the OIDs to tell which
mechanism is used. In addition, I found it is not useful because there
is no easy-to-spot pattern in the token header when I stare at the
binary dumps (well if you have a parser that helps, you will figure it
out based on sizes and other characteristics with or without the OIDs as
I stated above).

The saving in sizes by dropping the pseudo ASN.1 header is roughly 30%
for the SECBUFFER_TOKEN buffer, I did not argue if it is significant or
not in real word, but what I argued is that these bytes are useless in
per-message tokens.  

Furthermore, most applications can not take advantage of the embedded
length in the ASN.1 header, and choose to carry length information by
themselves, thus they end up having the length information encoded twice
in every message.  But again, this argument is based more on "taste"
than anything else.

What I saw so far is that most of your comments are emotional (mostly
negative towards folks at Redmond) rather than logical; any performance
measurements are extremely sensitive to the problem domains. If the
messages are smaller, the impact of dropping the extra 10+ bytes is
surely more significant. Of course the saving is marginal if we start to
talk about messages over 1K bytes in size.

I understand the value of having a self-describing token that is
explicitly tagged, but such excessive redundancy in per-message tokens
can NOT be justified even the resulting extra overhead is NOT
significant in your problem domains.

Have you ever used the pseudo-ASN.1 header in *per-message* tokens in
any place of your code or in your real-world diagnosis of problems?  I
asked this before but I did not get a straight answer from you.

Thanks,

Larry

-----Original Message-----
From: Martin Rex [mailto:[email-masked]] 
Sent: Friday, October 10, 2003 2:45 PM
To: Liqiang(Larry) Zhu
Cc: [email-masked]; [email-masked]
Subject: Re: generic token framing in gssapi-cfx-02

Do you have at least ONE reason to drop the header?

I've brought up so many reasons why the header is useful where it is,
and I've shown how you can sufficiently solve the problems that
you claimed to have.

Liqiang wrote:
> 
> Martin Rex wrote:
> >> The four most probable application errors that happen are:
>  >> 1. message truncation
>  >> 2. message with trailing garbage
>  >> 3. message corruption
>  >> 4. not a message
> 
> >> With the generic framing recognizing and distinguishing these
> >> errors is straight-forward and quite reliable.
>  
> >> GSS-API tokens are designed to be embedded into higher level
> >> protocols prior to transmission.  When looking at a hex dump
> >> of an application layer protocol it is extremely helpful if you
> >> can visually recognize and decode the boundaries of the
> >> gssapi token, in particular when you yourself didn't write
> >> the application but have to debug the problem.
>  
> my experience is that application protocols that use the embeded
Wrap/MIC tokens *always* carry information that describes the token
itself. 
>  
> That is in other words, without the pseduo ASN.1 header, all of your
> 4 classes of errors are diagnoseable too. If I use your very own word
> against yourself, portable GSS-API applications should not rely on
> the generic GSS-API frame to diagonse these issues, because the use
> of such framing is optional in non-initial tokens.

Portable applications should be able to rely on gssapi mechanism
designers and implementors to come up with mechanism that produce
helpful error messages -- because portable applications programmers
will have access to only very few of the mechanisms on which
the end users (customers) are going to put that application into
production.
But each time the applications aborts with an authentication error,
the end user (customer) will go and ask the application vendor
to tell him why the application doesn't work.

The amount of plausibility tests and the error reporting in Microsofts
SSPI
is poor.  That's why my gsskrb5.dll does a lot of additional checks and
reports problems that it finds.

If you look at GSSTEST, the tool that we also use to certify
interoperability
with our application, it tries to simulate all the most common
configuration
errors.  For certification I manually review all the error messages
that are returned for the various scenarios and by which calls they're
reported.  In order to succeed the certification those error messages
need to give a strong hint as to why an operation failed, so that
a customer has at least a chance of finding and fixing the error
by himself before opening a support call.  And if I still get the
support call, I hardly ever have to run debuggers or network monitors
to figure out what's going on, the errors reports that my code
writes plus the error message of the gssapi mechanism are all
that I need in 95% of the time.

I have the impression that you never had to support code.

I explicitly said that this was about bugs in the application code and
clueless application programmers that get an error from the gssapi
and then the security guys (like me) get the problem on their
table: "Your security API aborts with an error - you fix it".

In that situation it is extremely valuable if the gssapi mechanism
produces good enough error messages that I don't even have to capture
and look at network traces to see why that stupid application breaks.

>  
> I agree that by having the framing the GSS-API token is
self-described,
> but I am doubtful in the value of having such reduncy in per-message
> token where efficiency has the ultimost importance.

ROTFL.

The performance impact of parsing that small piece of framing
is NIL compared to the crypto stuff.  REALLY, I know what I'm
talking about.  Btw. gsstest does very detailed performance
statistics on every single GSS-API call -- I know the results
for a large variety of gssapi mechanism and implementations,
including NTLM and Kerberos SSP.  Maybe you should look at those
performance measurements before trying to impress people.

We've discussed this to death now, the small size increase of 17 Bytes
is definitely a moot point, it is negligable in ALL practical
environments
and completely in the noise and unmeasurable performance-wise.

>  
> Repeating an error that was made a decade ago is not sufficiently
> justified here.

Why do you keep saying this nonsense.
It was a counscious design decision to put the header at the beginning
of every token with valuable and significant advantages.

>
> Can you accept having the generic framing only in context level
tokens?

With each Mail it seems less and less likely.

Maybe if you raised a real and convincing technical point.
I haven't seen any so far.

"Taste" was the most plausible explanation I got so far,
but I consider that entirely insufficient to justify why an extremely
useful and well-engineered characteristic should be ripped from
the successor Kerberos5 gssapi mechanism.

-Martin