Subject: [Int-area] Comments on draft-ietf-intarea-gue-07 (and fllow-up from last IETF meeting)

From: Gorry Fairhurst <[email-masked]>
Date: Mon, 11 Mar 2019 09:40:03 +0000

I've just looked at draft-ietf-intarea-gue-07again and have a few comments:

First, I spoke at the Mic in INTEAREA last IETF meeting, and I then had 
a few serious concerns about this draft:

(1) References. This ID is incomplete because it relies on what appears 
to be abandoned work for important details. It does not really talk 
about the security and deployment concerns, or the extensions that are 
required to implement this. Instead, it points to a set of expired 
drafts (at least one expired in 2015). I do not believe these dependent 
drafts are mature. As far as I know they have not been adopted by a 
working group and have no status - at the very least these need to be 
adopted, or appropriate parts incorporated. This is still the case.

I suggest it is bad practice to require mechanisms in a PS that are 
informative references, especially ones that are not adopted. e.g. 
draft-herbert-transportsand this needs working group review.

The security issues relating to this document seem to be partly in 
another non-working-group document.

I would expect the fate of these other documents needs to be decided 
before the present document progresses.

(2) I could not understand why this is a “PS”? I suggest that the 
proposal could be too flexible and extensible to be usefully 
standardised and demonstrated to interoperate. If this is a “PS”, I 
would like to be sure that all the functionality described is 
unambiguous, has been implemented somewhere and is really needed. There 
are many points of extensibility and likely many issues that will be 
raised as these extensions emerge. I do not grasp why we need to specify 
all these possibilities at this time, with no future use identified.

--

As David noted in his recent TSV ART Review, from a transport 
perspective, there are some more major issues. My thoughts on this 
should be probably read as additional comments following David's review:

Tunnels & Endpoints

There are concerns because of the wide applicability of various 
combinations of features. Overall, this flexibility makes it hard to 
analyse this extensible framework from a transport perspective. At some 
point I stopped trying to follow the various pathologies that can arise 
with different combinations of use (so I think there are more issues). I 
am therefore surprised that this is being proposed as a "PS", because it 
seems there are at least likely to be unknowns and very likely 
applicability concerns.

I think there are serious issues that emerge because the method lumps 
tunnels and endpoint encapsulation together. One example is that there 
is insufficient specification of how congestion collapse is to be 
avoided, when acting as an endpoint for a non-congestion controlled 
payload (section 5.9 does not address congestion control). Another is 
the mis-use of the zero checksum text in 5.7.1 (a misquotation of the 
spec. RFC1122 states in 4.1.3.4, and incompatible with RFC6935).

This is another way of doing many things, many of which are already 
specified in existing RFCs - albeit with some extra (and interesting) 
features and a lot more flexibility. The disadvantage is that by 
widening the applicability it is less clear on the implications of 
specific techniques and could itself be extended in arbitrary 
directions. I think the IETF should consider this when determining what 
to do with this document, and seek to understand why we need 
interoperability standards in this area.

Also:

* Section 3.3.1
I do not understand the operation of the paired flags. I suggest that 
some combinations can result in significant complexity - is this 
something that the WG has considered, and what do they think about this? 
e.g.:
" Flags can be paired together to allow different lengths for an
extension field. "

* I do not understand this statement:
" If a decapsulator receives a GUE packet with private data, it MUST
validate the private data appropriately. "
- How does it do that, or what does "appropriately" mean? What are the 
costs, and the issues if verification fails?

- How does the receiver know this is being done? (If we don't 
standardise it, then why would need to specify it in a RFC?)
"An implementation MAY place security data in GUE private data which if 
present MUST be verified for packet acceptance."

* Section 4:
" Variant 1 of GUE allows direct encapsulation of IPv4 and IPv6 in UDP."
- How is congestion control handled in this case, I expected text on the 
congestion safety of this approach for use in different scenarios, but 
found none.

Endpoint checksum v Tunnel Checksum

Section 4:
Variant 1: This variant appears to be a tunnel that places a packet 
directly in a UDP packet.

* Section 5.2 seems way under-specified with respect to the 
pseudo-header calculation. This could be contained in anothe ID, I did 
not check, because I suggest it needs to be in this particular document.

* Section 5.7.1:
"By default, a
decapsulator SHOULD accept UDP packets with a zero checksum. A node
MAY be configured to disallow zero checksums per [RFC1122]."

I read this is as a misquotation of the spec. RFC1122 states in 4.1.3.4 
that:

"An application MAY optionally be able to
control whether a UDP checksum will be generated, but it
MUST default to checksumming on."

- Instead, I read RFC 6935 for IPv6 explicitly stating:

"As an alternative, certain protocols that use UDP as a tunnel
encapsulation MAY enable zero-checksum mode for a specific port
(or set of ports) for sending and/or receiving. Any node
implementing zero-checksum mode MUST follow the node requirements
specified in Section 4 of "Applicability Statement for the use of
IPv6 UDP Datagrams with Zero Checksums" [RFC6936]."

- I do not yet understand how GUE can safely vary this. The text is 
insufficient.

* Section 5.9

This states

" In the case that the encapsulated traffic does not implement any or
sufficient control, or it is not known whether a transmitter will
consistently implement proper congestion control, then congestion
control at the encapsulation layer MUST be provided per [RFC5405].
Note that this case applies to a significant use case in network
virtualization in which guests run third party networking stacks
that cannot be implicitly trusted to implement conformant congestion
control."

It then states:
"Out of band mechanisms such as rate limiting, Managed Circuit
Breaker [RFC8084], or traffic isolation MAY be used to provide
rudimentary congestion control. "

This may be just lack of clarity in the text, but I think thisseems like 
a "magic trick" to escape doing congestion control.
- which may be heading in a good direction, but really does not address 
the issue. This is particularly worrying since 5.10 actually describes 
the use of the method for multicast, broadcast etc, but still has no 
explanation of how to provide prevent congestion collapse.

I think the following statement is currently flawed and needs more clarity:
"For finer-grained congestion control
that allows alternate congestion control algorithms, reaction time
within an RTT, and interaction with ECN, in-band mechanisms might be
warranted."
- I think this needs to be removed and replaced by something that is 
more specific. I'm concerned the interaction with ECN seems 
under-specified (or perhaps should be removed - or at least be replaced 
with a mechanism that has IETF consensus).

* Section 5.8.2
- I would like to see discussion on whether 5.8.2 is safe or unsafe. I 
do not know how the integrity will be managed.

"The GUE header checksum (in version 0x0) provides a UDP-lite
[RFC3828] type of checksum capability as an optional field of the
GUE header."

Endpoint - NAT

* Section 5.7
- This seems about NAT. Is this appropriate?

Fragmentation

- Elsewhere in the iRTF fragmentation has been described as fragile, why 
is it safe in this spec?

* GUE level fragmentation is mentioned, and interesting as a concept. 
However, there is so much discussion of IPv6 fragments that I think this 
needs detailed consideration by the WG. It also directly competes with 
the TSVWG work on UDP-Options, albeit the IETF can decide to do two more 
methods that both use UDP, but I'd hope that if it specified either, it 
would carefully consider the issues in accepting fragments at a receiver.

* Section 5.4:
"Note that set flags in a GUE
header that are unknown to a decapsulator MUST NOT be ignored. If a
GUE packet is received by a decapsulator with unknown flags, ..."

- Does that imply silently discarded, why not logged?

Other comments:

"If a received GUE packet in IPv6 contains a
protocol number that is an extension header (e.g. Destination
Options) then the extension header is processed after the GUE header
is processed as though the GUE header is an extension header."

* Section 3.2.2
I do not understand the intended IANA allocation method:
" Control messages will be defined in an IANA registry. Control message
types 1 through 127 may be defined in standards. "
- What is the difference between the two use below, and why are they
separately mentioned? Are these differentiated in the registry?>
" Types 128 through
255 are reserved to be user defined for experimentation or private
control messages."

* Section 3.4.
I don't understand the normative MUST, it could just be that this just a 
truism, that a receiver can not use data that it does not understand?

* Section 5.4:
"Such events MAY be logged subject to configuration and rate limiting of 
logging messages. "

- I don't understand the MAY here. I could see why "REQUIRED" or 
"RECOMMENDED" is stated for operational reasons.
- Why is rate limiting only permitted by a "MAY" - should that be 
required or recommended?

* This is another way of doing many things that are specified in 
existing RFCs - albeit with some extra features and a lot more flexibility.

Section 6.2

- This is an alternative proposal to using existing IETF specifications.
It states:
"A number of different encapsulation techniques have been proposed for
the encapsulation of one protocol over another." ...

- What follows is mainly a list of PS specifications from IETF, not 
proposals.
And states:
"Several proposals exist for encapsulating packets over UDP including
ESP over UDP [RFC3948], TCP directly over UDP [TCPUDP], VXLAN
[RFC7348], LISP [RFC6830] which encapsulates layer 3 packets,
MPLS/UDP [RFC7510], GENEVE [GENEVE], and GRE-in-UDP Encapsulation
[RFC8086]."
- Many of these are PS specifications from IETF, not proposals. If the 
WG thinks another spec is needed, this should not regard the existing PS 
as "proposals", but clearly differentiate the benefits of the new approach.

----------------

If this document is to be published, I would expect it needs significant 
changes and I would say this would certainly require a much more 
detailed transport review together with the other drafts that form a 
part of the spec.

Best wishes,

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Mon, 11 Mar 2019 11:01:27 -0700

Hi Gorry,

Thanks for the comments. Some replies inline.

> adopted, or appropriate parts incorporated. This is still the case.
The [GUEEXTEN] reference is incorrect in the version 7. This should
refer to draft-ietf-intarea-gue-extensions-06 which is a WG item.

> draft-herbert-transportsand this needs working group review.
As I said, reference is incorrect. It will be fixed in next version.

> another non-working-group document.
draft-ietf-intarea-gue-extensions-06 defines security options and
security payload transforms (i.e. DTLS).

> before the present document progresses.
Right, all normative references are RFCs or the WG item.

> all these possibilities at this time, with no future use identified.

Again see draft-ietf-intarea-gue-extensions-06. That defines eight
initial options. The protocol is extensible, however beyond the
initial set of options I would expect at most one new option to be
introduced every couple of years. Note this is markedly different from
a protocol like Geneve that has a 24 bit TLV space allowing thousands
of option (including proprietary options),and as AFAICT haven't
proposed a single one as standard.

GUE is implemented and in deployment. It has been in upstream Linux since 2014.

This is a very general comment. Can you give a specific pathological
case not properly handled by the protocol?

> spec. RFC1122 states in 4.1.3.4, and incompatible with RFC6935).

Tunnels and encapsulation are separate concepts in the draft. Section
5.1 defines network tunnels. We can add definition in the terminology
section of 1.2.

Exactly where is the draft misquoting RFC1122?

Exactly how is this incompatible with RFC6935 (note David's point was
that the protocol needs to show how requirements of RFC6935 and
RFC6936 are met).

> interoperability standards in this area.
Again, a very general comment for which it is hard to provide a
specific response. GUE has been discussed in depth in two working
groups and in being run in the wild for a while. For instance, if this
is just "another way of doing many things, many of which are already
specified in existing RFCs", then my question is what exactly are
those RFCs? Note that section 6 gives a comparison with similar
protocols and describes differentiating features.

> something that the WG has considered, and what do they think about this?

IMO, they're actually quite simple to allow variable length fields.
The example in the draft is:

 "Two flag bits are paired, a field can possibly be three different
lengths-- that is bit value of 00  indicates no field present; 01, 10,
and 11 indicate three possible lengths for the field."

This is used in the security extension field described in

The values in the SEC flags are:
      o 000b - No security field
      o 001b - 64 bit security field
      o 010b - 128 bit security field
      o 011b - 256 bit security field
      o 100b - 320 bit security field (HMAC)
      o 101b, 110b, 111b - Reserved values

> costs, and the issues if verification fails?

That was comment David also made. The point will be clarified.

> congestion safety of this approach for use in different scenarios, but

In this case GUE is carrying IP protocol. Per RFC 8085 congestion
control is generally assumed when encapsulation carries an IP
protocol. Technically, GUE always carries an IP protocol (as opposed
to GREoUDP that carries an EtherType). The congestion considerations
in GUE are to cover cases where GUE would carry something like EtherIP
or GRE. Those can be  encapsulations of non-IP protocols hence should
have their own congestion considerations, however neither RFC2784 nor
RFC3378 mention congestion. I think it is valuable to have congestion
considerations in this doc to cover those cases.

> not check, because I suggest it needs to be in this particular document.
An example can be added should a simple TCP/GUE and the pseudo header
for TCP checksum calculation.

> MAY be configured to disallow zero checksums per [RFC1122]."
This is referring to receiver processing. It is allowed by requirement
in RFC1122:

"An application MAY optionally be able to control whether UDP
datagrams without checksums should be discarded or passed to the
application".

> - I do not yet understand how GUE can safely vary this. The text is
David made same comment, it will be addressed in next draft version.

> with a mechanism that has IETF consensus).
Section 5.9 is mistitled, it should be " Congestion Considerations".
It is based on section 8 in RFC8086 (GRE-UDP). More text can be
imported from RFC8086 if it helps to clarify.

> do not know how the integrity will be managed.
Which section are you referring to (there is no section 5.8.2).

> - This seems about NAT. Is this appropriate?
5.7 is about UDP checksum. NAT opertation with UDP checksums is
unaffected by use of GUE.

> - Elsewhere in the iRTF fragmentation has been described as fragile, why
You interchanged the concepts of  "fragile" and "not safe"-- they are
not equivalent. draft-ietf-intarea-frag-fragiledescribes how IP
fragmentation is fragile in the Internet. This is because IP fragments
are often dropped by intermediate nodes. Fragmentation in an
encapsulation is hidden to intermediate nodes so for that reason it
may be more deployable (for instance, ECMP works better with GUE
fragmentation than IPv4 fragmentation). As for being safe, it is as
safe as any other fragmentation and DOS mitigations for fragmentation
learned over the years are applicable. The GUE fragment header also
has a larger ID field to avoid the problem that was seen in IPv4.

> would carefully consider the issues in accepting fragments at a receiver.
They are very different. UDP-options are being defined as part of the
transport layer to allow fragmentation of packets for a UDP
application. GUE does fragmentation in the encapsulation layer and
addressed the problems of tunnel fragmentation described in RFC4459.
UDP options have nothing to do with tunnels. I'd also point out that
UDP options are very preliminary, there is no deployment and it has
yet to be proved if they are even deployable.

> - Does that imply silently discarded, why not logged?
They can be logged, will add text.

> 255 are reserved to be user defined for experimentation or private
Yes, only 0 to 127 are IANA assigned. Types 129 to 255 are site local.

> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 13 Mar 2019 16:07:58 +0000

So thanks for your clarifications. I read this draft because an early 
review was requested, and the purpose of an early review is often to 
judge where more work may be needed, or whether there may be issues if a 
formal review for IETF-LC were made. I think there are issues.

In places the comments are a little general - but they show the places 
where I would dig if I did a detailed reveiw, and should help you 
prepare a better draft.  Some replies are in-line, but likely I will 
await the next revision before commenting on the entire draft again.

My comment concerning applicability are more difficult to write in word, so to set the scene:

I think there are many things that can be run over UDP. Some of these are useful
to standardise in the IETF, often for interoperability reasons, and one of the more
difficult jobs for a WG to figure out is how to "limit" the scope of a proposal
to help achieve useful operational, security or more interop implementation. I'm
not sure how far the WG scoped this particular draft (i.e. who else
is needing the spec besides the implementor) and what The WG decided to
add or remove over the years - others in INTAREA will know that better than me
(and maybe in NVO03 - perhaps things have been refined since NVO3).
I think it would be helpful to understand this.

On 11/03/2019, 18:01, Tom Herbert wrote:
> On Mon, Mar 11, 2019 at 2:40 AM Gorry Fairhurst<[email-masked]>  wrote:
>>
GF: I suspect both documents will need reviewed together.

> security payload transforms (i.e. DTLS).
I was aware of those - but this document does not discuss other security 
considerations concerned with teh mechanisms in this document.
> Right, all normative references are RFCs or the WG item.
They were not, if this is fixed, the document is improved.

> GUE is implemented and in deployment.
> It has been in upstream Linux since 2014.
Has it also been implemented on other OS/platform? (i.e. what does it 
interop with?)
- this is really helpful to understand.
>> Tunnels&  Endpoints
How do you know there are none?
> 5.1 defines network tunnels. We can add definition in the terminology
I read the draft, and made the comment.
> Exactly where is the draft misquoting RFC1122?
Does it default to using a checksum?
> that the protocol needs to show how requirements of RFC6935 and
Relating to RFC6935 - there were a set of requirements for IPv6 usage, 
and I expected this to be clearly explained and asserted that hosts can 
not just disable the checksum and then encapapsulate a transport, etc.
> protocols and describes differentiating features.
Yes this was the sction that generated this comment. It seems that if we
need a new PS we should be very clear why people are not encoraged to
use the existing PS that have been specified.

I do encourage you not to call the, "proposals" when they marked as
PS in the RFC series.

This didn't appear necessarily as an advantage to me: "

GUE permits encapsulation of arbitrary IP protocols, which
         includes layer 2 3, and 4 protocols." :-)

Is the claim:
  GUE provides a uniform and
         extensible mechanism for encapsulating all IP protocols in UDP
         with minimal overhead (four bytes of additional header).

Is saving a  few bytes so important? I'm curious also about how this 
interacts with offload, and I think you have quite some experience here.

The same as the next:
"

GUE is extensible. New flags and extension fields can be
         defined."

You say multiplexing different protocols over the same port is a 
feature. I'm sure this does not come without issues - can you elaborate 
these please?

If this is the target usage:
"

   o The GUE header includes a header length field. This allows a
         network node to inspect an encapsulated packet without needing
         to parse the full encapsulation header.

"
I'd also comment that the spec is pretty wide-ranging and complex so are 
you really saying this is the designed usage?

I'd have expected this to have raised some security concerns.

"

       o Private data in the encapsulation header allows local
         customization and experimentation while being compatible with
         processing in network nodes (routers and middleboxes).

"
- I'd have expected this to have raised at least some security concerns.

"

  GUE includes a variant that encapsulates IPv4 and IPv6 packets
         directly within UDP.
"
Well - it just places the packets in UDP payloads. How would ICMP messages be handled? How are extension headers handled?
Etc.

So, I'm objecting  to simply listing lots of RFCs and they saying yours is different.

>        o 101b, 110b, 111b - Reserved values
Ahhh - So do you mean the "paired" bits form a 2-bit field?
> protocol.

> Technically, GUE always carries an IP protocol (as opposed
> to GREoUDP that carries an EtherType).
Dies the Spec say this explicitly - please also discuss in the security 
considerations.
>   The congestion considerations
> or GRE.

> Those can be  encapsulations of non-IP protocols hence should
> have their own congestion considerations,
Ah I thought so, and how will this be handled?
> however neither RFC2784 nor
And does that introduce issues with NAT or NAPT and how do you handle
MSS etc across this sort of encaps?
> datagrams without checksums should be discarded or passed to the
That is allowed. For an endpoint, the app is permitted to decide
whether datagrams are passed to the application when there is
a zero checksum. How does GUE handle this request?
> imported from RFC8086 if it helps to clarify.
Please consider carefully the applicability. Are you going to change to 
an applicability statement of only to be used in controlled environments?
> Which section are you referring to (there is no section 5.8.2).
I suspect I intended 5.7.2 - but you already said you would update.

>> [RFC3828] type of checksum capability as an optional field of the
Is that wise?
>> - Elsewhere in the IETF fragmentation has been described as fragile, why
>> is it safe in this spec?
>>
> You interchanged the concepts of  "fragile" and "not safe"
Not really.
> -- they are
> are often dropped by intermediate nodes.
And serious attacks can be made against the reassembly engine. There is
also a quetsion of how much capabiluity your remote receiver is expected
to supply - what range of fragements and how many outstanding frag/packets
it needs to support so that the encaps knows the network reordering and loss
etc are compatible with how it is fragmenting.
> Fragmentation in an
> fragmentation than IPv4 fragmentation).
That also is true.
> As for being safe, it is as
The poblem - was that wrap of the ID space?
> application.
Yes. The application could be a tunnel, but that is also the transport.
> GUE does fragmentation in the encapsulation layer and
> UDP options have nothing to do with tunnels.
I do not know if the authors think this.
> I'd also point out that
OK
> Yes, only 0 to 127 are IANA assigned. Types 129 to 255 are site local.
Site-local may not be the correct term? If you expect two endpoints to 
agree,
but I understand they are not IANA assigned.
>> truism, that a receiver can not use data that it does not understand?
I'd encourage to review that text to rephase that MUST as something 
other than a requirement.
>> - Why is rate limiting only permitted by a "MAY" - should that be
I'd encourage to review that text.
>> as "proposals", but clearly differentiate the benefits of the new approach.
I'm objecting again to simply listing lots of RFCs and they saying yours 
is different,
this does not represent a clear statement of the problem that this 
particular draft is trying to solve.

At the end of this, I now conclude there are transport-related issues 
that would need to be addressed. Thanks again for your helpful replies,

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Wed, 13 Mar 2019 09:39:16 -0700

Hi Gorry,

Thanks for the reply. I will reply to comments in time. However, I
would note that some number of the comments are more about UDP
encapsulation than GUE specifically. draft-ietf-rtgwg-dt-encap-02
provides a good description of considerations for UDP encapsulation
(including NAT handling, middleboxes, next protocol header indication,
etc.). We'll add a reference for this in the draft.

Tom