Subject: Re: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

From: Norman Finn <[email-masked]>
Date: Mon, 7 Jan 2019 06:19:49 +0000

1. You may well need more than one DSCP for DetNet region for different classes of service.
2. You don't want to discard unexpected packets with a DetNet DSCP -- you want to remark them to a non-DSCP code point.  Not good if configuring (or misconfiguring) a QoS feature destroys a bystander's connectivity.

-- Norm

________________________________
From: detnet [[email-masked]] on behalf of Andrew G. Malis [[email-masked]]
Sent: Monday, December 24, 2018 12:26 PM
To: Lou Berger
Cc: detnet WG; Black, David; János Farkas
Subject: Re: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

Lou,

Thanks for following up.

Cheers,
Andy

On Mon, Dec 24, 2018 at 3:24 PM Lou Berger <[email-masked]<mailto:[email-masked]>> wrote:
Andy,

Just following up on one point on DetNet-specific DSCPs:

On 12/19/2018 1:11 PM, Andrew G. Malis wrote:
> In that case, the DetNet flows would most probably be a using
> DetNet-specific DSCP, and if the ingress interface isn't properly
> provisioned to know how to process packets with that DSCP, they'll at
> least be reduced to best-effort treatment.. We can also recommend that
> packets with DetNet-specific DSCPs be dropped if they arrive at an
> ingress interface that isn't otherwise provisioned to expect such packets.

While I personally think use of DSCPs will be useful in supporting
aggregated DetNet flows, the IP solution draft has no requirement that
operators or vendors restrict DetNet flows to a specific DSCP value.  --
I personally think this is an important design point.

Happy Holidays!

Lou

---

From: Norman Finn <[email-masked]>
Date: Mon, 7 Jan 2019 06:26:52 +0000

I'm good with a SHOULD in the first paragraph of 3.3.2.

-- Norm
________________________________________
From: detnet [[email-masked]] on behalf of Black, David [[email-masked]]
Sent: Wednesday, December 19, 2018 9:37 AM
To: János Farkas; DetNet WG
Subject: Re: [Detnet] draft-ietf-detnet-architecture-10

Janos,

The terminology is definitely improved - the resulting draft is much
clearer - many thanks to you and the authors for doing that.

Where do we stand on the concern about protecting external
non-DetNet networks from DetNet flows that escape a DetNet
network, e.g., due to an erroneous network configuration change?

My current views are ...

There's no throttling in DetNet, e.g., this text in 4.3.2:

   There is no provision in DetNet for throttling DetNet flows, i.e.,
   the transmission rate cannot be reduced via explicit congestion
   notification [RFC3168].  The assumption is that a DetNet flow, to be
   useful, must be delivered in its entirety.

and no assurance that all DetNet-using applications will cease and desist
in the face of packet drops that indicate congestion.

Packet Replication and Elimination makes this worse via its ability to hide
network packet drops and congestion indications from endpoints, e.g.,
of those drops and/or congestion indications occur on only one of
the paths used by replicated packets.

The first paragraph in section 3.3.2 is a good start, but I think its lower
case "should" is too weak:

   Robust real-time systems require to reduce the number of possible
   failures.  Filters and policers should be used in a DetNet network to
   detect if DetNet packets are received on the wrong interface, or at
   the wrong time, or in too great a volume.  Furthermore, filters and
   policers can take actions to discard the offending packets or flows,
   or trigger shutting down the offending flow or the offending
   interface.

At a minimum, I think that the requirement to discard DetNet traffic
that would otherwise escape a DetNet network has to be stated as
a "MUST" requirement for any DetNet network that supports Packet
Replication and Elimination due to the ability of that functionality
to hide congestion from endpoints, making it impossible for higher
layers of the protocol stack to respond.

Beyond this, I think a "SHOULD" requirement to discard DetNet traffic
that would otherwise escape is needed, and the statement of that
requirement ought to be connected to the explanation of how to
implement that requirement.  The text in Section 3.3.2 has a lot of
the content needed - for a worked example of such "SHOULD" text,
see Section 5 of RFC 7510, in particular the itemized list at the end of
that section and the paragraph that introduces that list:
   https://tools.ietf.org/html/rfc7510#section-5

Thanks, --David

> -----Original Message-----
> From: detnet [mailto:[email-masked]] On Behalf Of János Farkas
> Sent: Wednesday, December 19, 2018 10:54 AM
> To: DetNet WG
> Subject: [Detnet] draft-ietf-detnet-architecture-10
>
>
> [EXTERNAL EMAIL]
>
> Hi,
>
> The DetNet architecture document has been updated to address the
> comments. The new revision (v10) is available at
> https://tools.ietf.org/html/draft-ietf-detnet-architecture-10.
>
> There have been a number of discussion items emails on the list. The
> updates are along the major discussion items, which also cover the
> derivative discussions, address the comments.
>
> The major items have been summarized in:
> https://www.ietf.org/mail-archive/web/detnet/current/msg01968.html.
> Follow up discussions implied terminology changes summarized in:
> https://www.ietf.org/mail-archive/web/detnet/current/msg02036.html:
> replace "transport sub-layer" with "forwarding sub-layer"
> https://www.ietf.org/mail-archive/web/detnet/current/msg02038.html:
> replace "congestion protection" with "resource" allocation"
> Consequently, the document had a good 'scurb".
>
> In addition, outstanding comments have been addressed along
> https://www.ietf.org/mail-archive/web/detnet/current/msg01943.html.
>
> Please let us know if any comment happened to be remained open.
>
> Best regards,
> Janos
>
> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet

_______________________________________________
detnet mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Jan 2019 06:53:49 +0000

Dear all:

This creates an interesting side effect:

Since we are aiming at std track and using uppercase then we need to reference BCP 14. Janos, could you please add this snippet as the first section of the terminology?
Note that if we take that path we need to reconsider all lowercase use of the terms listed below.

-----------------------------

<section anchor='bcp' title="BCP 14">
<t>

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
    "OPTIONAL" in this document are to be interpreted as described in BCP 14
    <xref target="RFC2119"/><xref target="RFC8174"/> when, and only when, they
    appear in all capitals, as shown here.

</t>
</section>   <!-- end section "BCP 14" -->

---------------------------

Happy new year you all!

Pascal

> From: detnet <[email-masked]> On Behalf Of Norman Finn
> Sent: lundi 7 janvier 2019 07:27
> To: Black, David <[email-masked]>; János Farkas
> <[email-masked]>; DetNet WG <[email-masked]>
> The terminology is definitely improved - the resulting draft is much clearer -
> many thanks to you and the authors for doing that.
> 
> Where do we stand on the concern about protecting external non-DetNet
> networks from DetNet flows that escape a DetNet network, e.g., due to an
> erroneous network configuration change?
> 
> My current views are ...
> 
> and no assurance that all DetNet-using applications will cease and desist in the
> face of packet drops that indicate congestion.
> 
> network packet drops and congestion indications from endpoints, e.g., of those
> drops and/or congestion indications occur on only one of the paths used by
> replicated packets.
> 
> The first paragraph in section 3.3.2 is a good start, but I think its lower case
> "should" is too weak:
> 
> At a minimum, I think that the requirement to discard DetNet traffic that
> would otherwise escape a DetNet network has to be stated as a "MUST"
> requirement for any DetNet network that supports Packet Replication and
> Elimination due to the ability of that functionality to hide congestion from
> endpoints, making it impossible for higher layers of the protocol stack to
> respond.
> 
> Beyond this, I think a "SHOULD" requirement to discard DetNet traffic that
> would otherwise escape is needed, and the statement of that requirement
> ought to be connected to the explanation of how to implement that
> requirement.  The text in Section 3.3.2 has a lot of the content needed - for a
> worked example of such "SHOULD" text, see Section 5 of RFC 7510, in
> particular the itemized list at the end of that section and the paragraph that
> introduces that list:

---

From: Lou Berger <[email-masked]>
Date: Mon, 07 Jan 2019 06:09:59 -0500

Norm (Pascal),

Please see my response to David -- wrt to *not* introducing capitalization 
in this case/document.

Lou

----------
On January 7, 2019 1:27:31 AM Norman Finn <[email-masked]>

---

From: "Black, David" <[email-masked]>
Date: Mon, 7 Jan 2019 15:44:06 +0000

Lou,

Attempting to summarize what needs to be stated without using MUST/SHOULD/MAY:

[1] Sending DetNet traffic into networks that have not been provisioned in advance to handle that DetNet traffic has to be treated as a fault.  Egress traffic filters to prevent this from happening are strongly recommended at the edges of a DetNet networks and DetNet supporting networks.  In this context, the term 'provisioned' has a broad meaning, e.g., provisioning could be performed via an administrative decision that the downstream network has the available capacity to carry the DetNet traffic that is being sent into it.

[2] Sending non-congestion-controlled DetNet traffic (i.e., DetNet traffic that does not comply with BCP 41/RFC 2914) into a network that is not provisioned to handle that traffic has to be prevented in addition to being treated as a fault.  DetNet traffic that is not known to be congestion controlled has to be treated as non-congestion-controlled DetNet traffic for this requirement.  DetNet replicated traffic is always non-congestion-controlled even if the original DetNet flow is congestion-controlled because DetNet packet elimination can hide congestion indications (e.g., drops, ECN CE marks, increased latency) from endpoints.  The responsibility for dealing with these requirements in general, and replicated traffic in particular, is solution-specific, as there are MPLS mechanisms that cannot be applied to the IP solution, and moreover, the current IP solution doesn't support DetNet traffic replication.

Thanks, --David

> From: Lou Berger [mailto:[email-masked]]
> Sent: Tuesday, December 25, 2018 11:45 AM
> To: Black, David; János Farkas; DetNet WG
> Hi David,
> On 12/24/2018 8:29 PM, Black, David wrote:
> > Hi Lou,
> >
> > Excerpting key paragraphs ...
> >
> >> To me clear, you are suggesting that text be added to say that sending
> >> detnet traffic out a non-detnet interfaces must be treated as a fault.
> >> Is this correct?
> > Not exactly ...
> >
> >> If so, I think this is too broad considering that
> >> there are networks that can provided the required detnet forwarding
> >> sub-layer service that are not detnet service aware as is mentioned at
> >> the end of section 4.3.2. (I'll refer to such as "DetNet supporting
> >> networks" in the next paragraph.)
> > I agree with where this is headed - sending DetNet traffic into a network
> > that has not been provisioned to provide the required detnet forwarding
> > service to that traffic has to be treated as a fault.  In this statement,
> > "provisioned" should be read broadly, e.g., it could be realized via an
> > administrative decision that the DetNet supporting network has the
> > available capacity to carry the DetNet traffic that is being sent into it.
> agreed.
> >> This said, I think it would be reasonable to make it a requirement that
> >> DetNet nodes must not forward traffic that doesn't conform with BCP41
> >> over a network that is neither a Detnet network or a DetNet supporting
> >> network.  Although this would require a change to the DetNet service
> >> model, i.e.,  to know which traffic was conformant/non-conformant.
> > That's fine with one important twist:
> >
> > Any traffic that has been replicated by DetNet packet replication has to
> > be treated as not conforming with BCP41 because DetNet packet
> > elimination (of replicated packets) may hide congestion indications
> > (e.g., delay, ECN CE marks, drops) from the endpoints - a scenario of
> > particular concern involves the other flow of replicated packets
> > following a DetNet network path along which nothing goes wrong.
> > BCP41 is about end-to-end-congestion control, which does not do
> > anything if DetNet packet elimination hides congestion indications
> > from the endpoints.
> 
> This seems reasonable to me.
> 
> 
> >
> >>> implement that requirement.
> >> I think adding a statement on output filters (and policers) to this
> >> section is reasonable.
> > Let's start with the inclusion of output filters (and policers) in the 3.3.2
> > text with the use of upper-case "SHOULD" as suggested by Pascal - I will
> > look this over and suggest any additional text that I think ought to be
> > included.
> 
> As an architecture document, the authors/WG made the call to not use
> conformance capitalization or reference 2119/8174.  I think this
> translates to the conformance language needing to be added to the data
> plane solution documents.
> 
> Cheers,
> 
> Lou
> 
> 
> > Thanks, --David
> >
> >> From: Lou Berger [mailto:[email-masked]]
> >> Sent: Monday, December 24, 2018 3:18 PM
> >> To: Black, David; János Farkas; DetNet WG
> >> Hi David,
> >> On 12/19/2018 12:37 PM, Black, David wrote:
> >>> Janos,
> >>>
> >> Happy to hear that.
> >> I'm not sure we have general solutions in the internet architecture for
> >> miss-configured/buggy systems other than dropping the packets on the
> >> floor (e.g., due to no route, no label, or ttl expiry).  What type of
> >> protection are you looking for here, and can you provide a reference to
> >> the type of router behavior you would like to see?  I have a suggestion
> >> that might address your concern below elated to non-BCP41 conformant
> >> traffic, but am unsure if this is what you are concerned about.
> >>
> >>> My current views are ...
> >>>
> >> Well looking at it this way, there is no throttling in any IP network as
> >> the routers generate the CN and only drop when their queues are full or
> >> implementing some sort of RED.
> >>
> >> In DetNet there is an expectation that intermediate nodes do more than
> >> this.  Specifically, see section 4.5: "Queuing, Shaping, Scheduling, and
> >> Preemption" and policing, which is mentioned as requirement in section
> >> 3.3.1.
> >>
> >> I think this is correct.
> >> To me clear, your are suggesting that text be added to say that sending
> >> detnet traffic out a non-detnet interfaces must be treated as a fault.
> >> Is this correct?  If so, I think this is too broad considering that
> >> there are networks that can provided the required detnet forwarding
> >> sub-layer service that are not detnet service aware as is mentioned at
> >> the end of section 4.3.2. (I'll refer to such as "DetNet supporting
> >> networks" in the next paragraph.)
> >>
> >> This said, I think it would be reasonable to make it a requirement that
> >> DetNet nodes must not forward traffic that doesn't conform with BCP41
> >> over a network that is neither a Detnet network or a DetNet supporting
> >> network.  Although this would require a change to the DetNet service
> >> model, i.e.,  to know which traffic was conformant/non-conformant.
> >>
> >>
> >>> implement that requirement.
> >> I think adding a statement on output filters (and policiers) to this
> >> section is reasonable.
> >>
> >>
> >>> The text in Section 3.3.2 has a lot of
> >> If I'm not mistaken, such mechanisms are already included in the
> >> document and you really are just asking them to be (re)listed to ensure
> >> that this import point isn't missed.  Is this correct? Besides what is
> >> mentioned above, is there anything else missing?
> >>
> >> Thanks!
> >>
> >> Lou
> >>
> >>> Thanks, --David
> >>>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Jan 2019 15:53:47 +0000

Hello Lou:

I can buy not using uppercase; the standard snippet is still useful even when there are none actually. This is how I wrote https://tools.ietf.org/html/draft-ietf-6tisch-architecture-19#section-2.1 

Take care,

Pascal

> From: detnet <[email-masked]> On Behalf Of Lou Berger
> Sent: lundi 7 janvier 2019 12:10
> To: Norman Finn <[email-masked]>; Black, David
> <[email-masked]>; János Farkas <[email-masked]>; DetNet
> WG <[email-masked]>
> Please see my response to David -- wrt to *not* introducing capitalization in
> this case/document.
> 
> Lou
> 
> 
> ----------
> > networks from DetNet flows that escape a DetNet network, e.g., due to
> > an erroneous network configuration change?
> >
> > My current views are ...
> >
> > and no assurance that all DetNet-using applications will cease and
> > desist in the face of packet drops that indicate congestion.
> >
> > Packet Replication and Elimination makes this worse via its ability to
> > hide network packet drops and congestion indications from endpoints,
> > e.g., of those drops and/or congestion indications occur on only one
> > of the paths used by replicated packets.
> >
> > The first paragraph in section 3.3.2 is a good start, but I think its
> > lower case "should" is too weak:
> >
> > that would otherwise escape a DetNet network has to be stated as a
> > "MUST" requirement for any DetNet network that supports Packet
> > the content needed - for a worked example of such "SHOULD" text, see
> > Section 5 of RFC 7510, in particular the itemized list at the end of

---

From: Lou Berger <[email-masked]>
Date: Mon, 7 Jan 2019 16:49:21 -0500

David,

     This looks good/right to me.  The only specific change I'd add is 
to clarify that we're talking PEROF traffic when saying "DetNet 
replicated traffic".

Thanks!

Lou

---

From: Norman Finn <[email-masked]>
Date: Mon, 7 Jan 2019 22:35:08 +0000

Thanks.  I missed that one.  I'm also good with not adding a SHOULD.

-- Norm

________________________________________
From: Lou Berger [[email-masked]]
Sent: Monday, January 07, 2019 3:09 AM
To: Norman Finn; Black, David; János Farkas; DetNet WG
Subject: Re: [Detnet] draft-ietf-detnet-architecture-10

Norm (Pascal),

Lou

----------

---

From: "Black, David" <[email-masked]>
Date: Tue, 8 Jan 2019 17:46:10 +0000

That's fine and what was intended.

Thanks, --David

> Sent: Monday, January 7, 2019 4:49 PM
> > [1] Sending DetNet traffic into networks that have not been provisioned in
> advance to handle that DetNet traffic has to be treated as a fault.  Egress
> traffic filters to prevent this from happening are strongly recommended at the
> edges of DetNet networks and DetNet supporting networks.  In this context,
> the term 'provisioned' has a broad meaning, e.g., provisioning could be
> performed via an administrative decision that the downstream network has the
> > [2] Sending non-congestion-controlled DetNet traffic (i.e., DetNet traffic that
> does not comply with BCP 41/RFC 2914) into a network that is not provisioned
> to handle that traffic has to be prevented in addition to being treated as a fault.
> DetNet traffic that is not known to be congestion controlled has to be treated as
> non-congestion-controlled DetNet traffic for this requirement.  DetNet
> replicated traffic is always non-congestion-controlled even if the original
> DetNet flow is congestion-controlled because DetNet packet elimination can
> hide congestion indications (e.g., drops, ECN CE marks, increased latency) from
> endpoints.  The responsibility for dealing with these requirements in general,
> and replicated traffic in particular, is solution-specific, as there are MPLS
> mechanisms that cannot be applied to the IP solution, and moreover, the
> current IP solution doesn't support DetNet traffic replication.
> >
> > Thanks, --David
> >
> hide

---

From: Balázs Varga A <[email-masked]>
Date: Fri, 11 Jan 2019 09:18:37 +0000

Great. I am fine to add these explicit statements to make text more clear.

New text is inline with the general statement in the introduction of the draft:
"  DetNet is for networks that are under a single
   administrative control or within a closed group of administrative
   control; these include campus-wide networks and private WANs.  DetNet
   is not for large groups of domains such as the Internet.  "

Cheers
Bala'zs

Ps: Also, note that "DetNet flows" are unidirectional ... :---))))

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Black, David
Sent: Tuesday, January 8, 2019 6:46 PM
To: Lou Berger <[email-masked]>; Janos Farkas <[email-masked]>; DetNet WG <[email-masked]>
Subject: Re: [Detnet] draft-ietf-detnet-architecture-10

Thanks, --David

> advance to handle that DetNet traffic has to be treated as a fault.  
> Egress traffic filters to prevent this from happening are strongly 
> recommended at the edges of DetNet networks and DetNet supporting 
> networks.  In this context, the term 'provisioned' has a broad 
> meaning, e.g., provisioning could be performed via an administrative 
> decision that the downstream network has the available capacity to carry the DetNet traffic that is being sent into it.
> >
> does not comply with BCP 41/RFC 2914) into a network that is not 
> provisioned to handle that traffic has to be prevented in addition to being treated as a fault.
> DetNet traffic that is not known to be congestion controlled has to be 
> treated as non-congestion-controlled DetNet traffic for this 
> requirement.  DetNet replicated traffic is always 
> non-congestion-controlled even if the original DetNet flow is 
> congestion-controlled because DetNet packet elimination can hide 
> congestion indications (e.g., drops, ECN CE marks, increased latency) 
> from endpoints.  The responsibility for dealing with these 
> requirements in general, and replicated traffic in particular, is 
> solution-specific, as there are MPLS mechanisms that cannot be applied to the IP solution, and moreover, the current IP solution doesn't support DetNet traffic replication.
> >
> > Thanks, --David
> >
> >>>> To me clear, you are suggesting that text be added to say that 
> >>>> sending detnet traffic out a non-detnet interfaces must be treated as a fault.
> >>>> Is this correct?
> >>> Not exactly ...
> >>>
> >>>> If so, I think this is too broad considering that there are 
> >>>> networks that can provided the required detnet forwarding 
> >>>> sub-layer service that are not detnet service aware as is 
> >>>> mentioned at the end of section 4.3.2. (I'll refer to such as 
> >>>> "DetNet supporting networks" in the next paragraph.)
> >>> I agree with where this is headed - sending DetNet traffic into a 
> >>> network that has not been provisioned to provide the required 
> >>> detnet forwarding service to that traffic has to be treated as a 
> >>> fault.  In this statement, "provisioned" should be read broadly, 
> >>> e.g., it could be realized via an administrative decision that the 
> >>> DetNet supporting network has the available capacity to carry the DetNet traffic that is being sent into it.
> >> agreed.
> >>>> This said, I think it would be reasonable to make it a 
> >>>> requirement that DetNet nodes must not forward traffic that 
> >>>> doesn't conform with BCP41 over a network that is neither a 
> >>>> Detnet network or a DetNet supporting network.  Although this 
> >>>> would require a change to the DetNet service model, i.e.,  to know which traffic was conformant/non-conformant.
> >>> Any traffic that has been replicated by DetNet packet replication 
> >>> has to be treated as not conforming with BCP41 because DetNet 
> >>> packet elimination (of replicated packets) may hide congestion 
> >>> indications (e.g., delay, ECN CE marks, drops) from the endpoints 
> >>> - a scenario of particular concern involves the other flow of 
> >>> replicated packets following a DetNet network path along which nothing goes wrong.
> >>>>> Beyond this, I think a "SHOULD" requirement to discard DetNet 
> >>>>> traffic that would otherwise escape is needed, and the statement 
> >>>>> of that requirement ought to be connected to the explanation of 
> >>>>> how to implement that requirement.
> >>>> I think adding a statement on output filters (and policers) to 
> >>>> this section is reasonable.
> >>> Let's start with the inclusion of output filters (and policers) in 
> >>> the 3.3.2 text with the use of upper-case "SHOULD" as suggested by 
> >>> Pascal - I will look this over and suggest any additional text 
> >>> that I think ought to be included.
> >> As an architecture document, the authors/WG made the call to not 
> >> use conformance capitalization or reference 2119/8174.  I think 
> >> this translates to the conformance language needing to be added to 
> >> the data plane solution documents.
> >>
> >> Cheers,
> >>
> >> Lou
> >>
> >>
> >>> Thanks, --David
> >>>
> >>>>> The terminology is definitely improved - the resulting draft is 
> >>>>> much clearer - many thanks to you and the authors for doing that.
> >>>> Happy to hear that.
> >>>>
> >>>>
> >>>> I'm not sure we have general solutions in the internet 
> >>>> architecture for miss-configured/buggy systems other than 
> >>>> dropping the packets on the floor (e.g., due to no route, no 
> >>>> label, or ttl expiry).  What type of protection are you looking 
> >>>> for here, and can you provide a reference to the type of router 
> >>>> behavior you would like to see?  I have a suggestion that might 
> >>>> address your concern below elated to non-BCP41 conformant traffic, but am unsure if this is what you are concerned about.
> >>>>
> >>>>> My current views are ...
> >>>>>
> >>>> Well looking at it this way, there is no throttling in any IP 
> >>>> network as the routers generate the CN and only drop when their 
> >>>> queues are full or implementing some sort of RED.
> >>>>
> >>>> In DetNet there is an expectation that intermediate nodes do more 
> >>>> than this.  Specifically, see section 4.5: "Queuing, Shaping, 
> >>>> Scheduling, and Preemption" and policing, which is mentioned as 
> >>>> requirement in section 3.3.1.
> >>>>
> >>>>> and no assurance that all DetNet-using applications will cease 
> >>>>> and desist in the face of packet drops that indicate congestion.
> >>>> I think this is correct.
> >>>>
> >>>>
> >>>>> network packet drops and congestion indications from endpoints, 
> >>>>> e.g., of those drops and/or congestion indications occur on only 
> >>>>> one of the paths used by replicated packets.
> >>>>>
> >>>>> The first paragraph in section 3.3.2 is a good start, but I 
> >>>>> think its lower case "should" is too weak:
> >>>>>
> >>>>> At a minimum, I think that the requirement to discard DetNet 
> >>>>> traffic that would otherwise escape a DetNet network has to be 
> >>>>> stated as a "MUST" requirement for any DetNet network that 
> >>>>> supports Packet Replication and Elimination due to the ability 
> >>>>> of that functionality to hide congestion from endpoints, making 
> >>>>> it impossible for higher layers of the protocol stack to respond.
> >>>> To me clear, your are suggesting that text be added to say that 
> >>>> sending detnet traffic out a non-detnet interfaces must be treated as a fault.
> >>>> Is this correct?  If so, I think this is too broad considering 
> >>>> that there are networks that can provided the required detnet 
> >>>> forwarding sub-layer service that are not detnet service aware as 
> >>>> is mentioned at the end of section 4.3.2. (I'll refer to such as 
> >>>> "DetNet supporting networks" in the next paragraph.)
> >>>>
> >>>> This said, I think it would be reasonable to make it a 
> >>>> requirement that DetNet nodes must not forward traffic that 
> >>>> doesn't conform with BCP41 over a network that is neither a 
> >>>> Detnet network or a DetNet supporting network.  Although this 
> >>>> would require a change to the DetNet service model, i.e.,  to know which traffic was conformant/non-conformant.
> >>>>
> >>>>
> >>>>> Beyond this, I think a "SHOULD" requirement to discard DetNet 
> >>>>> traffic that would otherwise escape is needed, and the statement 
> >>>>> of that requirement ought to be connected to the explanation of 
> >>>>> how to implement that requirement.
> >>>> I think adding a statement on output filters (and policiers) to 
> >>>> this section is reasonable.
> >>>>
> >>>>
> >>>>> The text in Section 3.3.2 has a lot of the content needed - for 
> >>>>> a worked example of such "SHOULD" text, see Section 5 of RFC 
> >>>>> 7510, in particular the itemized list at the end of that section 
> >>>>> and the paragraph that introduces that list:
> >>>> document and you really are just asking them to be (re)listed to 
> >>>> ensure that this import point isn't missed.  Is this correct? 
> >>>> Besides what is mentioned above, is there anything else missing?
> >>>>
> >>>> Thanks!
> >>>>
> >>>> Lou
> >>>>
> >>>>> Thanks, --David
> >>>>>
> >>>>>> From: detnet [mailto:[email-masked]] On Behalf Of 
> >>>>>> János
> Farkas
> >>>>>> The DetNet architecture document has been updated to address 
> >>>>>> the comments. The new revision (v10) is available at 
> >>>>>> There have been a number of discussion items emails on the 
> >>>>>> list. The updates are along the major discussion items, which 
> >>>>>> also cover the derivative discussions, address the comments.
> >>>>>>

---

From: Balázs Varga A <[email-masked]>
Date: Fri, 11 Jan 2019 09:36:43 +0000

Hi Norm,

> 1. You may well need more than one DSCP for DetNet region for different classes of service.
Yes, and we have defined the 6-tupple match for flow identification, what allows multiple DSCP
values.

> 2. You don't want to discard unexpected packets with a DetNet DSCP -- you want to remark
> them to a non-DSCP code point.  Not good if configuring (or misconfiguring) a QoS feature
> destroys a bystander's connectivity.
That is an interesting topic. What do You mean by "unexpected packet"? If it breaks the
traffic profile we must drop. I am not sure remarking is a good idea. What do You mean by
"non-DSCP code point"?

General, I think configuration errors may destroy DetNet (and any other) flows connectivity.
We have to live with that. You can mitigate them e.g., by "proper maintenance procedures",
etc. That points to the importance of DetNet OAM being able to detect configuration (and
other) errors.

Cheers
Bala'zs

From: detnet <[email-masked]> On Behalf Of Norman Finn
Sent: Monday, January 7, 2019 7:20 AM
To: Andrew G. Malis <[email-masked]>; Lou Berger <[email-masked]>
Cc: detnet WG <[email-masked]>; Janos Farkas <[email-masked]>; Black, David <[email-masked]>
Subject: Re: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

-- Norm

Thanks for following up.

Cheers,
Andy

Happy Holidays!

Lou

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 11 Jan 2019 19:16:19 +0100

I really don't know what the benefit of this text would be. IMHO it is
mostly IESG code language but not really helpfull IMHO.

Something like Googles worldwide network could support DetNet if Google
wanted to ?! Its a private WAN, right ? Aka: there is no physcial scale
limitation, right ? Why don't we say that ?

What is a "large number of domains" ? How about a nationwide research
network such as Internet 2 with hundreds of attached organizations. If
Internet2 introduced DetNet services and allowed interested Internet2
participants to participate in it, extending DetNet services into their
couses ? How about a nationwide service provider and its customers ?
We do want DetNet service, right ?

The only real requirement for DetNet between A and B is that all transit
domains between A and B support DetNet. This is fundamentally not
different from the requirement to make ECN work. As soon as i have a
single hop along a path not supporting ECN, my ECN based congestion
control will have to fall back to delay-variation or loss-based. And ECN is
defined to be an Internet technology. So why would DetNet not be ?

"When we finally get to DetNet Control Plane, we don't want to bother
with all the complexity of multi-operator control plane because its
hard enough to figure out how to make one DetNet SDN controller work"

Thats one possible honest exlanation, but given how i think we can not
precdit what type of multi-opeator environments control plane the
community will have interest to work on over time, it might be more
appropriate to say that and not to come up with an underspecified binary
distinction of what is and what is not DetNet.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 11 Jan 2019 16:44:23 -0500

Toerless,

We could certainly truthfully say that it's not intended for the
multidomain best-effort public Internet infrastructure.

Cheers,
Andy

On Fri, Jan 11, 2019 at 1:16 PM Toerless Eckert <[email-masked]> wrote:

> > To: Lou Berger <[email-masked]>; Janos Farkas <
> [email-masked]>; DetNet WG <[email-masked]>
> > > decision that the downstream network has the available capacity to
> carry the DetNet traffic that is being sent into it.
> > > >
> > > solution-specific, as there are MPLS mechanisms that cannot be applied
> to the IP solution, and moreover, the current IP solution doesn't support
> DetNet traffic replication.
> > > >
> > > > Thanks, --David
> > > >
> > > >>> DetNet supporting network has the available capacity to carry the
> DetNet traffic that is being sent into it.
> > > >> agreed.
> > > >>>> would require a change to the DetNet service model, i.e.,  to
> know which traffic was conformant/non-conformant.
> > > >>>> address your concern below elated to non-BCP41 conformant
> > > >>>> would require a change to the DetNet service model, i.e.,  to
> know which traffic was conformant/non-conformant.
> > > >>>>
> > > >>>>

---

From: Norman Finn <[email-masked]>
Date: Sun, 13 Jan 2019 22:44:21 +0000

We may have been talking about different things, and I mistyped.

I identified DetNet flows that exceed bandwidth must be dropped.

Traffic that cannot be identified as a DetNet flow, but have a DetNet DSCP, may need to be marked down.  For example, a host that is not authorized to transmit DetNet traffic sends something with the DetNet DSCP.  In a network using CQF, you don't need to identify every packet's flow by 6-tuple; the DSCP is sufficient to assign the packet to the right DetNet queue.  You may well do 6-tuple flow identification at the edge, only.  So, a non-DetNet packet with a DetNet DSCP has to be remarked.  That's the case I was talking about.

-- Norm

________________________________
From: Balázs Varga A [[email-masked]]
Sent: Friday, January 11, 2019 1:36 AM
To: Norman Finn; Andrew G. Malis; Lou Berger
Cc: detnet WG; Janos Farkas; Black, David
Subject: RE: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

Hi Norm,

> destroys a bystander's connectivity.
That is an interesting topic. What do You mean by “unexpected packet”? If it breaks the
traffic profile we must drop. I am not sure remarking is a good idea. What do You mean by
“non-DSCP code point”?

General, I think configuration errors may destroy DetNet (and any other) flows connectivity.
We have to live with that. You can mitigate them e.g., by “proper maintenance procedures”,
etc. That points to the importance of DetNet OAM being able to detect configuration (and
other) errors.

Cheers
Bala’zs

-- Norm

Thanks for following up.

Cheers,
Andy

Happy Holidays!

Lou

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 14 Jan 2019 06:12:01 +0000

Hi,
Yes we can, as it was stated in the charter:
“
The Working Group will initially focus on solutions for networks that
are under a single administrative control or within a closed group of
administrative control; these include not only campus-wide networks but
also can include private WANs. The DetNet WG will not spend energy on
solutions for large groups of domains such as the Internet.
”
Cheers
Bala’zs

From: Andrew G. Malis <[email-masked]>
Sent: Friday, January 11, 2019 10:44 PM
To: Toerless Eckert <[email-masked]>
Cc: Balázs Varga A <[email-masked]>; DetNet WG <[email-masked]>; Black, David <[email-masked]>; Janos Farkas <[email-masked]>; Lou Berger <[email-masked]>
Subject: Re: [Detnet] draft-ietf-detnet-architecture-10

Toerless,

We could certainly truthfully say that it's not intended for the multidomain best-effort public Internet infrastructure.

Cheers,
Andy

On Fri, Jan 11, 2019 at 1:16 PM Toerless Eckert <[email-masked]<mailto:[email-masked]>> wrote:
I really don't know what the benefit of this text would be. IMHO it is
mostly IESG code language but not really helpfull IMHO.

Cheers
    Toerless

On Fri, Jan 11, 2019 at 09:18:37AM +0000, Bal?zs Varga A wrote:
> From: detnet <[email-masked]<mailto:[email-masked]>> On Behalf Of Black, David
> To: Lou Berger <[email-masked]<mailto:[email-masked]>>; Janos Farkas <[email-masked]<mailto:[email-masked]>>; DetNet WG <[email-masked]<mailto:[email-masked]>>
> > From: Lou Berger [mailto:[email-masked]<mailto:[email-masked]>]
> > >> From: Lou Berger [mailto:[email-masked]<mailto:[email-masked]>]
> > >>>> From: Lou Berger [mailto:[email-masked]<mailto:[email-masked]>]
> > >>>>>> From: detnet [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
> > >>>>>> János
> > Farkas
> > >>>>>> [email-masked]<mailto:[email-masked]>
> > >>>>> [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>

--
---
[email-masked]<mailto:[email-masked]>

_______________________________________________
detnet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/detnet

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 14 Jan 2019 06:19:50 +0000

Hi Norm,
OK, now I see your case better. Thanks for the clarification.
In such a case remarking is OK. :--)))
Cheers
Bala'zs

From: Norman Finn <[email-masked]>
Sent: Sunday, January 13, 2019 11:44 PM
To: Balázs Varga A <[email-masked]>; Andrew G. Malis <[email-masked]>; Lou Berger <[email-masked]>
Cc: detnet WG <[email-masked]>; Janos Farkas <[email-masked]>; Black, David <[email-masked]>
Subject: RE: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

-- Norm

Cheers
Bala'zs

From: detnet <[email-masked]<mailto:[email-masked]>> On Behalf Of Norman Finn
Sent: Monday, January 7, 2019 7:20 AM
To: Andrew G. Malis <[email-masked]<mailto:[email-masked]>>; Lou Berger <[email-masked]<mailto:[email-masked]>>
Cc: detnet WG <[email-masked]<mailto:[email-masked]>>; Janos Farkas <[email-masked]<mailto:[email-masked]>>; Black, David <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Detnet] IP Solution and use of DetNet-specific DSCPs (was Re: draft-ietf-detnet-architecture-10)

-- Norm

Thanks for following up.

Cheers,
Andy

Happy Holidays!

Lou

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 26 Jan 2019 21:44:54 +0100

Sure, but there is no useful technical explanation in the DetNet
documents why those are out of scope, and even less so text to make
readers understand what actually is in scope.

This was the type of text i was suggesting in my first email:

"The more operators are involved in the end-to-end path, the more
 difficult it will be to guarantee well-coordinated hop-by-hop
 guarantee of DetNet properties" - That would be an explanation
 of calling multi-operator networks to be out of scope.

But what is a "private WAN" ? If i would guess then every reader
would interpret this differently. I for once think that DetNet
should be possible to offer by SPs as an extension of L3VPN services.
But most readers likely think thats not a private WAN. So instead
they would build DetNet on top of ethernet pseudowire services they
buy from some ISP. Of course: ROTFL, there is no fundamental difference
except that in the ethernet case, the enterprise might be lucky and
find an actual TSN enabled metro-ethernet service. Whereas the
enterprise wanting to use DetNet and the ISP wanting to offer it
as part of L3VPN read the DetNet RFCs and wonder what it means.

Aka: I can easily see that its easier to build a larger scale
TSN than DetNet service if i was trying to interpret the applicability
statements in DetNet.

So, i continue to claim that the current text rather has the opportunity
to stop DetNet in its tracks and can only serve to attempt to fend of
solving any issues that might be brough up. There is not even a
statement that DetNet should be supported in at least those scenarios
where TSN would be feasible, if not for the need to support L3 or
MPLS.

Cheers
    Toerless

On Mon, Jan 14, 2019 at 06:12:01AM +0000, Bal?zs Varga A wrote:
> ???
> ???
> Bala???zs

-- 
---
[email-masked]