Subject: [Teas] A review of draft-ietf-teas-enhanced-vpn

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 3 Sep 2019 19:29:45 +0100

Hi authors,

Time for a careful review of draft-ietf-teas-enhanced-vpn.

Lots of small points. Nothing alarming.

Hope this helps.

Best,
Adrian

===

idnits notes...

  == Unused Reference: 'RFC2119' is defined on line 1476, but no explicit
     reference was found in the text

  == Outdated reference: draft-ietf-detnet-use-cases has been published as
     RFC 8578

---

Abstract

OLD
Enhanced Virtual Private Networks (VPN+)
NEW
Enhanced Virtual Private Network (VPN+)
END

OLD
5G services by utilizing
NEW
5G services, by utilizing
END

---

Section 1

OLD
   Customers of a network operator may request enhanced overlay services
   with advanced characteristics such as complete isolation from other
   services so that changes in network load or event of other services
   have no effect on the throughput or latency of the service provided
   to the customer.
NEW
   Customers of a network operator may request enhanced overlay services
   with advanced characteristics such as complete isolation from other
   services so that changes in one service (such as changes in network
   load, or events such as congestion or outages) have no effect on the
   throughput or latency of other services provided to the customer.
END

OLD
   Network slicing is an approach to network operations that builds on
   the concept of network abstraction to provide programmability,
   flexibility, and modularity.
NEW
   Network slicing builds on the concept of resource management,
   network virtualization and abstraction to provide performance assurance, 
   flexibility, programmability and modularity.
END

How about adding some text to provide a normative definition of Network
Slicing? Since this draft merged in work from
draft-king-teas-applicability-actn-slicing, it would be OK to:
ADD
   Network Slice: An agreement between a consumer and a service
   provider to deliver network resources according to a specific service
   level agreement. A slice could span multiple technology (e.g., radio,
   transport and cloud) and administrative domains.

   A transport network slice construct provides an end-to-end logical
   network, often with compute functions and utilising shared underlying
   (physical or virtual) network resources.  This logical network is
   separated from other, often concurrent, logical networks each with
   independent control and management, and each of which can be created
   or modified on demand.
END

Para 5
   Network Function Virtualization (NFV)
Could use a reference here. Possibly RFC 8568?

   Additionally, the tenant may
   ask for some level of control to their virtual networks
s/to/of/

OLD
VPN+ is built on a virtual
NEW
VPN+ is built from a virtual
END

OLD
   strict guaranteed performance.
NEW
   strict performance guarantees.
END

   o  The necessary protocols in both the underlay and the overlay of
      enhanced VPN.
s/enhanced/the enhanced/

   Operation, Administration and Management (OAM)
s/ and/, and/

   (SLA) are met
s/are/is/

OLD
   The required network layered structure
NEW
   The required layered network structure
END

---

Section 2.1

s/Such feature/Such a feature/

OLD
   The terms hard and soft isolation are introduced to give example of
   different isolation cases.
NEW
   The terms hard and soft isolation are introduced to identify the
   different isolation cases.
END

OLD
   The requirement is for an operator to provide both hard and soft
   isolation between the tenants/applications using one enhanced VPN and
   the tenants/applications using another enhanced VPN.
NEW
   The requirement is for an operator to offer its customers a choice of
   different degrees of isolation ranging from soft isolation up to hard
   isolation so that the traffic of tenants/applications using one enhanced
   VPN can be separated from the traffic of tenants/applications using
   another enhanced VPN appropriately.
END

OLD
   An example of hard isolation is a network supporting both emergency
   services and public broadband multi-media services.
NEW
   An example of the requirement for hard isolation is a network
   supporting both emergency services and public broadband multi-media
   services.
END

   In order to provide the required isolation, resources may have to be
   reserved in the data plane of the underlay network and dedicated to
   traffic from a specific VPN or a specific group of VPNs.  This may
   introduce scalability concerns, thus some trade-off needs to be
   considered to provide the required isolation between network slices
   while still allowing reasonable sharing inside each network slice.
The second sentence is "suddenly" talking about slices where everything
so far has talked about isolation in terms of VPNs. So, how about...
   In order to provide the required isolation, resources may have to be
   reserved in the data plane of the underlay network and dedicated to
   traffic from a specific VPN or a specific group of VPNs to form
   slices in the underlay network.  This may introduce scalability
   concerns, thus some trade-off needs to be considered to provide the
   required isolation between network slices while still allowing
   reasonable sharing inside each network slice.

---

Section 2.1.1

OLD
   Thus, for example, using FlexE or a virtual sub-interface together
   with packet scheduling as isolation mechanism of interface resources,
NEW
   Thus, for example, using FlexE or a virtual sub-interface together
   with packet scheduling as the isolation mechanism for interface
   resources,
END

s/or may need/or may need a/

---

Section 2.2

s/errors is already/errors already/

---

Section 2.3

   A solution to the enhanced VPN problem has to provide close
   integration of both overlay VPN and the underlay network resource.

I think this could use a reason: why does the VPN solution have to 
provide close integration?

s/resource/resources/

How about replacing the above with:
   The only way to achieve the enhanced characteristics provided by an
   enhanced VPN (such as guaranteed or predicted performance) is by
   integrating the overlay VPN with a particular set of network resources
   in the underlay network.

---

Section 2.4

s/particular the traffic in/particular whether the traffic in/

s/disrupted can/disrupted, can/

   Dynamic changes both to the VPN and to the underlay transport network
   need to be managed to avoid disruption to sensitive services.
"Sensitive" is a difficult word. I wonder if we can clarify either as:
   Dynamic changes both to the VPN and to the underlay transport network
   need to be managed to avoid disruption to services that are sensitive to
   the change of network performance?

---

Section 2.5

I see why you have a reference to Section 4.5. I wonder whether you
should also have a reference to Section 4.4.

---

Section 2.6

s/number types/number of types/

Should you add "network slicing" to the list, or maybe remove VN from this
list?

---

Section 2.7

Although there is a very brief mention of "domain" in Section 1, this
is really the first proper discussion of domains. I wonder whether we
need a bit of a definition of "domain", or some examples.

---

Section 3

OLD
3.  Architecture of Enhanced VPN
NEW
3.  Architecture of Enhanced VPNs
END

   o  A enhanced data plane
s/A/An/

In the bullet lists in this section, should we have "telemetry" and "oam"?
Not sure which list you would put them in.  

Maybe make a third list in Section 3 to contain OAM and telemetry.

You do have a section (6) for OAM: maybe also need a new section for
telemetry, or a new subsection of section 6?

---

Section 3.1

   These techniques can be used to provision the virtual networks with
   dedicated resources that they need.  To get the required
s/dedicated/the dedicated/

   accordingly.  However within a virtual network these resources can if
   required be managed via a dynamic control plane.  This provides the
   required scalability and isolation.
s/ if required/, if required,/

---

Section 3.2

OLD
3.2.  Multi-Point to Multi-Point
NEW
3.2.  Multi-Point to Multi-Point (MP2MP)
END

   At the VPN service level, the connectivity are usually mesh or
   partial-mesh.  To support such kind of VPN service, the corresponding
s/are/is/
s/kind/kinds/

---

Section 3.4

   VPNs are instantiated as overlays on top of an operators network and
   offered as services to the operators customers.  An important feature
s/operators/operator's/ (twice)

OLD
   because traditional VPN would be enough for most existing
   services.
NEW
   because pre-existing VPN techniques would be good enough to meet the
   needs of most existing VPN-type services.
END

   In general, it is not required that the state in the network to be
s/to be/be/

   share a same virtual network and the same set of network resources
s/a/the/

---

Section 4

   We cannot for example
   share the tunnel between enhanced VPNs which require hard isolation.

Firstly, some nits...

   We cannot, for example,
   share a tunnel between enhanced VPNs which require hard isolation.

But, I think this is also not quite true with hierarchical tunnels. But,
I'm not sure how to say that in this section.

When I talk about hierarchical tunnels, I'm referring to VPN traffic
encapsulated in MPLS (i.e., tunnelled), and then bundled within an MPLS-TE
tunnel. You can see an RSVP-TE tunnel as a virtual link (with its bandwidth
well defined by the reservation when the LSP was set up). Then other RSVP-TE
LSPs can be signalled down the tunnel and make reservations out of the
tunnel bandwidth. Of course, those reservations are only applied at the
tunnel end points, but that's OK because the reservation will apply down the
whole length of the tunnel. If one of those LSPs oversteps its allotted
bandwidth there will be a problem, and that requires some form of handling
at the head end (it is either policing or treating over-subscription as best
effort), but this behaviour is exactly the same with a physical link.

---

Sections 4.1 and 4.2

The two layers you describe are "underlay" and "network" layers. I think
the name "network layer" may be generally confusing and elsewhere in the
document you talk about the "overlay".

Maybe this could be helped with a new figure in Section 3 to show the
different layers and their roles in delivering the service.

It might also help if section 4.2 included a sentence explaining the
concept.

In summary, I think section 4.1 is about the Layer 2 packet data plane,
section 4.2 is about the Layer 3 data plane, and section 4.3 is about
non-packet data plane technologies. 

---

Section 4.1.1

   multiple slower links  

I think "low capacity" is a better term. The use of "slow" in relation
to links is in common usage, but it feels to me that the speed of
transmission is not the same as the capacity of the link.

---

Section 4.1.1

   for enhanced VPN is for further study.

s/is/are/

---

Section 4.1.2

The paragraph in this section worries me a bit.

It sounds as though you are saying that DiffServ falls a long way short
of providing enough separation for VPN+. This feels like a 
contradiction with the scoping text that implies "only a small number"
of enhanced VPNs will be needed.

Maybe this could be qualified as something like "DiffServ doesn't always
provide enough class-based queues". You could even include some specific
numbers for
IP and MPLS - that would give you a much better cause for your
suggestions.

Furthermore, you could say that "DiffServ, as currently implemented, mainly
provides relative priority-based scheduling, and is difficult to achieve
quantitive resource reservation.

You also have
   Routers usually have large amount of queues
   and sophisticated queuing systems
Maybe you could say,
   Some routers have large amount of queues
   and sophisticated queuing systems

---

Section 4.2.1

OLD
the chance of all packets being lost
NEW
the chance of all copies of a packet being lost
END

---

Section 4.2.3

This section is surprisingly long compared to 4.2.1 and 4.2.2.

I don't think that adding text to the earlier sections is a good solution,
so I wonder whether anything can be trimmed from this section.

A way to look at this is to re-read 4.2.1 and 4.2.2 and absorb the level of
what they are saying (in terms of overview and introduction) and try to just
say something similar in this section.

---

Section 4.3

   management techniques such as ACTN ([RFC8453] and Section 4.4) can be

I think this should be 4.6

---

Section 4.6.1

Figure 3 is not referenced from the text and is, as far as I can see, a
very close model of Figure 2 from RFC 8453. Perhaps it could simply be 
left out and we can rely on Figure 4 which is much more important for
this document.

Figure 4 seems to be missing a line on the top of the top-most VPN+

---

Should we have a new Section 5.3?

5.3  SDN Scaling

   The centralized approach of SDN requires state to be stored in the
   network, but does not have the overhead of also requiring control
   plane state to be maintained.  Each individual network node may need
   to maintain a communication channel with the SDN controller, but that
   compares favourably with the need for a control plane to maintain 
   communication with all neighbors.

   However, SDN may transfer some of the scalability concerns from the
   network to the centralized controller.  In particular, there may be
   a heavy processing burden at the controller, and a heavy load in the
   network surrounding the controller.

---

Section 6

   A study of OAM in SR networks has been documented in [RFC8403].

I think you should move this sentence from the top to the bottom of the
section.

---

I have a note on file that we are supposed to add a section on
Operational Considerations.  I can't recall what prompted the note, but
it sounds like a good idea!  I suppose it would come in around Section 7

Maybe at least add the section with "TBD in a future revision"

You could look at RFC 5706 for general advice and RFC 6123 for more specific
advice. This is advice about Manageability Considerations and not about
Operational Considerations, but it could offer some ideas. My favourite
example of a Manageability Considerations section is in RFC 5220.

---

Section 8

I think I would add a further paragraph.

   While an enhanced VPN service may be sold as offering encryption and
   other security features as part of the service, customers would be
   well advised to take responsibility for their own security
   requirements themselves possibly by encrypting traffic before handing
   it off to the service provider.

Maybe, with the privacy concerns, we could also add...

   The privacy of enhanced VPN service customers must be preserved.  It
   should not be possible for one customer to discover the existence of
   another customer, nor should the sites that are members of an
   enhanced VPN be externally visible.

---

Section 12.

I wonder whether more of the references are normative. Since this is an
Informational document, it doesn't matter if they are normative, so we
can afford to move some of them. The test should be: is it necessary to
read a referenced document to understand this document, or does the
referenced document just provide additional information?

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 4 Sep 2019 12:00:11 +0000

Hi Adrian, 

Thanks a lot for your detailed review and comments. Most of the comments and suggestions will be incorporated in a new revision of this document. And please see some replies inline:

> -----Original Message-----
> From: Adrian Farrel [mailto:[email-masked]]
> Sent: Wednesday, September 04, 2019 2:30 AM
> To: [email-masked]
> Cc: 'TEAS WG' <[email-masked]>
> Subject: A review of draft-ietf-teas-enhanced-vpn
> 
> Hi authors,
> 

Thanks for catching this. Will fix in next revision.

> ---
> 
> Abstract
> 
> OLD
> Enhanced Virtual Private Networks (VPN+) NEW Enhanced Virtual Private
> Network (VPN+) END
> 
> OLD
> 5G services by utilizing
> NEW

The suggested text looks good.

>    flexibility, programmability and modularity.

The suggested text looks good.
>    independent control and management, and each of which can be created

Good suggestion. Will build the definition based on the proposed text. 

> 
> Para 5
>    Network Function Virtualization (NFV) Could use a reference here. Possibly
> RFC 8568?
> 
>    ask for some level of control to their virtual networks s/to/of/
> 
> OLD
>    Operation, Administration and Management (OAM) s/ and/, and/
> 
>    (SLA) are met
> s/are/is/
> 
> OLD
>    The required network layered structure NEW
>    The required layered network structure END
> 
> ---
> 
> Section 2.1
> 
> The second sentence is "suddenly" talking about slices where everything so far
> has talked about isolation in terms of VPNs. So, how about...

Agree this needs to be revised to clarify the relationship between VPN overlay and network slice. Will revise based on your proposed text.

>    with packet scheduling as isolation mechanism of interface resources, NEW
> I think this could use a reason: why does the VPN solution have to provide close
> integration?
> 
> s/resource/resources/
> 

The suggested text looks good, thanks.

>    the change of network performance?

The suggested text looks good.

> 
> ---
> 
> Section 2.5
> 
> I see why you have a reference to Section 4.5. I wonder whether you should
> also have a reference to Section 4.4.

OK, will add the reference in new revision.

> Should you add "network slicing" to the list, or maybe remove VN from this list?

In my understanding, the list here is the overlay VPNs which can be used as components to provide network slice or enhanced VPN. Thus VN may be removed from this list. 

> ---
> 
> Section 2.7
> 
> Although there is a very brief mention of "domain" in Section 1, this is really the
> first proper discussion of domains. I wonder whether we need a bit of a
> definition of "domain", or some examples.

OK, I will add some definition or description of domain to this section.

> telemetry, or a new subsection of section 6?

Yes we have a section for OAM, it is reasonable to also add it to the list in section 3. And telemetry will be added either as a new section or a subsection of section 6. 

>    dedicated resources that they need.  To get the required s/dedicated/the
> dedicated/
> 
> 3.2.  Multi-Point to Multi-Point (MP2MP) END
> 
> s/are/is/ s/kind/kinds/
>    In general, it is not required that the state in the network to be s/to be/be/
> 
> But, I think this is also not quite true with hierarchical tunnels. But, I'm not sure
> how to say that in this section.
> 
> tunnel. You can see an RSVP-TE tunnel as a virtual link (with its bandwidth well
> defined by the reservation when the LSP was set up). Then other RSVP-TE LSPs
> can be signalled down the tunnel and make reservations out of the tunnel
> bandwidth. Of course, those reservations are only applied at the tunnel end
> points, but that's OK because the reservation will apply down the whole length
> of the tunnel. If one of those LSPs oversteps its allotted bandwidth there will be
> a problem, and that requires some form of handling at the head end (it is either
> policing or treating over-subscription as best effort), but this behaviour is
> exactly the same with a physical link.

We may have some further discussion about this. 

If all the traffic are from the same ingress and to the same egress along the same path, and the resource requirement has been taken into consideration in RSVP-TE resource reservation, it may be OK to just perform policing and remarking at the ingress node, although it is possible that traffic burst can still happen due to the different interface throughput along the path. The problem is per-TE LSP states still need to be maintained on intermediate nodes to make sure the resource reservation information is updated correctly. 

If there is traffic which comes from different ingress and to different egress merge at the some links in the middle, resource reservation will become more complicated and the RSVP-TE based hierarchical tunnels may not be suitable.

> ---
> 
> Sections 4.1 and 4.2
> 
> The two layers you describe are "underlay" and "network" layers. I think the
> name "network layer" may be generally confusing and elsewhere in the
> Maybe this could be helped with a new figure in Section 3 to show the different
> layers and their roles in delivering the service.
> 
> It might also help if section 4.2 included a sentence explaining the concept.
> 
> In summary, I think section 4.1 is about the Layer 2 packet data plane, section
> 4.2 is about the Layer 3 data plane, and section 4.3 is about non-packet data
> plane technologies.

That's a lot for the summary, it is much clearer with the proposed text.

> 
> ---
> 
> Section 4.1.1
> 
>    multiple slower links
> 
> I think "low capacity" is a better term. The use of "slow" in relation to links is in
> common usage, but it feels to me that the speed of transmission is not the
> same as the capacity of the link.

Agreed, will change to low capacity in the new revision.

> It sounds as though you are saying that DiffServ falls a long way short of
> providing enough separation for VPN+. This feels like a contradiction with the
> scoping text that implies "only a small number"
> numbers for IP and MPLS - that would give you a much better cause for your
> suggestions.
> 
> provides relative priority-based scheduling, and is difficult to achieve quantitive
> resource reservation.
> 
> You also have

We could mention the limited number of traffic classes in IP and MPLS, and modify the statement about DiffServ and router implementations as you suggested.

> the chance of all copies of a packet being lost END
> 
> ---
> 
> Section 4.2.3
> 
> I don't think that adding text to the earlier sections is a good solution, so I
> wonder whether anything can be trimmed from this section.
> 
> A way to look at this is to re-read 4.2.1 and 4.2.2 and absorb the level of what
> they are saying (in terms of overview and introduction) and try to just say
> something similar in this section.

Will make some change as you suggested. 

> Figure 3 is not referenced from the text and is, as far as I can see, a very close
> model of Figure 2 from RFC 8453. Perhaps it could simply be left out and we can
> rely on Figure 4 which is much more important for this document.
> 

Thanks for catching this.

>    network surrounding the controller.

Good suggestion, will add this new section to the draft.

> I have a note on file that we are supposed to add a section on Operational
> Considerations.  I can't recall what prompted the note, but it sounds like a
> good idea!  I suppose it would come in around Section 7
> 

Thanks for the pointers, will go through these drafts to find some guidance.

>    enhanced VPN be externally visible.

Suggestions about security section are always appreciated:)

> Informational document, it doesn't matter if they are normative, so we can
> afford to move some of them. The test should be: is it necessary to read a
> referenced document to understand this document, or does the referenced
> document just provide additional information?

Thanks for the suggestion, will go through the reference list and move some of them.

Best regards,
Jie

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 4 Sep 2019 14:12:23 +0100

Looks good, Jie.

We may have some more discussions about tunnels, the need (or not) to
maintain state for tunnel payload at transit nodes, and the implications of
tunnel merging. But that is a relatively small point.

Best,
Adrian

-----Original Message-----
From: Dongjie (Jimmy) <[email-masked]> 
Sent: 04 September 2019 13:00
To: [email-masked]; [email-masked]
Cc: 'TEAS WG' <[email-masked]>
Subject: RE: A review of draft-ietf-teas-enhanced-vpn

Hi Adrian, 

Thanks a lot for your detailed review and comments. Most of the comments and
suggestions will be incorporated in a new revision of this document. And

>    reasonable sharing inside each network slice.

Agree this needs to be revised to clarify the relationship between VPN
overlay and network slice. Will revise based on your proposed text.

NEW

list?

In my understanding, the list here is the overlay VPNs which can be used as
components to provide network slice or enhanced VPN. Thus VN may be removed
from this list. 

> telemetry, or a new subsection of section 6?

Yes we have a section for OAM, it is reasonable to also add it to the list
in section 3. And telemetry will be added either as a new section or a
subsection of section 6. 

>    dedicated resources that they need.  To get the required
s/dedicated/the
> But, I think this is also not quite true with hierarchical tunnels. But,
I'm not sure

If all the traffic are from the same ingress and to the same egress along
the same path, and the resource requirement has been taken into
consideration in RSVP-TE resource reservation, it may be OK to just perform
policing and remarking at the ingress node, although it is possible that
traffic burst can still happen due to the different interface throughput
along the path. The problem is per-TE LSP states still need to be maintained
on intermediate nodes to make sure the resource reservation information is
updated correctly. 

If there is traffic which comes from different ingress and to different
egress merge at the some links in the middle, resource reservation will
become more complicated and the RSVP-TE based hierarchical tunnels may not
be suitable.

> The two layers you describe are "underlay" and "network" layers. I think
the
> Maybe this could be helped with a new figure in Section 3 to show the
different
concept.
> In summary, I think section 4.1 is about the Layer 2 packet data plane,
section

> provides relative priority-based scheduling, and is difficult to achieve
quantitive
>    and sophisticated queuing systems

We could mention the limited number of traffic classes in IP and MPLS, and
modify the statement about DiffServ and router implementations as you
suggested.

> I don't think that adding text to the earlier sections is a good solution,
so I

> Figure 3 is not referenced from the text and is, as far as I can see, a
very close

> document just provide additional information?

Thanks for the suggestion, will go through the reference list and move some
of them.

Best regards,
Jie