Subject: [OPSAWG] AD review for draft-ietf-opsawg-model-automation-framework-04

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 4 Sep 2020 17:22:22 +0000

Apologies for the lengthy delay in performing the AD review.

I found that this document to be well written so I would like to thank the authors, WG, and doc shepherd for that.  My more significant comments relate to questions on the scope of this architecture.

More significant comments:

1. By "Data Model" does this document mean "YANG data model"?  And if so, does it take this meaning all through this document, or only some of the time?

2. Generically, service data models are not necessarily written in YANG (e.g., I think that MEF are defining them using OpenAPI).  So, related to (1), is this architecture intended to be tied to only service models defined in YANG, or be more broadly applicable?

3. This architecture seems to quite strictly represent 3 layers (service, network, device).  Does it envisage that these layers may themselves be deconstructed?  E.g. a customer service can be constructed from underlying services (e.g., as discussed in section 3.1, but more as an East-West relationship).  Similarly, device models could also be deconstructed, e.g., if the dataplane is decoupled from the control plane, or if a device itself acts as a controller managing other devices.

4. My minimal understanding of the MEF LSO architecture was that they put quite a lot of emphasis on East-West models, probably at the service layer.  Is this effectively the same as what is described in Figure 1 in section 3.1?  Does the potential existence of these East-West APIs need to be described in any more detail?

Minor comments/clarifications:

Section 3.1: Data Models: Layering and Representation

5. Network Models are mainly network resource-facing modules; they
   describe various aspects of a network infrastructure, including
   devices and their subsystems, and relevant protocols operating at the
   link and network layers across multiple devices (e.g., network
   topology and traffic-engineering tunnel modules).
   
Would it be fair to say that Network Models might be protocol specific, or might be generalized?  If so, is that worth mentioning?

6. Re: DOTS & RFC 8783, I'm not sure how well the YANG model defined in that drafts fits into the category of Service YANG model.

7. Pipe vs hose vs funnel.  Are these terms, or do they need to be, defined somewhere?  In particular it is not obvious to me what the distinction is between pipe vs hose.

In Appendix A:
8. Would it be useful to discuss or reference YANG Catalog (as a source of querying YANG models), the public YANG github repository, or YANG module tags as a method of organizing YANG models?

9.
   o  Tunnel identities to ease manipulating extensions to specific
      tunnels [RFC8675].
      
I found this sentence slightly unclear.  Perhaps it could be reworded?

10. 
   o  Generic Policy Model:

   The Simplified Use of Policy Abstractions (SUPA) policy-based ...
   
It does not like this draft is going anywhere, and I'm not convinced that it
is really helpful to reference it here.  Or, if it must be referenced, it
should be caveated accordingly.

11.
A.3.  Device Models: Samples

I think that it would be helpful if this diagram, and the list in section A3.2, had references to the interface YANG module.

12.
A.3.1.  Model Composition

   o  Device Model
   
   [I-D.ietf-rtgwg-device-model] presents ...
   
Again, I'm not sure that this is a helpful reference, given that the approach defined in this draft did not gain traction, and instead, a more loosely coupled structure was preferred.  E.g., I see that tags (and arguably schema mount) solve this organizational problem in a more flexible way.

13.
A.3.1.1.  Schema Mount

   That capability does not cover design time.

This sentence is unclear on its own.  Perhaps either expand it or remove it.

Also, I wouldn't regard schema mount as necessarily being specific to device models, and could be used for network and service YANG models as well.  Although there may not be a good place to put it.

14. 
A.3.2.  Device Models: Samples

As above, having a section for interfaces/interface management would be useful.  I also think think it would be good to have a section for device management (e.g. system, nacm)

I would potentially reorder the list of modules:
   - Move Core Routing up, near the top of the list (above BGP)
   - L2VPN (next to) but before EVPN
   - Perhaps move BGP down, and having routing policy next to it might be helpful
   - NAT and Stateless Address Sharing could perhaps move down.

Editorial nits to check:

1. Network Operator -> network operator?
2. Perhaps "it can accommodate modules" -> "it can also accommodate modules"?
3. "follow top-down approach" -> "follow a top-down approach"
4. "validated during the implementation time" -> "validated during implementation"
5. For Diagram A2, possibly could have just used the full names and not requried the legend.
6. "[RFC8345] with TE topologies specifics." -> "[RFC8345] with TE topology related content."
7. "Network Topology Models" -> Network Topologies Model"?
8. "TE Topology Models" => "TE Topology Model"?
9. "Layer 3 Topology Models:" -> "Layer 3 Topology Model:"
10. "Layer 3 topologies specifics" -> "Layer 3 topology specifics"
11. "Layer 2 Topology Models:" -> "Layer 2 Topology Model:"
12. "Layer 2 topologies specifics" -> "Layer 2 topology specifics"
13. Figure 4: "Config Validate" -> "Config Validation", and realign "Monitoring"

Regards,
Rob

---

From: <[email-masked]>
Date: Mon, 7 Sep 2020 14:34:54 +0000

Hi Rob, 

Thank you for the detailed review. 

Please see inline.

I let my co-authors further comment. 

Cheers,
Med

> -----Message d'origine-----
> De : Rob Wilton (rwilton) [mailto:[email-masked]]
> Envoyé : vendredi 4 septembre 2020 19:22
> À : opsawg <[email-masked]>; draft-ietf-opsawg-model-automation-
> [email-masked]
> Objet : AD review for draft-ietf-opsawg-model-automation-framework-04
> 
> I found that this document to be well written so I would like to thank
> the authors, WG, and doc shepherd for that.  My more significant
> comments relate to questions on the scope of this architecture.
> 
> 
> 1. By "Data Model" does this document mean "YANG data model"?  And if
> so, does it take this meaning all through this document, or only some
> of the time?
> 

[Med] "Data Model" is used as a generic term as per rfc3444#section-4. We are using "YANG data models" or "YANG module" when we wanted to be more specific about the DM flavour. 

I double checked the text to make sure this is consistently used along the document. Updated some occurrences in the text where the use of "YANG data model" is more accurate.  

> 2. Generically, service data models are not necessarily written in
> YANG (e.g., I think that MEF are defining them using OpenAPI).  So,
> related to (1), is this architecture intended to be tied to only
> service models defined in YANG, or be more broadly applicable?

[Med] We don't require the service models to be YANG-modelled (see for instance the example depicted in Figure 2). That's said, the use of YANG in all levels would ease mapping operations.

> 
> 3. This architecture seems to quite strictly represent 3 layers
> (service, network, device).  Does it envisage that these layers may
> themselves be deconstructed?  E.g. a customer service can be
> constructed from underlying services

[Med] Yes, that's not excluded. The architecture can be "recursive" (Such as rfc8597#section-3.1.3).    

 (e.g., as discussed in section
> 3.1, but more as an East-West relationship).  Similarly, device models
> could also be deconstructed, e.g., if the dataplane is decoupled from
> the control plane, or if a device itself acts as a controller managing
> other devices.

[Med] The document does not make assumptions on the organic structure or where the functions are provided. Having a controller co-located with other YANG-controlled functions is not excluded. 

> 
> 4. My minimal understanding of the MEF LSO architecture was that they
> put quite a lot of emphasis on East-West models, probably at the
> service layer. Is this effectively the same as what is described in
> Figure 1 in section 3.1?

[Med] Inter-domain interactions between two services or two adjacent networks are not shown in Figure 1. This figure focuses mainly on the interface between a service and an underlying network.  

  Does the potential existence of these East-
> West APIs need to be described in any more detail?
> 

[Med] A network can act as a "customer" and request services from other networks. The peer network will then follow the levels depicted in the architecture. This is for example hinted in this text for example:

   o  allow customers (or Network Operators) to dynamically adjust the
      network resources based on service requirements as described in
      Service Models (e.g., Figure 2) and the current network
      performance information described in the telemetry modules.

We can add some more text if needed.  

Section 4.3 discusses how multi-domain mapping can be handled at the server level. This assumes that the interaction is not between the domains themselves but driven by the service layer.    

> Would it be fair to say that Network Models might be protocol
> specific, or might be generalized?  If so, is that worth mentioning?
> 

[Med] Network models may include some protocol-specific parameters. I'm neutral whether this needs to be mentioned in the text. 

> 
> 6. Re: DOTS & RFC 8783, I'm not sure how well the YANG model defined
> in that drafts fits into the category of Service YANG model.
> 

[Med] RFC8783 defines the filtering service requested by a client/customers; how this request triggers the selection of the devices that need to be configured, the exact set of ACLs, and the enforcing of the ACLs in these devices is managed by other layers (RFC8519). From that perspective, we do think that the module in RFC 8783 satisfies the following from RFC8309:

   "Details
   included in the service model include a description of the service as
   experienced by the customer, but not features of how that service is
   delivered or realized by the service provider. "

> 7. Pipe vs hose vs funnel.  Are these terms, or do they need to be,
> defined somewhere?  In particular it is not obvious to me what the
> distinction is between pipe vs hose.

[Med] "pipe" means that only point-to-point communications are allowed while "hose" means that communications from one to N is allowed. 

We can text to explain this. 

> 
> 
> In Appendix A:
> 8. Would it be useful to discuss or reference YANG Catalog (as a
> source of querying YANG models), the public YANG github repository, or
> YANG module tags as a method of organizing YANG models?
> 

[Med] Good point. Will consider adding some text. 

> I found this sentence slightly unclear.  Perhaps it could be reworded?

[Med] Fair point. Updated to :

" o  Tunnel identities: [RFC8675] defines a collection of YANG identities used
     as interface types for tunnel interfaces."

> It does not like this draft is going anywhere, and I'm not convinced
> that it is really helpful to reference it here.  Or, if it must be
> referenced, it should be caveated accordingly.
> 

[Med] Fair enough. I tend to agree with you. 

> I think that it would be helpful if this diagram, and the list in
> section A3.2, had references to the interface YANG module.
> 

[Med] Point taken. 

> Again, I'm not sure that this is a helpful reference, given that the
> approach defined in this draft did not gain traction, and instead, a
> more loosely coupled structure was preferred.  E.g., I see that tags
> (and arguably schema mount) solve this organizational problem in a
> more flexible way.
> 

[Med] Fair point. 

> This sentence is unclear on its own.  Perhaps either expand it or
> remove it

[Med] OK.

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: <[email-masked]>
Date: Tue, 8 Sep 2020 12:27:32 +0000

Rob,

FWIW, an updated version with changes to address your review is available online:

URL:            https://www.ietf.org/id/draft-ietf-opsawg-model-automation-framework-05.txt 
Status:         https://datatracker.ietf.org/doc/draft-ietf-opsawg-model-automation-framework/ 
Htmlized:       https://datatracker.ietf.org/doc/html/draft-ietf-opsawg-model-automation-framework 
Htmlized:       https://tools.ietf.org/html/draft-ietf-opsawg-model-automation-framework-05 
Diff:           https://www.ietf.org/rfcdiff?url2=draft-ietf-opsawg-model-automation-framework-05

Please let us know if any of your comments is still pending. Thank you.

Cheers,
Med

> De : [email-masked]
> [mailto:[email-masked]]
> Envoyé : lundi 7 septembre 2020 16:35
> À : Rob Wilton (rwilton) <[email-masked]>; opsawg <[email-masked]>
> Cc : [email-masked]
> Objet : RE: AD review for draft-ietf-opsawg-model-automation-
> framework-04
> 
> Hi Rob,
> 
> > De : Rob Wilton (rwilton) [mailto:[email-masked]] Envoyé :
> vendredi
> > 4 septembre 2020 19:22 À : opsawg <[email-masked]>;
> > draft-ietf-opsawg-model-automation-
> > [email-masked]
> [Med] "Data Model" is used as a generic term as per rfc3444#section-4.
> We are using "YANG data models" or "YANG module" when we wanted to be
> more specific about the DM flavour.
> 
> I double checked the text to make sure this is consistently used along
> the document. Updated some occurrences in the text where the use of
> "YANG data model" is more accurate.
> 
> [Med] We don't require the service models to be YANG-modelled (see for
> instance the example depicted in Figure 2). That's said, the use of
> YANG in all levels would ease mapping operations.
> 
> >
> [Med] The document does not make assumptions on the organic structure
> or where the functions are provided. Having a controller co-located
> with other YANG-controlled functions is not excluded.
> 
> >
> [Med] Inter-domain interactions between two services or two adjacent
> networks are not shown in Figure 1. This figure focuses mainly on the
> interface between a service and an underlying network.
> 
> [Med] A network can act as a "customer" and request services from
> other networks. The peer network will then follow the levels depicted
> in the architecture. This is for example hinted in this text for
> example:
> 
> Section 4.3 discusses how multi-domain mapping can be handled at the
> server level. This assumes that the interaction is not between the
> domains themselves but driven by the service layer.
> 
> >
> [Med] Network models may include some protocol-specific parameters.
> I'm neutral whether this needs to be mentioned in the text.
> 
> >
> [Med] RFC8783 defines the filtering service requested by a
> client/customers; how this request triggers the selection of the
> devices that need to be configured, the exact set of ACLs, and the
> enforcing of the ACLs in these devices is managed by other layers
> (RFC8519). From that perspective, we do think that the module in RFC
> 8783 satisfies the following from RFC8309:
> 
>    "Details
> [Med] "pipe" means that only point-to-point communications are allowed
> while "hose" means that communications from one to N is allowed.
>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 18 Sep 2020 17:20:37 +0000

Hi Med,

The changes that you have made look good.  But I had some comments at the end of my review email that it doesn't look like you have responded to, or actioned.  Hence, I'm wondering if my review email might have been truncated, or the end of it missed (from "Also, I ...") onwards (copied below).

A.3.1.1.  Schema Mount

14. 

Editorial nits to check:

Please can you check?

Regards,
Rob

> -----Original Message-----
> From: [email-masked] <[email-masked]>
> Sent: 08 September 2020 13:28
> To: Rob Wilton (rwilton) <[email-masked]>; opsawg <[email-masked]>
> Cc: [email-masked]
> Subject: RE: AD review for draft-ietf-opsawg-model-automation-framework-04
> 
> Rob,
> 
> __________________________________________________________________________
> _______________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;

---

From: <[email-masked]>
Date: Mon, 21 Sep 2020 08:12:12 +0000

Hi Bob, 

Thank you for double checking. 

I confirm that your initial AD review message I received was truncated (see attached). 

Will update the draft to take into account the missing part.

Cheers,
Med

> Envoyé : vendredi 18 septembre 2020 19:21
> À : BOUCADAIR Mohamed TGI/OLN <[email-masked]>; opsawg
> <[email-masked]>
> The changes that you have made look good.  But I had some comments
> at the end of my review email that it doesn't look like you have
> responded to, or actioned.  Hence, I'm wondering if my review email
> might have been truncated, or the end of it missed (from "Also, I
> ...") onwards (copied below).
> 
> 
> A.3.1.1.  Schema Mount
> 
> remove it.
> Also, I wouldn't regard schema mount as necessarily being specific
> to device models, and could be used for network and service YANG
> models as well.  Although there may not be a good place to put it.
> 
> 14.
> As above, having a section for interfaces/interface management would
> be useful.  I also think think it would be good to have a section
> for device management (e.g. system, nacm)
> 
> > Subject: RE: AD review for
> > draft-ietf-opsawg-model-automation-framework-04
> >
> > Rob,
> >
> > FWIW, an updated version with changes to address your review is
> > available
> > online:
> >
> > Status:         https://datatracker.ietf.org/doc/draft-ietf-
> opsawg-model-
> > automation-framework/
> > Htmlized:       https://datatracker.ietf.org/doc/html/draft-ietf-
> opsawg-
> > model-automation-framework
> > Htmlized:       https://tools.ietf.org/html/draft-ietf-opsawg-
> model-
> > automation-framework-05
> > Diff:           https://www.ietf.org/rfcdiff?url2=draft-ietf-
> opsawg-model-
> > automation-framework-05
> >
> > > Envoyé : lundi 7 septembre 2020 16:35 À : Rob Wilton (rwilton)
> > > <[email-masked]>; opsawg <[email-masked]> Cc :
> > > [email-masked]
> > > > Objet : AD review for
> > > > draft-ietf-opsawg-model-automation-framework-
> > > 04
> > > >
> > > > I found that this document to be well written so I would like
> to
> > > thank
> > > > 1. By "Data Model" does this document mean "YANG data model"?
> And
> > > if
> > > > so, does it take this meaning all through this document, or
> only
> > > some
> > > > of the time?
> > > >
> > >
> > > We are using "YANG data models" or "YANG module" when we wanted
> to
> > > be more specific about the DM flavour.
> > >
> > > I double checked the text to make sure this is consistently used
> > > along the document. Updated some occurrences in the text where
> the
> > > use of "YANG data model" is more accurate.
> > >
> > > > YANG (e.g., I think that MEF are defining them using OpenAPI).
> > > > So, related to (1), is this architecture intended to be tied
> to
> > > > only service models defined in YANG, or be more broadly
> applicable?
> > >
> > > [Med] We don't require the service models to be YANG-modelled
> (see
> > > for instance the example depicted in Figure 2). That's said, the
> use
> > > of YANG in all levels would ease mapping operations.
> > >
> > > >
> > > > (service, network, device).  Does it envisage that these
> layers
> > > > may themselves be deconstructed?  E.g. a customer service can
> be
> > > [Med] Yes, that's not excluded. The architecture can be
> "recursive"
> > > (Such as rfc8597#section-3.1.3).
> > >
> > > > 3.1, but more as an East-West relationship).  Similarly,
> device
> > > models
> > > > could also be deconstructed, e.g., if the dataplane is
> decoupled
> > > from
> > > [Med] The document does not make assumptions on the organic
> > > structure or where the functions are provided. Having a
> controller
> > > co-located with other YANG-controlled functions is not excluded.
> > >
> > > >
> > > > 4. My minimal understanding of the MEF LSO architecture was
> that
> > > they
> > > > service layer. Is this effectively the same as what is
> described
> > > > in Figure 1 in section 3.1?
> > >
> > > networks are not shown in Figure 1. This figure focuses mainly
> on
> > > the interface between a service and an underlying network.
> > >
> > > other networks. The peer network will then follow the levels
> > > depicted in the architecture. This is for example hinted in this
> > > text for
> > > example:
> > >
> > > >    devices and their subsystems, and relevant protocols
> operating
> > > > at the
> > > > 6. Re: DOTS & RFC 8783, I'm not sure how well the YANG model
> > > > defined in that drafts fits into the category of Service YANG
> model.
> > > >
> > >
> > >    experienced by the customer, but not features of how that
> service
> > > is
> > > > 7. Pipe vs hose vs funnel.  Are these terms, or do they need
> to
> > > > be, defined somewhere?  In particular it is not obvious to me
> what
> > > > the distinction is between pipe vs hose.
> > >
> > > [Med] "pipe" means that only point-to-point communications are
> > > allowed while "hose" means that communications from one to N is
> allowed.
> > >
> > > > source of querying YANG models), the public YANG github
> > > > repository,
> > > or
> > > > It does not like this draft is going anywhere, and I'm not
> > > > convinced that it is really helpful to reference it here.  Or,
> if
> > > > it must be referenced, it should be caveated accordingly.
> > > >
> > >
> > > > Again, I'm not sure that this is a helpful reference, given
> that
> > > > the approach defined in this draft did not gain traction, and
> > > > instead, a more loosely coupled structure was preferred.
> E.g., I
> > > > see that tags (and arguably schema mount) solve this
> > > > organizational problem in a more flexible way.
> > > >
> > >
> > > [Med] Fair point.
> > >
> > > > 13.
> > > > A.3.1.1.  Schema Mount
> > > >

Subject: AD review for draft-ietf-opsawg-model-automation-framework-04
From: "Rob Wilton (rwilton)" <[email-masked]>
To: opsawg <[email-masked]>, "[email-masked]" <[email-masked]>
Date: Fri, 4 Sep 2020 17:22:22 +0000

In Appendix A:

9.

10.
   o  Generic Policy Model:

11.

12.
A.3.1.  Model Composition

   o  Device Model

13.
A.3.1.1.  Schema Mount

This sentence is unclear on its own.  Perhaps either expand it or remove it

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 24 Sep 2020 12:41:29 +0000

Hi Med,

Thanks for the final updates.  I've requested IETF LC on -06.

Regards,
Rob

> Sent: 21 September 2020 09:12