Subject: [bmwg] ISSU Charter paragraph

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 19 Jan 2014 22:51:49 -0500

BMWG,

As we continue to build our new charter step-by-step, here's
the first draft paragraph Sarah and I put together for your review:

In Service Software Upgrade (ISSU): Develop new methods and benchmarks to characterize the upgrade of network devices while in-service, considering both data and control plane operations and impacts. These devices are generally expected to maintain control plane session integrity, including routing connections. Packet loss should be measured and recovery behavior noted accordingly.

We'd like to have any comments on this proposal in the next 2 weeks.

As a reminder, we have already reviewed the items below,
and we have at least one or more items to sort-out before we
re-charter.

regards,
Sarah and Al

* Traffic Management: Develop the methods to characterize the capacity of
traffic management features in network devices, such as classification, 
policing, shaping, and active queue management. 
Existing terminology will be used where appropriate.
Configured operation will be verified as a part of the methodology.   
The goal is a methodology to assess the maximum forwarding performance that a
network device can sustain without dropping or impairing packets, or
compromising the accuracy of multiple instances of traffic management
functions. This is the benchmark for comparison between devices.
Another goal is to devise methods that utilize flows with congestion-aware 
transport as part of the traffic load and still produce repeatable results in
the isolated test environment.

-=-=-=-=-=-=-=-=-=-

IPv6 Neighbor Discovery-related benchmarking:
Large address space in IPv6 subnets presents several networking 
challenges, as described in RFC 6583. Indexes to describe the 
performance of network devices, such as the number of reachable
devices on a sub-network, are useful benchmarks to the operations
community. The working group will develop the necessary terminology
and methodologies to measure such benchmarks.

---

From: joel jaeggli <[email-masked]>
Date: Sun, 19 Jan 2014 22:56:54 -0800

On 1/19/14, 7:51 PM, MORTON, ALFRED C (AL) wrote:
> BMWG,
> 
> As we continue to build our new charter step-by-step, here's the
> first draft paragraph Sarah and I put together for your review:
> 
> In Service Software Upgrade (ISSU): Develop new methods and
> benchmarks to characterize the upgrade of network devices while
> in-service, considering both data and control plane operations and
> impacts. These devices are generally expected to maintain control
> plane session integrity, including routing connections. Packet loss
> should be measured and recovery behavior noted accordingly.

Quantification of impact does seem like the key deliverable.

> As a reminder, we have already reviewed the items below, and we have
> at least one or more items to sort-out before we re-charter.
> 
> regards, Sarah and Al
> 
> 
> * Traffic Management: Develop the methods to characterize the
> capacity of traffic management features in network devices, such as
> classification, policing, shaping, and active queue management. 
> Existing terminology will be used where appropriate. Configured
> operation will be verified as a part of the methodology. The goal is
> a methodology to assess the maximum forwarding performance that a 
> compromising the accuracy of multiple instances of traffic
> management functions. This is the benchmark for comparison between
> devices. Another goal is to devise methods that utilize flows with
> congestion-aware transport as part of the traffic load and still
> produce repeatable results in the isolated test environment.
> 
> -=-=-=-=-=-=-=-=-=-
> 
> IPv6 Neighbor Discovery-related benchmarking: Large address space in
> IPv6 subnets presents several networking challenges, as described in
> RFC 6583. Indexes to describe the performance of network devices,
> such as the number of reachable devices on a sub-network, are useful
> benchmarks to the operations community. The working group will
> develop the necessary terminology and methodologies to measure such
> benchmarks. _______________________________________________ bmwg
> mailing list [email-masked] 
> https://www.ietf.org/mailman/listinfo/bmwg
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 20 Jan 2014 07:56:36 -0500

Good edit Joel. Let's include that point in the last sentence:

Quantification of Upgrade impact will include packet loss measurement,
and other forms of recovery behavior will be noted accordingly.

> -----Original Message-----
> From: joel jaeggli [mailto:[email-masked]]
> Sent: Monday, January 20, 2014 1:57 AM
> To: MORTON, ALFRED C (AL); [email-masked]
> Subject: Re: [bmwg] ISSU Charter paragraph
>

---

From: "Fernando Calabria (fcalabri)" <[email-masked]>
Date: Mon, 20 Jan 2014 13:20:20 +0000

Al. et all, 

This looks ok to me

Rgds 

Fernando 

On 1/20/14, 7:56 AM, "MORTON, ALFRED C (AL)" <[email-masked]> wrote:

>_______________________________________________
>bmwg mailing list
>[email-masked]

---

From: joel jaeggli <[email-masked]>
Date: Mon, 20 Jan 2014 09:12:22 -0800

> and other forms of recovery behavior will be noted accordingly.

Good one,

thanks

---

From: <[email-masked]>
Date: Tue, 21 Jan 2014 09:13:31 +0000

>From: joel jaeggli >Sent: Monday, January 20, 2014 7:57 AM
>

+1

Packet loss in the forwarding plane is the main impact. Multiple forwarding planes may be involved: IPv4, IPv6, MPLS, Ethernet (including MPLS VPNs)
Other impacts may be:
- control plane impact on neighbor. Ideally no impact, but some implementations may delegate some work on the neighbors (e.g. for BGP: Graceful Restart, Route Refresh))
- delayed rerouting (e.g. how long are routing messages delayed)
- missed control plane events (e.g. ARP, DHCP, IGMP...) especially on the subscribed side

Ideally, all impact should be identified as this may impact when and how the maintenance/ISSU will be performed.
E.g. during the day or the night (typically depending on the forwarding plane impact); the need for someone to be present/available/ready to go on the peers promises (the more control plane impacts on the peer, the more risks)... 

>> produce repeatable results in the isolated test environment.

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Banks, Sarah" <[email-masked]>
Date: Tue, 21 Jan 2014 12:04:05 -0600

Hi Bruno,
	Thanks for your input! I wanted to touch base on a point you raise below,
however. In Service Software Upgrades imply that they¹re done in service;
now, we all know that there aren¹t SP¹s or operators out there doing this
in service, for a variety of reasons, but in adhering to the tenet of ³In
Service², this would actually preclude Graceful Restart (GR); it would
actually require Non Stop Routing (NSR) for supported routing protocols to
be enabled. Indeed, it¹s our approach that the moment a routing neighbor
flaps, you record the behavior, but you¹ve actually failed ISSU, because
you interrupted the ³service² of ³in service². While we continue to find
ways to state this without sounding like a conformance paragraph - because
BMWG doesn¹t do conformance testing - it¹s certainly worth gathering
input. :) Would you agree that NSR is a fundamental requirement for an
ISSU operation?

Kind regards,
Sarah

On 1/21/14, 1:13 AM, "[email-masked]"
<[email-masked]> wrote:

>Packet loss in the forwarding plane is the main impact. Multiple
>forwarding planes may be involved: IPv4, IPv6, MPLS, Ethernet (including
>MPLS VPNs)
>Other impacts may be:
>- control plane impact on neighbor. Ideally no impact, but some
>implementations may delegate some work on the neighbors (e.g. for BGP:
>Graceful Restart, Route Refresh))
>E.g. during the day or the night (typically depending on the forwarding
>plane impact); the need for someone to be present/available/ready to go
>on the peers promises (the more control plane impacts on the peer, the
>more risks)... 
>
>
>
>__________________________________________________________________________
>Ce message et ses pieces jointes peuvent contenir des informations
>confidentielles ou privilegiees et ne doivent donc
>pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
>recu ce message par erreur, veuillez le signaler
>a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
>electroniques etant susceptibles d'alteration,
>This message and its attachments may contain confidential or privileged
>information that may be protected by law;

---

From: <[email-masked]>
Date: Wed, 22 Jan 2014 09:48:27 +0000

Hi Sarah,

>From: Banks, Sarah [mailto:[email-masked]] >Sent: Tuesday, January 21, 2014 7:04 PM
>
>Hi Bruno,
>you interrupted the ³service² of ³in service². 

So it really depends on the definition and scope of "service" that you are considering.
For IP routers, the main service is forwarding IP packets, in the forwarding plane. So if they can be upgraded while the forwarding plane be kept running, I (and afaik routers vendors) call this ISSU.

e.g. among first hits on google

"The major elements supporting the functionality of the SSO and ISSU for Cisco IOS XE VPN 6vPE and 6PE features are the following:

[...]
    BGP Graceful Restart--The BGP Graceful Restart feature is responsible for negotiating graceful restart capabilities, exchanging forwarding preservation states, and coordinating advertisements after session restarts. MPLS VPNs interact with BGP to exchange VPN routing and forwarding (VRF) routes and labels."

http://www.cisco.com/en/US/docs/ios-xml/ios/mp_ha/configuration/xe-3s/mp-6vpe-6pe-issu-sso.html

Now if IP forwarding is probably the main service of a core router, an edge router may provide additional services.
If you go that way, the first step may be to define what the service is.

>While we continue to find

AFAIK, no. But if you have an agreed upon definition of ISSU, I would be interested in reading it.

On a side note, we could also discuss the meaning of NSR, which probably mostly means that a routing session with a vanilla router stays up. But this does not mean that ISSU has no impact on the peer router (e.g. some implementation sends a BGP Route Refresh). Also AFAIK, NSR says nothing about other control plane services.(e.g. DHCP relay/server, IGMP request...)

Finally, my comment seems in synch with the proposed high level goal of characterizing the impact of ISSU on both the forwarding and control plane:

>>> impacts.

Best regards,
Bruno

---

From: "Fernando Calabria (fcalabri)" <[email-masked]>
Date: Wed, 22 Jan 2014 11:06:53 +0000

Bruno, while some definition of  ISSU are "quite vague /  open”  for this
specific vendor you are referring,  take a look at the following page
where it’s ISSU definition and basic required architecture are defined ..

I agree with you point and to me is  technically valid, but the main idea
of real ISSU is not rely on another device  to piggy back the process /
failure  I.E GR procedures. What would happens for example in the case of
the L3 VPN provider with thousands of unmanaged Ces that do not support GR
? 

Check the following  documentation from the same vendor and yes, you will
find quite a few contradicting information and definitions floating around
.. :) This is one of the main golas / objectives of the document we are
proposing , which is basically :  standarize  the definition describe the
procedure and how to qualify , benchmark the duration of the outage /
overall process impact …

http://www.cisco.com/en/US/products/ps7149/products_ios_protocol_group_home
.html

And more specifically the following PDF..

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6550/prod_pres
entation0900aecd80456cb8.pdf

Rgds 

Fer

On 1/22/14, 4:48 AM, "[email-masked]"
<[email-masked]> wrote:

>For IP routers, the main service is forwarding IP packets, in the
>forwarding plane. So if they can be upgraded while the forwarding plane
>be kept running, I (and afaik routers vendors) call this ISSU.
>
>"The major elements supporting the functionality of the SSO and ISSU for
>Cisco IOS XE VPN 6vPE and 6PE features are the following:
>
>[...]
>    BGP Graceful Restart--The BGP Graceful Restart feature is responsible
>for negotiating graceful restart capabilities, exchanging forwarding
>preservation states, and coordinating advertisements after session
>restarts. MPLS VPNs interact with BGP to exchange VPN routing and
>forwarding (VRF) routes and labels."
>
>Now if IP forwarding is probably the main service of a core router, an
>edge router may provide additional services.
>On a side note, we could also discuss the meaning of NSR, which probably
>mostly means that a routing session with a vanilla router stays up. But
>this does not mean that ISSU has no impact on the peer router (e.g. some
>implementation sends a BGP Route Refresh). Also AFAIK, NSR says nothing
>about other control plane services.(e.g. DHCP relay/server, IGMP
>request...)
>
>
>Finally, my comment seems in synch with the proposed high level goal of
>characterizing the impact of ISSU on both the forwarding and control
>plane:
>
>>>- missed control plane events (e.g. ARP, DHCP, IGMP...) especially on
>>>the
>>>subscribed side
>>>

---

From: <[email-masked]>
Date: Wed, 22 Jan 2014 11:51:48 +0000

Fernando,

>From: Fernando Calabria (fcalabri) [mailto:[email-masked]] >Sent: Wednesday, January 22, 2014 12:07 PM
>
>Bruno, while some definition of  ISSU are "quite vague /  open"  for this
>where it's ISSU definition and basic required architecture are defined ..

Reading the docs, I understand that ISSU is mostly about preserving the IP forwarding (*) and that NSR is not required.
Note that the pages you are referring to are from the same "specific vendor".

(*) Still a 2 seconds stop of forwarding is (self) defined as acceptable, while for dual-attached customers/services I would consider that dynamic rerouting could potentially be faster, and a make before break procedure can even achieve 0 packet loss. (cf BGP graceful shutdown, which is implemented by this specific vendor)

>the L3 VPN provider with thousands of unmanaged Ces that do not support GR

- First version of ISSU, as marketed by vendors marketing, required GR on those CE routers. Then VPN Service Providers brought the CE argument in order to get NSR from router vendors. So please don't bring back that argument to me :-)
- Up to very recently (or still now), some implementation required the support the BGP Route Refresh on the CE. Which contradict to your own definition of not relying on another device. Idem, while NSR was supported on the CE side, it was not supported on the Route Reflector side where GR was required.

>overall process impact .

Good. I'm fine with this goal. And in particular, I'm fine with the following text:

>>>> Develop new methods and

However, the following text seems too specific to me at this time. Especially since we don't yet have a definition of ISSU or what "service" we are referring to.
And it would be good to have an (IETF) definition of ISSU that we can agree on. This can clearly be done after the charter, but IMO should be done at the beginning of the work

Regards,
Bruno

>>>>> These devices are generally expected to maintain control
>>>>> plane session integrity, including routing connections

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 22 Jan 2014 13:34:09 -0500

So, we can append the need to define ISSU to the paragraph,
at least:

In Service Software Upgrade (ISSU): Develop new methods and
benchmarks to characterize the upgrade of network devices while
in-service, considering both data and control plane operations and
impacts. These devices are generally expected to maintain control
plane session integrity, including routing connections.
Quantification of Upgrade impact will include packet loss measurement,
and other forms of recovery behavior will be noted accordingly.
The work will produce a definition of ISSU, which will help
refine the scope.

What else needs clarification?
Al

> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, January 22, 2014 6:52 AM
> To: Fernando Calabria (fcalabri); Banks, Sarah
> Cc: MORTON, ALFRED C (AL); [email-masked]
> Subject: RE: [bmwg] ISSU Charter paragraph
>
> Fernando,
>
>
> Reading the docs, I understand that ISSU is mostly about preserving the IP
> forwarding (*) and that NSR is not required.
> (*) Still a 2 seconds stop of forwarding is (self) defined as acceptable,
> while for dual-attached customers/services I would consider that dynamic
> rerouting could potentially be faster, and a make before break procedure
> can even achieve 0 packet loss. (cf BGP graceful shutdown, which is
> implemented by this specific vendor)
>
> - First version of ISSU, as marketed by vendors marketing, required GR on
> those CE routers. Then VPN Service Providers brought the CE argument in
> order to get NSR from router vendors. So please don't bring back that
> argument to me :-)
> - Up to very recently (or still now), some implementation required the
> support the BGP Route Refresh on the CE. Which contradict to your own
> definition of not relying on another device. Idem, while NSR was supported
> on the CE side, it was not supported on the Route Reflector side where GR
> was required.
>
>
> However, the following text seems too specific to me at this time.
> Especially since we don't yet have a definition of ISSU or what "service"
> we are referring to.
> And it would be good to have an (IETF) definition of ISSU that we can
> agree on. This can clearly be done after the charter, but IMO should be
> done at the beginning of the work
>
> Regards,
> Bruno
>
> >>http://www.cisco.com/en/US/docs/ios-xml/ios/mp_ha/configuration/xe-
> 3s/mp-6
> >>vpe-6pe-issu-sso.html
> >>
> >>
> >>>>Ideally, all impact should be identified as this may impact when and
> how
> >>>>the maintenance/ISSU will be performed.
> >>>>______________________________________________________________________
> __
> >>>>__
> >>>>Orange decline toute responsabilite si ce message a ete altere,
> deforme
> >>>>ou falsifie. Merci.
> >>>>

---

From: "Reinhard Schrage" <[email-masked]>
Date: Thu, 23 Jan 2014 01:18:43 +0100

If doing ISSU it might also be worthwhile to liaise with the Open Networking
Foundation regarding OpenFlow features, as OpenFlow is aiming to separate
the control from the forwarding plane and make the control plane accessible
to software interfacing. 

Best regards
Reinhard Schrage
t:     +49 (0) 5137 909540
m:   +49 (0) 172  26.36.046
[email-masked]

-----Original Message-----
From: bmwg [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: 22 January 2014 19:34
To: [email-masked]; Fernando Calabria (fcalabri); Banks, Sarah
Cc: [email-masked]
Subject: Re: [bmwg] ISSU Charter paragraph

So, we can append the need to define ISSU to the paragraph, at least:

In Service Software Upgrade (ISSU): Develop new methods and benchmarks to
characterize the upgrade of network devices while in-service, considering
both data and control plane operations and impacts. These devices are
generally expected to maintain control plane session integrity, including
routing connections.
Quantification of Upgrade impact will include packet loss measurement, and
other forms of recovery behavior will be noted accordingly.
The work will produce a definition of ISSU, which will help refine the
scope.

> >Bruno, while some definition of  ISSU are "quite vague /  open"  for 
> >this specific vendor you are referring,  take a look at the following 
> >page where it's ISSU definition and basic required architecture are
defined ..
>
> Reading the docs, I understand that ISSU is mostly about preserving 
> the IP forwarding (*) and that NSR is not required.
> (*) Still a 2 seconds stop of forwarding is (self) defined as 
> acceptable, while for dual-attached customers/services I would 
> consider that dynamic rerouting could potentially be faster, and a 
> make before break procedure can even achieve 0 packet loss. (cf BGP 
> graceful shutdown, which is implemented by this specific vendor)
>
> >I agree with you point and to me is  technically valid, but the main 
> >idea of real ISSU is not rely on another device  to piggy back the 
> >process / failure  I.E GR procedures. What would happens for example 
> >in the case of the L3 VPN provider with thousands of unmanaged Ces 
> >that do not support
> GR
> >?
>
> - First version of ISSU, as marketed by vendors marketing, required GR 
> on those CE routers. Then VPN Service Providers brought the CE 
> argument in order to get NSR from router vendors. So please don't 
> bring back that argument to me :-)
> definition of not relying on another device. Idem, while NSR was 
> supported on the CE side, it was not supported on the Route Reflector 
> side where GR was required.
>
>
> >Check the following  documentation from the same vendor and yes, you 
> >will find quite a few contradicting information and definitions 
> >floating
> around
> >.. :) This is one of the main golas / objectives of the document we 
> >are proposing , which is basically :  standarize  the definition 
> >describe the procedure and how to qualify , benchmark the duration of 
> >the outage / overall process impact .
>
> >>>> in-service, considering both data and control plane operations 
> >>>> and impacts.
>
> agree on. This can clearly be done after the charter, but IMO should 
> be done at the beginning of the work
>
> Regards,
> Bruno
>
> >>>>> These devices are generally expected to maintain control plane 
> >>>>> session integrity, including routing connections
>
>
> >http://www.cisco.com/en/US/products/ps7149/products_ios_protocol_grou
> >p_ho
> me
> >.html
> >
> >http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6550/pro
> >d_pr
> es
> >>>   Thanks for your input! I wanted to touch base on a point you 
> >>>raise below, however. In Service Software Upgrades imply that 
> >>>they¹re done in
> service;
> >>>now, we all know that there aren¹t SP¹s or operators out there 
> >>>doing
> this
> >>>in service, for a variety of reasons, but in adhering to the tenet 
> >>>of
> ³In
> >>>Service², this would actually preclude Graceful Restart (GR); it 
> >>>would actually require Non Stop Routing (NSR) for supported routing 
> >>>protocols to be enabled. Indeed, it¹s our approach that the moment 
> >>>a routing
> neighbor
> >>forwarding plane. So if they can be upgraded while the forwarding 
> >>plane be kept running, I (and afaik routers vendors) call this ISSU.
> >>
> >>"The major elements supporting the functionality of the SSO and ISSU 
> >>for Cisco IOS XE VPN 6vPE and 6PE features are the following:
> >>
> >>[...]
> >>Now if IP forwarding is probably the main service of a core router, 
> >>an edge router may provide additional services.
> >>>ways to state this without sounding like a conformance paragraph - 
> >>>because BMWG doesn¹t do conformance testing - it¹s certainly worth 
> >>>gathering input. :) Would you agree that NSR is a fundamental 
> >>>requirement for an ISSU operation?
> >>
> >>On a side note, we could also discuss the meaning of NSR, which 
> >>probably mostly means that a routing session with a vanilla router 
> >>stays up. But this does not mean that ISSU has no impact on the peer 
> >>router (e.g. some implementation sends a BGP Route Refresh). Also 
> >>AFAIK, NSR says nothing about other control plane services.(e.g. 
> >>DHCP relay/server, IGMP
> >>request...)
> >>
> >>
> >>Finally, my comment seems in synch with the proposed high level goal 
> >>of characterizing the impact of ISSU on both the forwarding and 
> >>control
> >>plane:
> >>
> >>>>>in-service, considering both data and control plane operations 
> >>>>>and  impacts.
> >>
> >>
> >>Best regards,
> >>Bruno
> >>
> >>>Kind regards,
> >>>Sarah
> >>>
> >>>>>> in-service, considering both data and control plane operations 
> >>>>>> and impacts. These devices are generally expected to maintain 
> >>>>>> control plane session integrity, including routing connections. 
> >>>>>> Packet loss should be measured and recovery behavior noted
accordingly.
> >>>>plane impact); the need for someone to be present/available/ready 
> >>>>to
> go
> >>>>on the peers promises (the more control plane impacts on the peer, 
> >>>>the more risks)...
> >>>>
> >>>>
> >>>>
> >>>>>> capacity of traffic management features in network devices, 
> >>>>>> such as classification, policing, shaping, and active queue
management.
> >>>>>> operation will be verified as a part of the methodology. The 
> >>>>>> goal
> is
> >>>>>> a methodology to assess the maximum forwarding performance that 
> >>>>>> a network device can sustain without dropping or impairing 
> >>>>>> packets,
> or
> >>>>>> management functions. This is the benchmark for comparison 
> >>>>>> between devices. Another goal is to devise methods that utilize 
> >>>>>> flows with congestion-aware transport as part of the traffic 
> >>>>>> load and still produce repeatable results in the isolated test
environment.
> >>>>>>
> >>>>>> -=-=-=-=-=-=-=-=-=-
> >>>>>>
> >>>>>> IPv6 Neighbor Discovery-related benchmarking: Large address 
> >>>>>> space
> in
> >>>>>> IPv6 subnets presents several networking challenges, as 
> >>>>>> described
> in
> >>>>>> RFC 6583. Indexes to describe the performance of network 
> >>>>>> devices, such as the number of reachable devices on a 
> >>>>>> sub-network, are
> useful
> >>>>>> develop the necessary terminology and methodologies to measure 
> >>>>>> such benchmarks. 
> >>>>>> _______________________________________________ bmwg mailing 
> >>>>>> list [email-masked] https://www.ietf.org/mailman/listinfo/bmwg
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>confidentielles ou privilegiees et ne doivent donc pas etre 
> >>>>diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> >>>>ce message par erreur, veuillez le signaler a l'expediteur et le 
> >>>>detruire ainsi que les pieces jointes. Les
> messages
> >>>>electroniques etant susceptibles d'alteration, Orange decline 
> >>>>toute responsabilite si ce message a ete altere,
> deforme
> >>>>ou falsifie. Merci.
> >>>>
> >>>>information that may be protected by law; they should not be 
> >>>>distributed, used or copied without authorisation.
> >>>>If you have received this email in error, please notify the sender 
> >>>>and delete this message and its attachments.
> >>>>As emails may be altered, Orange is not liable for messages that 
> >>>>have been modified, changed or falsified.
> >>>>Thank you.
> >>>>
> >>____________________________________________________________________
> >>____
> __
> >>confidentielles ou privilegiees et ne doivent donc pas etre 
> >>diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> >>ce message par erreur, veuillez le signaler a l'expediteur et le 
> >>detruire ainsi que les pieces jointes. Les messages electroniques 
> >>etant susceptibles d'alteration, Orange decline toute responsabilite 
> >>si ce message a ete altere, deforme ou falsifie. Merci.
> >>
> >>This message and its attachments may contain confidential or 
> >>privileged information that may be protected by law; they should not 
> >>be distributed, used or copied without authorisation.
> >>If you have received this email in error, please notify the sender 
> >>and delete this message and its attachments.
> >>As emails may be altered, Orange is not liable for messages that 
> >>have been modified, changed or falsified.
> >>Thank you.
> >>
> ____ _______________________________________________
>
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi 
> que les pieces jointes. Les messages electroniques etant susceptibles 
> d'alteration, Orange decline toute responsabilite si ce message a ete 
> altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not 
> be distributed, used or copied without authorisation.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 22 Jan 2014 20:07:13 -0500

Hi Reinhard,

A liaison could make sense, if we take-up the work,
to indicate the new relevant effort to ONF, and others.

Does anyone know of work in any standards org. that
is overlapping what we propose in BMWG?
(that would be a reason to liaise now)

Al

> From: Reinhard Schrage [mailto:[email-masked]]
> Sent: Wednesday, January 22, 2014 7:19 PM
> To: MORTON, ALFRED C (AL); [email-masked]; 'Fernando Calabria
> (fcalabri)'; 'Banks, Sarah'
> Cc: [email-masked]
> > privileged

---

From: "Reinhard Schrage" <[email-masked]>
Date: Thu, 23 Jan 2014 02:15:38 +0100

Hi Al,

I am editor in IEEE WG 1903 (NGSON) and we are currently reaching out to ONF
and ETSI (for NFV) to check liaison possibilities. 
ETSI is meeting this week and we have an IEEE webex conf next week.
I should know more by end next week, but definitely early enough for IETF 89
in London.

Best regards
Reinhard Schrage
t:     +49 (0) 5137 909540
m:   +49 (0) 172  26.36.046

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: 23 January 2014 02:07
To: Reinhard Schrage; [email-masked]; 'Fernando Calabria

Hi Reinhard,

A liaison could make sense, if we take-up the work, to indicate the new
relevant effort to ONF, and others.

Does anyone know of work in any standards org. that is overlapping what we
propose in BMWG?
(that would be a reason to liaise now)

Al

> To: MORTON, ALFRED C (AL); [email-masked]; 'Fernando 
> Calabria (fcalabri)'; 'Banks, Sarah'
> Cc: [email-masked]
> If doing ISSU it might also be worthwhile to liaise with the Open 
> Networking Foundation regarding OpenFlow features, as OpenFlow is 
> aiming to separate the control from the forwarding plane and make the 
> control plane accessible to software interfacing.
>
> Best regards
> Reinhard Schrage
> t:     +49 (0) 5137 909540
> m:   +49 (0) 172  26.36.046
> From: bmwg [mailto:[email-masked]] On Behalf Of MORTON, ALFRED 
> C
> (AL)
> In Service Software Upgrade (ISSU): Develop new methods and benchmarks 
> to characterize the upgrade of network devices while in-service, 
> considering both data and control plane operations and impacts. These 
> devices are generally expected to maintain control plane session 
> integrity, including routing connections.
> > >Bruno, while some definition of  ISSU are "quite vague /  open"  
> > >for this specific vendor you are referring,  take a look at the 
> > >following page where it's ISSU definition and basic required 
> > >architecture are
> defined ..
> >
> > >I agree with you point and to me is  technically valid, but the 
> > >main idea of real ISSU is not rely on another device  to piggy back 
> > >the process / failure  I.E GR procedures. What would happens for 
> > >example in the case of the L3 VPN provider with thousands of 
> > >unmanaged Ces that do not support
> > GR
> > >?
> >
> > - First version of ISSU, as marketed by vendors marketing, required 
> > GR on those CE routers. Then VPN Service Providers brought the CE 
> > - Up to very recently (or still now), some implementation required 
> > the support the BGP Route Refresh on the CE. Which contradict to 
> > your own definition of not relying on another device. Idem, while 
> > NSR was supported on the CE side, it was not supported on the Route 
> > Reflector side where GR was required.
> >
> >
> > >Check the following  documentation from the same vendor and yes, 
> > >you will find quite a few contradicting information and definitions 
> > >floating
> > around
> > >describe the procedure and how to qualify , benchmark the duration 
> > >of the outage / overall process impact .
> >
> > And it would be good to have an (IETF) definition of ISSU that we 
> > can agree on. This can clearly be done after the charter, but IMO 
> > should be done at the beginning of the work
> >
> > Regards,
> > Bruno
> >
> > >>>in service, for a variety of reasons, but in adhering to the 
> > >>>tenet of
> > ³In
> > >>>would actually require Non Stop Routing (NSR) for supported 
> > >>>routing protocols to be enabled. Indeed, it¹s our approach that 
> > >>>the moment a routing
> > neighbor
> > >>"The major elements supporting the functionality of the SSO and 
> > >>ISSU for Cisco IOS XE VPN 6vPE and 6PE features are the following:
> > >>
> > >>[...]
> > >>for negotiating graceful restart capabilities, exchanging 
> > >>forwarding preservation states, and coordinating advertisements 
> > >>after session restarts. MPLS VPNs interact with BGP to exchange 
> > >>VPN routing and forwarding (VRF) routes and labels."
> > >>
> > >>Now if IP forwarding is probably the main service of a core 
> > >>router, an edge router may provide additional services.
> > >>If you go that way, the first step may be to define what the 
> > >>service
> is.
> > >>
> > >>>ways to state this without sounding like a conformance paragraph 
> > >>>- because BMWG doesn¹t do conformance testing - it¹s certainly 
> > >>>worth gathering input. :) Would you agree that NSR is a 
> > >>>fundamental requirement for an ISSU operation?
> > >>
> > >>stays up. But this does not mean that ISSU has no impact on the 
> > >>peer router (e.g. some implementation sends a BGP Route Refresh). 
> > >>Also AFAIK, NSR says nothing about other control plane services.(e.g.
> > >>DHCP relay/server, IGMP
> > >>request...)
> > >>
> > >>
> > >>Finally, my comment seems in synch with the proposed high level 
> > >>goal of characterizing the impact of ISSU on both the forwarding 
> > >>and control
> > >>plane:
> > >>
> > >>>>>> benchmarks to characterize the upgrade of network devices 
> > >>>>>> while in-service, considering both data and control plane 
> > >>>>>> operations and impacts. These devices are generally expected 
> > >>>>>> to maintain control plane session integrity, including routing
connections.
> > >>>>implementations may delegate some work on the neighbors (e.g. 
> > >>>>for
> BGP:
> > >>>>plane impact); the need for someone to be 
> > >>>>present/available/ready to
> > go
> > >>>>on the peers promises (the more control plane impacts on the 
> > >>>>peer, the more risks)...
> > >>>>
> > >>>>
> > >>>>
> > >>>>>> As a reminder, we have already reviewed the items below, and 
> > >>>>>> we
> > have
> > >>>>>> Configured operation will be verified as a part of the 
> > >>>>>> methodology. The goal
> > is
> > >>>>>> a methodology to assess the maximum forwarding performance 
> > >>>>>> that a network device can sustain without dropping or 
> > >>>>>> impairing packets,
> > or
> > >>>>>> between devices. Another goal is to devise methods that 
> > >>>>>> utilize flows with congestion-aware transport as part of the 
> > >>>>>> traffic load and still produce repeatable results in the 
> > >>>>>> isolated test
> environment.
> > >>>>>>
> > >>>>>> -=-=-=-=-=-=-=-=-=-
> > >>>>>>
> > >>>>>> benchmarks to the operations community. The working group 
> > >>>>>> will develop the necessary terminology and methodologies to 
> > >>>>>> measure such benchmarks.
> > >>>>________________________________________________________________
> > >>>>__
> > >>>>____
> > __
> > >>>>__
> > >>>>Ce message et ses pieces jointes peuvent contenir des 
> > >>>>informations confidentielles ou privilegiees et ne doivent donc 
> > >>>>pas etre diffuses, exploites ou copies sans autorisation. Si 
> > >>>>vous avez recu ce message par erreur, veuillez le signaler a 
> > >>>>l'expediteur et le detruire ainsi que les pieces jointes. Les
> > messages
> > >>>>If you have received this email in error, please notify the 
> > >>>>sender and delete this message and its attachments.
> > >>etant susceptibles d'alteration, Orange decline toute 
> > >>responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> > >>
> > >>privileged information that may be protected by law; they should 
> > >>not be distributed, used or copied without authorisation.
> > __ ____ _______________________________________________
> >

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 22 Jan 2014 20:40:15 -0500

I think it's fair to say that ETSI NFV is discussing system resiliency
which may include upgrades, but their scope doesn't include benchmarking
methodology development...

> Sent: Wednesday, January 22, 2014 8:16 PM