Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

From: Norman Finn <[email-masked]>
Date: Mon, 6 Feb 2017 22:22:11 +0000

Networks that have but one application can run some profile of IEEE 1588.  IMO, that's running 1588, not time as a service.  I'm not sure if that is your 2, below, or not.

You can tailor some networks to treat PTP preferentially, presumably as extremely high priority, perhaps allowing it to do preemption.  (Your 1, below, I think)  This gets problematical as the rate of PTP traffic grows, perhaps because many applications or many users want their own independent PTP instances, and of course, the individual instances are not cheap to supply with masters and maintain.  Jumps in time happen when a netquake alters path lengths.

A better approach for a great many networks will be for the network to synchronize all its components in a secure and robust manner, so that there is a "good clock" in or near every PE. That "good clock" serves as a master clock to its attached users, using most any PTP profile, perhaps one that hasn't been created, supplying time as a service.  I'm not sure if that is your 2, below, or not.  This way, there is only one good, managed, stable, PTP instance (or multiple instances, for redundancy -- that's up to the network administration).  Time is hard.  The end-to-end model may be great for lots of things, but not necessarily for time.

-- Norm

###############
 On Feb 6, 2017, at 04:22, Jiangyuanlong <[email-masked]> wrote:

Synchronization traffic can be categorized into two cases:
1. Synchronization as a service, high availability can be an advantage for this kind of service
Synchronization service is transported from an ingress PE to an egress PE, replication/elimination can optionally be applied between two PEs (i.e., on S-PEs), while all network nodes including PEs and S-PEs are not aware of the sync message themselves (that is, they don't need to process the PTP messages in the service). And the egress PE will terminate and provide a single copy of each sync packet to a destination CE.

2. Synchronization as an infrastructure
Since existing technology including SyncE and IEEE 1588 rely heavily on a tree topology to work properly, replication/elimination must not be used in this case.

My 2 cents,
Yuanlong

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of
Jouni Korhonen
Sent: Wednesday, February 01, 2017 4:25 AM
To: Balázs Varga A
Cc: Loa Andersson; Tal Mizrahi; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization
Traffic

Indeed. Not all traffic will receive the replication/elimination service. That is
based on the configuration.

- Jouni

--
Jouni Korhonen, Broadcom Ltd.
M: +1-408-391-7160

On Jan 31, 2017, at 12:13 PM, Balázs Varga A
<[email-masked]> wrote:

Hi,
For sync usually we need hop-by-hop forwarding.
So it is not clear why to consider replication (and elimination) at all?
Cheers
Bala'zs

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of
Tal Mizrahi
Sent: Friday, January 27, 2017 8:37 AM
To: Jouni Korhonen <[email-masked]>
Cc: [email-masked]; Loa Andersson <[email-masked]>
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach -
Synchronization Traffic

In a path redundancy case would the PTP messages on multiple paths be
exact copies a single origin PTP message or different PTP messages?

They would be two different PTP messages.
This implies that we would not like PTP packets to be subject to replication
and elimination.

Tal.

-----Original Message-----
From: Jouni Korhonen [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 8:40 PM
To: Tal Mizrahi
Cc: Loa Andersson; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach -
Synchronization Traffic

On Jan 25, 2017, at 11:10 PM, Tal Mizrahi <[email-masked]> wrote:

Hi Loa,

- If the answer is yes: not sure replication and elimination makes
sense for
PTP.

Why is that?

In order for PTP to work well, PTP packets have to be consistently
sent over the
same path. If packets are intermittently sent through two different
paths (with two different delays) the PTP slave will not be able to
synchronize in a stable way.
However, it *does* make sense for PTP to be sent through two
different paths
*without* elimination at the end. Such path redundancy allows the
slave to always receive the sync information through both paths,
which actually allows more robust synchronization than over a single
path.

So path redundancy is good for PTP, but elimination - not so good.

In a path redundancy case would the PTP messages on multiple paths be
exact copies a single origin PTP message or different PTP messages?

- JOuni

Cheers,
Tal.

-----Original Message-----
From: Loa Andersson [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 9:01 AM
To: Tal Mizrahi; [email-masked]
Subject: [EXT] Re: [Detnet-dp-dt] Data Plane Approach -
Synchronization Traffic

External Email

-------------------------------------------------------------------
-
-
-
Tal,

On 2017-01-26 14:44, Tal Mizrahi wrote:
Hi,

I realize this is a bit off-topic, but one of the things I am
wondering about is how
synchronization traffic (PTP) is forwarded in the context of the
data plane approaches we have been considering.
Is PTP transported as a DETNET flow?
- If the answer is no: does that mean we are assuming that
synchronization is
achieved at a different layer, possibly requiring dedicated
communication between the customer and provider?
- If the answer is yes: not sure replication and elimination makes
sense for
PTP.

Why is that?

/Loa

Will be happy to hear your thoughts, and apologies if this has
already been
discussed elsewhere and I am not aware.

Cheers,
Tal.
_______________________________________________
Detnet-dp-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet-dp-dt

--

Loa Andersson                        email:
[email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

_______________________________________________
Detnet-dp-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet-dp-dt

_______________________________________________
Detnet-dp-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet-dp-dt

_______________________________________________
Detnet-dp-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet-dp-dt
_______________________________________________
Detnet-dp-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet-dp-dt

---

From: Jiangyuanlong <[email-masked]>
Date: Tue, 7 Feb 2017 02:30:53 +0000

Norm, please see my inline comments prefixed with [YJ].

Thanks,
Yuanlong

From: Norman Finn [mailto:[email-masked]] 
Sent: Tuesday, February 07, 2017 6:22 AM
To: Jiangyuanlong
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

Networks that have but one application can run some profile of IEEE 1588.  IMO, that's running 1588, not time as a service.  I'm not sure if that is your 2, below, or not.
[YJ] Yes, I think this is case 2. DetNet network needs a high precision synchronization itself, and IEEE 1588 seems to be the appropriate technology.

You can tailor some networks to treat PTP preferentially, presumably as extremely high priority, perhaps allowing it to do preemption.  (Your 1, below, I think)  This gets problematical as the rate of PTP traffic grows, perhaps because many applications or many users want their own independent PTP instances, and of course, the individual instances are not cheap to supply with masters and maintain.  Jumps in time happen when a netquake alters path lengths.
[YJ] Yes, this is case 1. Synchronization service can also be differentiated, for example, to 3 classes: gold, silver, bronze. Only the critical users need the gold sync service (DetNet can be the very underlying technology), but the cost will be high. Some other business only need a moderate sync service, but more reliable or more accurate than the time service available over the Internet (totally free of charge;).

A better approach for a great many networks will be for the network to synchronize all its components in a secure and robust manner, so that there is a "good clock" in or near every PE. That "good clock" serves as a master clock to its attached users, using most any PTP profile, perhaps one that hasn't been created, supplying time as a service.  I'm not sure if that is your 2, below, or not.  This way, there is only one good, managed, stable, PTP instance (or multiple instances, for redundancy -- that's up to the network administration).  Time is hard.  The end-to-end model may be great for lots of things, but not necessarily for time. 
[YJ] I think this scenario is more or less aligned with Figure 4 & Figure 5 in ITU-T G.8275, that is, the PRTC (Primary Reference Time Clock) is distributed to the edge of the backhaul network. Since the sync processing is local and there is no need of network transport, this case seems out of scope of DetNet. 
[YJ] BTW, we happened to come across the same problem (synchronization in DetNet) last year, please see more details in https://mailarchive.ietf.org/arch/msg/scsn/HmC7hoCxO1TabhZdGHhFY_VOnbg

-- Norm

###############

My 2 cents,
Yuanlong

-----Original Message-----

- Jouni

Hi,

-----Original Message-----

Tal.

-----Original Message-----
From: Jouni Korhonen [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 8:40 PM
To: Tal Mizrahi
Cc: Loa Andersson; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach -
Synchronization Traffic

Hi Loa,

Why is that?

- JOuni

Cheers,
Tal.

-----Original Message-----
From: Loa Andersson [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 9:01 AM
To: Tal Mizrahi; [email-masked]
Subject: [EXT] Re: [Detnet-dp-dt] Data Plane Approach -
Synchronization Traffic

External Email

Hi,

Why is that?

/Loa

Cheers,
Tal.

--

Loa Andersson                        email:
[email-masked]

---

From: Balázs Varga A <[email-masked]>
Date: Tue, 7 Feb 2017 06:50:34 +0000

Hi,

we do not have an use-case to transport "sync over DetNet" and in my view that is intentional.
DetNet functions may rely on "time sync" of network nodes. So the "better approach" as described 
by Norm is the preferred solution. Namely, the network synchronizes all its components in a secure 
and robust manner. That "infrastructure clock" serves as a master clock to its attached endpoints and 
used by the DetNet nodes. 

If "gold, silver, bronze" sync service have to be provided for endpoints, than gold requirements are 
needed in the transport network nodes and You can destroy the accuracy level (over the UNI interface)
to "bronze" if You intend to do so.

Cheers
Bala'zs

-----Original Message-----
From: Jiangyuanlong [mailto:[email-masked]] 
Sent: Tuesday, February 07, 2017 3:31 AM
To: Norman Finn 
Cc: [email-masked]; Balázs Varga A ; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

Thanks,
Yuanlong

-- Norm

###############

Synchronization traffic can be categorized into two cases:
1. Synchronization as a service, high availability can be an advantage for this kind of service Synchronization service is transported from an ingress PE to an egress PE, replication/elimination can optionally be applied between two PEs (i.e., on S-PEs), while all network nodes including PEs and S-PEs are not aware of the sync message themselves (that is, they don't need to process the PTP messages in the service). And the egress PE will terminate and provide a single copy of each sync packet to a destination CE.

2. Synchronization as an infrastructure Since existing technology including SyncE and IEEE 1588 rely heavily on a tree topology to work properly, replication/elimination must not be used in this case.

My 2 cents,
Yuanlong

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Jouni Korhonen
Sent: Wednesday, February 01, 2017 4:25 AM
To: Balázs Varga A
Cc: Loa Andersson; Tal Mizrahi; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

Indeed. Not all traffic will receive the replication/elimination service. That is based on the configuration.

- Jouni

On Jan 31, 2017, at 12:13 PM, Balázs Varga A <[email-masked]> wrote:

Hi,

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Tal Mizrahi
Sent: Friday, January 27, 2017 8:37 AM
To: Jouni Korhonen <[email-masked]>
Cc: [email-masked]; Loa Andersson <[email-masked]>
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

In a path redundancy case would the PTP messages on multiple paths be exact copies a single origin PTP message or different PTP messages?

They would be two different PTP messages.
This implies that we would not like PTP packets to be subject to replication and elimination.

Tal.

-----Original Message-----
From: Jouni Korhonen [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 8:40 PM
To: Tal Mizrahi
Cc: Loa Andersson; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

Hi Loa,

- If the answer is yes: not sure replication and elimination makes sense for PTP.

Why is that?

In order for PTP to work well, PTP packets have to be consistently sent over the same path. If packets are intermittently sent through two different paths (with two different delays) the PTP slave will not be able to synchronize in a stable way.

However, it *does* make sense for PTP to be sent through two different paths
*without* elimination at the end. Such path redundancy allows the slave to always receive the sync information through both paths, which actually allows more robust synchronization than over a single path.

In a path redundancy case would the PTP messages on multiple paths be exact copies a single origin PTP message or different PTP messages?

- JOuni

Cheers,
Tal.

-----Original Message-----
From: Loa Andersson [mailto:[email-masked]]
Sent: Thursday, January 26, 2017 9:01 AM
To: Tal Mizrahi; [email-masked]
Subject: [EXT] Re: [Detnet-dp-dt] Data Plane Approach - Synchronization Traffic

External Email

Hi,

I realize this is a bit off-topic, but one of the things I am wondering about is how synchronization traffic (PTP) is forwarded in the context of the data plane approaches we have been considering.

Is PTP transported as a DETNET flow?
- If the answer is no: does that mean we are assuming that synchronization is achieved at a different layer, possibly requiring dedicated communication between the customer and provider?

- If the answer is yes: not sure replication and elimination makes sense for PTP.

Why is that?

/Loa

Will be happy to hear your thoughts, and apologies if this has already been discussed elsewhere and I am not aware.

Cheers,
Tal.

--

Loa Andersson                        email:
[email-masked]

Senior MPLS Expert                          [email-masked] Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: Loa Andersson <[email-masked]>
Date: Tue, 7 Feb 2017 23:06:25 +0800

Folks,

Let me re-iterate the PTP and replication/elimination discussion from my 
understanding of synchronization (which is close to zero).

The reason why we not want to use a DetNet network that do replication 
and elimination is that the intermediate nodes "fiddle" with the
synchronization packets, and when doing so they leave traces, so the
target node will have a hard time (or can't) coordinate the packet flow
and the synchronization is lost.

On the other hadnd this needs not be  problem since not all DetN,
traffic will b e replicated/eliminated, just send the traffic on a
path that is true hop-by-hop.

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 7 Feb 2017 16:05:24 +0000

Hi Loa,

Here is my understanding:

>The reason why we not want to use a DetNet network that do replication and
>elimination is that the intermediate nodes "fiddle" with the synchronization
>packets, and when doing so they leave traces, so the target node will have a hard
>time (or can't) coordinate the packet flow and the synchronization is lost.

The reason we do not want replication/elimination is that if PTP packets are intermittently sent through two different paths, they would intermittently have two delay values. Having a non-constant delay makes it hard for the PTP slave algorithm to accurately synchronize.

There are other issues that have been discussed on this thread, and I will try to summarize these in the DETNET DT meeting tomorrow (or today, depending on the time zone :).

Cheers,
Tal.

>From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Loa
>Andersson
>Sent: Tuesday, February 07, 2017 5:06 PM
>To: [email-masked]
>The reason why we not want to use a DetNet network that do replication and
>elimination is that the intermediate nodes "fiddle" with the synchronization
>packets, and when doing so they leave traces, so the target node will have a hard
>time (or can't) coordinate the packet flow and the synchronization is lost.
>
>On the other hadnd this needs not be  problem since not all DetN, traffic will b e
>replicated/eliminated, just send the traffic on a path that is true hop-by-hop.
>
>/Loa
>
>> DetNet functions may rely on "time sync" of network nodes. So the
>> "better approach" as described by Norm is the preferred solution.
>> Namely, the network synchronizes all its components in a secure and
>> robust manner. That "infrastructure clock" serves as a master clock to its
>attached endpoints and used by the DetNet nodes.
>>
>> If "gold, silver, bronze" sync service have to be provided for
>> endpoints, than gold requirements are needed in the transport network
>> nodes and You can destroy the accuracy level (over the UNI interface) to
>"bronze" if You intend to do so.
>>
>> Cheers
>> Bala'zs
>>
>> Cc: [email-masked]; [email-masked];
>> [email-masked]; [email-masked]; [email-masked]
>> Networks that have but one application can run some profile of IEEE 1588.
>IMO, that's running 1588, not time as a service.  I'm not sure if that is your 2,
>below, or not.
>> [YJ] Yes, I think this is case 2. DetNet network needs a high precision
>synchronization itself, and IEEE 1588 seems to be the appropriate technology.
>>
>> You can tailor some networks to treat PTP preferentially, presumably as
>extremely high priority, perhaps allowing it to do preemption.  (Your 1, below, I
>think)  This gets problematical as the rate of PTP traffic grows, perhaps because
>many applications or many users want their own independent PTP instances, and
>of course, the individual instances are not cheap to supply with masters and
>maintain.  Jumps in time happen when a netquake alters path lengths.
>> [YJ] Yes, this is case 1. Synchronization service can also be differentiated, for
>example, to 3 classes: gold, silver, bronze. Only the critical users need the gold
>sync service (DetNet can be the very underlying technology), but the cost will be
>high. Some other business only need a moderate sync service, but more reliable
>or more accurate than the time service available over the Internet (totally free of
>charge;).
>>
>> A better approach for a great many networks will be for the network to
>synchronize all its components in a secure and robust manner, so that there is a
>"good clock" in or near every PE. That "good clock" serves as a master clock to its
>attached users, using most any PTP profile, perhaps one that hasn't been created,
>supplying time as a service.  I'm not sure if that is your 2, below, or not.  This way,
>there is only one good, managed, stable, PTP instance (or multiple instances, for
>redundancy -- that's up to the network administration).  Time is hard.  The end-
>to-end model may be great for lots of things, but not necessarily for time.
>> [YJ] I think this scenario is more or less aligned with Figure 4 & Figure 5 in ITU-T
>G.8275, that is, the PRTC (Primary Reference Time Clock) is distributed to the
>edge of the backhaul network. Since the sync processing is local and there is no
>need of network transport, this case seems out of scope of DetNet.
>> [YJ] BTW, we happened to come across the same problem (synchronization
>> in DetNet) last year, please see more details in
>>
>https://mailarchive.ietf.org/arch/msg/scsn/HmC7hoCxO1TabhZdGHhFY_VOnbg
>>
>> -- Norm
>>
>> ###############
>> 1. Synchronization as a service, high availability can be an advantage for this
>kind of service Synchronization service is transported from an ingress PE to an
>egress PE, replication/elimination can optionally be applied between two PEs (i.e.,
>on S-PEs), while all network nodes including PEs and S-PEs are not aware of the
>sync message themselves (that is, they don't need to process the PTP messages in
>the service). And the egress PE will terminate and provide a single copy of each
>sync packet to a destination CE.
>>
>> 2. Synchronization as an infrastructure Since existing technology including
>SyncE and IEEE 1588 rely heavily on a tree topology to work properly,
>replication/elimination must not be used in this case.
>>
>> My 2 cents,
>> Yuanlong
>>
>>
>> In a path redundancy case would the PTP messages on multiple paths be exact
>copies a single origin PTP message or different PTP messages?
>>
>> In order for PTP to work well, PTP packets have to be consistently sent over the
>same path. If packets are intermittently sent through two different paths (with
>two different delays) the PTP slave will not be able to synchronize in a stable way.
>>
>> *without* elimination at the end. Such path redundancy allows the slave to
>always receive the sync information through both paths, which actually allows
>more robust synchronization than over a single path.
>>
>>
>> In a path redundancy case would the PTP messages on multiple paths be exact
>copies a single origin PTP message or different PTP messages?
>>
>> - JOuni
>>
>>
>>
>>
>> Cheers,
>> Tal.
>>
>>
>>
>> I realize this is a bit off-topic, but one of the things I am wondering about is how
>synchronization traffic (PTP) is forwarded in the context of the data plane
>approaches we have been considering.
>>
>> - If the answer is no: does that mean we are assuming that synchronization is
>achieved at a different layer, possibly requiring dedicated communication
>between the customer and provider?
>>

---

From: Loa Andersson <[email-masked]>
Date: Wed, 8 Feb 2017 12:57:49 +0800

Tal,

On 2017-02-08 00:05, Tal Mizrahi wrote:
> The reason we do not want replication/elimination is that if PTP packets are intermittently sent through two different paths, they would intermittently have two delay values.

This may be self-evident, but I don't really follow.

Any network will give you some delay variation, the PTP packet might 
sometime fall behind a larger packet, sometimes be forwarded ahead of
the large packet.

Now in the DetNet scenario (the slides we had before) where A
replicate the packet to B and C, which forward to D, and where D
eliminate the late packet.

There is only one packet coming through, the one that had the least 
delay of a replicated pair. I don't immediately see why this is a
problem.

Is it obvious that the DetNEt scenario causes larger delay variation 
than a network without DetNet cpabilities?

Looking forward to the discussion.

/Loa

Having a non-constant delay makes it hard for the PTP slave algorithm to 
accurately synchronize.

--

---

From: Norman Finn <[email-masked]>
Date: Sat, 11 Feb 2017 00:45:56 +0000

There are at least two scenarios of interest:

1. Occasional lost packets.  I'm not sure what the clock recovery algorithm will do if occasionally, a packet is delivered late (because the packet on the fast path was lost, so the backup path supplies the packet).  Some algorithms are better at discarding noise than others.  You're probably better off dropping the late packet.  It's less work, and any algorithm should be able to handle occasional losses.

2. The fast path fails, the slow path takes over, and perhaps some time later, the fast path is restored.  I'm sure that the clock recovery algorithm will do something reasonable with the sudden delay jump.  But, I would imagine that the recovered clock will wander for a while after both events.  I think a much better method is to let the recovery algorithm receive both streams, with no DetNet interference, and deal with failures in its own clever way.  We have lots of reasons to want to handle multiple paths and/or multiple masters, including security; let's take advantage of that ability.

So, I see PTP-over-Packet-Replication as extra work for no benefit.

-- Norm
________________________________________
From: Detnet-dp-dt [[email-masked]] on behalf of Loa Andersson [[email-masked]]
Sent: Tuesday, February 07, 2017 8:57 PM
To: Tal Mizrahi; [email-masked]
Subject: Re: [Detnet-dp-dt] [EXT] Re: Data Plane Approach - Synchronization Traffic

Tal,

/Loa

--