Subject: Re: [ntpwg] [ntp:questions] Idea to improve ntpd accuracy

From: Tal Mizrahi <[email-masked]>
Date: Fri, 26 Feb 2016 09:22:36 +0000

Hi,

I believe NTP hardware timestamping is certainly a very interesting direction.
It is certainly worth taking a look at https://datatracker.ietf.org/doc/draft-ietf-ntp-checksum-trailer

I wonder what is the level of accuracy you are looking for.

The factor that affects the NTP client's accuracy more than anything else is the distance (latency) between the NTP server and the NTP client, which is roughly proportional to the delay variation.
If the NTP server and the NTP client are on the same LAN, I would estimate that you can achieve an accuracy on the order of tens of microseconds with software timestamping. In this case hardware timestamping may improve the accuracy.
If the latency between the NTP server and client is tens of *milliseconds*, then hardware timestamping will hardly affect the accuracy.

Please note that even if you have hardware-based timestamping, using a hardware clock in the NIC, you still need to synchronize the hardware clock with the system clock (for example, this can be done using phc2sys). This procedure will typically add some inaccuracy to the overall calculation...

>>>> What if the poll response packet contained a flag or indication of
>>>> some sort which means "this is an approximate transmit timestamp".
>>>> That packet would then be immediately followed by a second response
>>>> packet with a more accurate transmit time. The second packet could
>>>> be otherwise identical to the first, or it could be a new flavor of
>>>> packet that contained only the transmit time (that would save on network
>bandwidth).
>>>>

Sounds a bit like interleave mode, right (a bit similar to PTP two-step clocks)?

Regards,
Tal.

>-----Original Message-----
>From: Danny Mayer [mailto:[email-masked]]
>Sent: Friday, February 26, 2016 5:51 AM
>To: Weber
>Cc: [email-masked]; Tal Mizrahi
>Subject: Re: [ntp:questions] Idea to improve ntpd accuracy
>
>On 2/25/2016 10:29 PM, Weber wrote:
>> Thanks for the link. I'm not surprised that someone else already had
>> the idea. I was poking around and see that 1588 does something similar
>> with a "follow up" packet.
>>
>
>I should have mentioned that this checksum trailer extension field is needed to
>get the idea to work for NTP. Unfortunately you can't have security or a MAC
>to do this as it needs to make changes to the timestamps and then add the
>extension field to fix the UDP checksum. Tal can say more.
>
>Danny
>>
>> On 2/25/2016 7:09 PM, Danny Mayer wrote:
>>> Already thought of so it's a good idea! See
>>> https://datatracker.ietf.org/doc/draft-ietf-ntp-checksum-trailer/ for
>>> details.
>>>
>>> Danny
>>>
>>> On 2/25/2016 4:52 PM, Weber wrote:
>>>> This may or may not be worthwhile, but I thought I'd throw it out
>>>> there and see what happens:
>>>>
>>>> Recent work testing some microsecond-accurate NTP servers lead me to
>>>> an idea that could improve accuracy of measurements made by ntpd.
>>>> These NTP servers have hardware timestamps on receive but that's not
>>>> possible on transmit w/o a custom NIC. I've seen this issue discussed
>before.
>>>>
>>>> The next best thing is to generate the transmit timestamp based on a
>>>> guess as to how long it takes the NIC to get on the wire and send
>>>> the packet. That works pretty well as long as there's no other
>>>> network traffic. In this situation, it is possible to make use of
>>>> microsecond accuracy in an NTP server.
>>>>
>>>> Now, add some typical network traffic and the time it takes the NIC
>>>> to get on the wire becomes unpredictable to the tune of 200us or
>>>> more (for
>>>> 100 base-T Ethernet). The server's microsecond accuracy is largely
>>>> lost in the noise.
>>>>
>>>> The NIC generates an interrupt after the packet is sent which can
>>>> result in a fairly accurate trailing hardstamp. The problem is...the
>>>> packet is already gone and has the wrong transmit timestamp.
>>>>
>>>> Here's my idea:
>>>>
>>>> What if the poll response packet contained a flag or indication of
>>>> some sort which means "this is an approximate transmit timestamp".
>>>> That packet would then be immediately followed by a second response
>>>> packet with a more accurate transmit time. The second packet could
>>>> be otherwise identical to the first, or it could be a new flavor of
>>>> packet that contained only the transmit time (that would save on network
>bandwidth).
>>>>
>>>> The ntpd process would need to use the receive time of the first
>>>> packet (the one with an approximate tx timestamp) and merge in the
>>>> following accurate tx timestamp before performing the normal
>>>> processing associated with a poll response.
>>>>
>>>> Here are the pros and cons I can think of:
>>>>
>>>> Pros
>>>>
>>>> * Possible accuracy improvement of 1-2 orders of magnitude. I know
>>>> ntpd already does some work to try and filter out network delay
>>>> variation so the improvement might not be a full 2 orders of magnitude.
>>>> * Could potentially be made compatible backwards compatible with ntp
>>>> 3/4 protocols
>>>>
>>>> Cons
>>>>
>>>> * Increased network traffic
>>>> * Improvement to that level of accuracy might not be of interest to
>>>> anyone
>>>> * Could be a fair bit of work for at least a couple of folks
>>>> * I may have (or probably) missed some stuff regarding network
>>>> behavior that would reduce the level of improvement that could be
>realized.
>>>> * Perhaps this is less of an issue on G-bit Ethernet?
>>>>
>>>> Wondering if anyone thinks this idea is worth pursuing further...?
>>>
>>>
>> _______________________________________________
>> questions mailing list
>> [email-masked]
>> http://lists.ntp.org/listinfo/questions
>>
>>

_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Martin Burnicki <[email-masked]>
Date: Fri, 26 Feb 2016 14:21:47 +0100

> Please note that even if you have hardware-based timestamping, using a hardware clock in the NIC, you still need to synchronize the hardware clock with the system clock (for example, this can be do…

And the remaining problem will be that there are switches and routers
which do hardware timestamping on PTP packets to measure the latency of
forwarded packets, but (AFAIK) there are no such devices which can do
this with NTP packets.

So even if both end nodes support hardware timestamping, the result will
unfortunately still be worse than with PTP.

> Sounds a bit like interleave mode, right (a bit similar to PTP two-step clocks)?

But the problem is still that (again, AFAIK) NTP interleave mode works
only in broadcast mode.

In broadcast mode ntpd only tries to measure the packet delay when the
association is first created, but the network route etc. can change at
any time, so IMO interleave mode provides only limited benefit.

Please let me know if I'm missing something.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Miroslav Lichvar <[email-masked]>
Date: Fri, 26 Feb 2016 16:19:26 +0100

> But the problem is still that (again, AFAIK) NTP interleave mode works

It works also in the symmetric mode.

> any time, so IMO interleave mode provides only limited benefit.

It probably wouldn't be difficult to modify ntpd to run the delay
measurement periodically.

A bigger problem is that the delay is measured using the client/server
mode packets, for which there is no interleaved mode. If the delay is
inaccurate, the offset is inaccurate too, even if the tx timestamp in
the broadcast packet was accurate.

Broadcast clients would need to create ephemeral symmetric
associations with the server in order to measure the delay accurately.

-- 
Miroslav Lichvar