Subject: [ntpwg] Will NTPv5 be part of Next IETF discussion agenda ?

From: "Anil Kumar S N (VRP Network BL)" <[email-masked]>
Date: Mon, 5 Oct 2015 12:08:00 +0000

Hi Folks,

I think we need to address below questions :
Do we really need NTPv5 ?
Will it help to solve current NTP issues?
Are we expecting it in near future  ?
What could be the problem statement/requirements and Use cases for NTPv5 ?

Few thoughts :
In the old times before we had the Internet our computers already had an internal clock. It was just up to us to make sure it shows the correct time. These days we have something much more convenient - and less secure. We can set our clocks through the Internet from time servers. This is usually done with NTP.
Network Time Protocol (NTP) is one of the earliest things to run over the Internet, and it is also one of the longest-running applications on the Internet. It's 30 years' ongoing development and maintenance have been almost exclusively performed by volunteers.
Anyone can tamper with your NTP requests and send you a wrong time. Is this a problem? It turns out it is. Modern TLS connections increasingly rely on the system time as a part of security concepts. This includes certificate expiration, OCSP revocation checks, HSTS and HPKP. All of these have security considerations that in one way or another expect the time of your system to be correct.

There is a perception that the standardized attempts to add authentication to NTP are a) fairly horrible (ASN.1 etc.) and b) rarely deployed.

Problems with NTP :

1. Firewall administration's understandable reticence to open UDP port 123 on the perimeter to a public NTP server on the Internet.

2. Network administration's understandable reticence to trust the network time of an external time source.

3. The possibility that the source of the time signal could be spoofed, particularly as communication is over UDP, resulting in an incorrect time being utilized.

4. The possibility that UDP port 123 could be subjected to a DoS attack, therefore preventing time synchronization.

5. The possibility of a remote exploit that could give external access to the internal NTP server.

6. NTP version 3 and SNTP have no built in security. Version 4 can optionally be secured but the balance is that encrypting traffic and or verifying checksums is going to slow down the transfer of packets therefore making the system inaccurate. Windows clients use NTP V3. Most NTP systems are not secured.

7. The most troublesome problems have involved NTP server addresses hardcoded in the firmware of consumer networking devices. One particularly common software error is to generate query packets at short (less than five second) intervals until a response is received. When such an implementation finds itself behind a packet filter that refuses to pass the incoming response, this results in a never-ending stream of requests to the NTP server.

8. Kiss-o'-Death (KoD) packets
"In NTPv4 and SNTPv4, packets of this kind are called Kiss-o'-Death (KoD) packets, and the ASCII messages they convey are called kiss codes. The KoD packets got their name because an early use was to tell clients to stop sending packets that violate server access controls".

The new requirements of the NTP protocol do not work retroactively, and old clients and implementations of earlier version of the protocol do not recognize KoD and act on it. For the time being there are no good technical means to counteract misuse of NTP servers.

9. To trace or debug all major issues had a huge cost to pay with respect to time & revenue. Robust OAM for NTP is needed.

10. swisstime.ethz.ch and the Providers
For over 20 years ETH Zurich has provided open access to the time server swisstime.ethz.ch for operational time synchronization. Due to excessive bandwidth usage, averaging upwards of 20 GB / day, it has become necessary to direct external usage to public time server pools, such as ch.pool.ntp.org. Misuse, caused mostly by IT-providers synchronizing their client infrastructures, has made unusually high demands on network traffic, thereby causing ETH to take effective measures. As of Fall 2012, the availability of swisstime.ethz.ch has been changed to Closed Access. Since beginning of July 2013, access to the server is blocked entirely for the ntp protocol.

10. Below suggestion from David L. Mills RFC 4330 is usually not respected :

   1.  A client MUST NOT under any conditions use a poll interval less
       than 15 seconds.

   2.  A client SHOULD increase the poll interval using exponential
       backoff as performance permits and especially if the server does
       not respond within a reasonable time.

   3.  A client SHOULD use local servers whenever available to avoid
       unnecessary traffic on backbone networks.

   4.  A client MUST allow the operator to configure the primary and/or
       alternate server names or addresses in addition to or in place of
       a firmware default IP address.

   5.  If a firmware default server IP address is provided, it MUST be a
       server operated by the manufacturer or seller of the device or
       another server, but only with the operator's permission.

   6.  A client SHOULD use the Domain Name System (DNS) to resolve the
       server IP addresses, so the operator can do effective load
       balancing among a server clique and change IP address binding to
       canonical names.

   7.  A client SHOULD re-resolve the server IP address at periodic
       intervals, but not at intervals less than the time-to-live field
       in the DNS response.

   8.  A client SHOULD support the NTP access-refusal mechanism so that
       a server kiss-o'-death reply in response to a client request
       causes the client to cease sending requests to that server and to
       switch to an alternate, if available.

Reference & More Information :
http://www.dba-oracle.com/forensics/t_forensics_ntp.htm
https://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse
https://blog.hboeck.de/archives/863-Dont-update-NTP-stop-using-it.html
http://support.ntp.org/bin/view/Main/SecurityNotice#Buffer_overflow_in_ctl_putdata
https://marc.info/?l=openbsd-tech&m=142356166731390&w=2

Thanks & Regards
Anil S N

"Be liberal in what you accept, and conservative in what you send" - Jon Postel

_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 5 Oct 2015 17:53:05 +0200

Hi Anil,

> Will it help to solve current NTP issues? 

I think this mainly depends on what you call "NTP issues". See my
comments below.

> In the old times before we had the Internet our computers already had an
> internal clock. It was just up to us to make sure it shows the correct
> time. These days we have something much more convenient – and less
> secure. We can set our clocks through the Internet from time servers.
> This is usually done with NTP.
> 
> Network Time Protocol (NTP) is one of the earliest things to run over
> the Internet, and it is also one of the longest-running applications on
> the Internet. It’s 30 years’ ongoing development and maintenance have
> been almost exclusively performed by volunteers.
> 
> Anyone can tamper with your NTP requests and send you a wrong time. Is
> this a problem? It turns out it is. Modern TLS connections increasingly
> rely on the system time as a part of security concepts. This includes
> certificate expiration, OCSP revocation checks, HSTS and HPKP. All of
> these have security considerations that in one way or another expect the
> time of your system to be correct.

I think this problem is not specific to NTP. If you use e.g. DNS without
security features then you can also easily be spoofed by "bad guys".

For NTP, autokey (and also symmetric keys alreeady supported by NTPv3)
were intended to make this more secure. Recently it has turned out that
autokey isn't as secure as expected, and algorithms used with symmetric
keys might have to be replaced by newer ones since the original ones can
easily be cracked with today's conmputing power.

NTS is going to be a replacement for autokey, but the question is
whether this justifies a new major version of the protocol, or if we can
simply call it a new feature of NTPv4?

> There is a perception that the standardized attempts to add
> authentication to NTP are a) fairly horrible (ASN.1 etc.) and b) rarely
> deployed.

If existing security features are not used then you can always be
fooled, e.g. if you'd access your bank's web site via http instead of https.

And of course, if a security feature turns out to be vulnerable then
this has to be changed and fixed, but again IMO this is not a specific
problem with NTP but with every protocol.

> Problems with NTP :
> 
>  
> 
> 1. Firewall administration's understandable reticence to open UDP port
> 123 on the perimeter to a public NTP server on the Internet.

If they don't want to provide NTP service they don't need to open the
port. If I want to provide http service I also have to open port 80, or
use a non-well-known port (security by obscurity).

> network time of an external time source.

Alternatively they can install their own primary time server driven by a
hardware refclock.

> 3. The possibility that the source of the time signal could be spoofed,
> particularly as communication is over UDP, resulting in an incorrect
> time being utilized.

It's also very easy to spoof the time of a radio clock. So even if you
you have the most secure network protocol then bad guys could easily
spoof your long wave or GPS receivers.

You'd have to do plausibility checks across a number of independent time
sources, don't accept time steps, etc., to reduce the vulnerability
against such attacks.

> 4. The possibility that UDP port 123 could be subjected to a DoS attack,
> therefore preventing time synchronization.

Again, this is not specific to NTP. You could also send of flood of DNS
requersts or something else as a DoS attack.

Again, the alternative would be to use an own independent time server on
a network protected by a firewall.

> to the internal NTP server.

This is also possible with other services if a vulnerability is
detected, and not only with ntpd.

> 6. NTP version 3 and SNTP have no built in security.

Didn't it support symmetric keys?

> Version 4 can
> optionally be secured but the balance is that encrypting traffic and or
> verifying checksums is going to slow down the transfer of packets
> therefore making the system inaccurate.

> Windows clients use NTP V3.

Which Windows clients? w32time which has anyway only been a SNTP
implementation? Ntpd is available for Windows and supports the security
features.

> Most NTP systems are not secured.

As said above, if users/admins don't use existing security features,
what would you do against it?

A possible way would be to *force* usage of some security features. I'm
not a crypto expert, but as far as I can see this also works *only*
reliable if folks keep their private keys really secret, or use some 3rd
party certifacate authority.

> 7. The most troublesome problems have involved NTP server addresses
> hardcoded in the firmware of consumer networking devices. One
> particularly common software error is to generate query packets at short
> (less than five second) intervals until a response is received. When
> such an implementation finds itself behind a packet filter that refuses
> to pass the incoming response, this results in a never-ending stream of
> requests to the NTP server.

If there are dumb manufacturers who do this, why would you blame the NTP
protocol?

If some network device have hardcoded DNS server entries or whatever,
would you blame the DNS protocol?

If there was a dumb manufacturer who writes his software such that each
DNS lookup is directly sent to the root DNS servers instead of the ones
provided by your ISP, would you blame the DNS protocol or the manufacturer?

> "In NTPv4 and SNTPv4, packets of this kind are called Kiss-o'-Death
> (KoD) packets, and the ASCII messages they convey are called kiss codes.
> The KoD packets got their name because an early use was to tell clients
> to stop sending packets that violate server access controls".
> 
> The new requirements of the NTP protocol do not work retroactively, and
> old clients and implementations of earlier version of the protocol do
> not recognize KoD and act on it. For the time being there are no good
> technical means to counteract misuse of NTP servers.

This is a general problem. The NTP reference implementation which would
respect KoD packets received from a server usually doesn't need to do
this because the implementation is restrictive anyway.

On the other hand, if someone does a dumb implementation which floods a
server with requests, what else should a protocol do against it?

Coming back to my DNS example, what would you say if some one implements
a DNS clients which sends the same request to a DNS server repeatedly if
the DNS server is unable to resolve the requested name? Blame the DNS
protocol?

> 9. To trace or debug all major issues had a huge cost to pay with
> respect to time & revenue. Robust OAM for NTP is needed.

This is always the case if you have to chase a bug, or need to find out
why something doesn't work as you expected.

> For over 20 years ETH Zurich has provided open access to the time server
> swisstime.ethz.ch for operational time synchronization. Due to excessive
> bandwidth usage, averaging upwards of 20 GB / day, it has become
> necessary to direct external usage to public time server pools, such as
> ch.pool.ntp.org. Misuse, caused mostly by IT-providers synchronizing
> their client infrastructures, has made unusually high demands on network
> traffic, thereby causing ETH to take effective measures. As of Fall
> 2012, the availability of swisstime.ethz.ch has been changed to Closed
> Access. Since beginning of July 2013, access to the server is blocked
> entirely for the ntp protocol.

This is more related to something like netiquette than to a specific
protocol.

>        switch to an alternate, if available.

All of the above sounds good, but what if someone doesn't care if he
implements the protocol.

Please don't get me wrong. I not against a new protocol version, but
there should be some benefit from it, e.g. support for different time
scales.

IMO nost of the points above are neither specific to the NTP protocol,
nor to the reference implementation, nor could they be fixed/avoided by
just rewriting ntpd or a new version of the protocol.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
_______________________________________________
ntpwg mailing list
[email-masked]
http://lists.ntp.org/listinfo/ntpwg

---

From: "Anil Kumar S N (VRP Network BL)" <[email-masked]>
Date: Tue, 6 Oct 2015 04:09:20 +0000

Martin,

I am neither for new NTP version nor against it. If at all we are going for new version then what are we considering is the question
As I remember in last webex meeting in NTPv5 topic was poped up. 

I would like take up below comment specifically :
As said above, if users/admins don't use existing security features,
what would you do against it?

I feel, We should see the difficulties faced by network administrators why they are not using latest security features proposed for NTP. 
and we should reason it out. I am sure its not misconfiguration or they over looked it unintensionally.

Need for NTP can be seen in Data center VMs, SDN framework,  increase in number of devices in the network etc... 
Lot of application assume time is synchronized if NTP is configured. NTP is basic building block but the real usecase is different.

My point is NTP need to be adapt to these different sceanrios. NTP algorithm no doubt its stable and has fewer issues. 
Adaptability of the NTP with respect to different network sceanrio needs more flexibility in NTP.

Ex: NTP should be able to run on top of UDP, TCP, Raw IP ext. Basically detaching algorithm from underlay network. 
In case of access network a huge number of client to a single server under single service provider, here Scalabilty is important so UDP can be prefered. 
In case of using external NTP server security takes high importance TCP can be used. 
If any application has a secured connection to a NTP server and if policy permits can just exchange time stamps.
Use IGP/multicast protocol to flood Time stamps, it might be possible.
Any usecase best suited for an user and his sceanrio where he feels secure and comfortable.

Summary :
I feel split the NTP RFC into parts :
All topics can be evolved with time.
a) Algorithms
b) Architecture
c) Security
d) Underlay network. 

Please see Inline [[Anil >>]]

Thanks & Regards
Anil S N

> -----Original Message-----
> From: Martin Burnicki [mailto:[email-masked]]
> Sent: 05 October 2015 21:23
> To: Anil Kumar S N (VRP Network BL); [email-masked]
> Cc: [email-masked]
> Subject: Re: [ntpwg] Will NTPv5 be part of Next IETF discussion agenda ?
> 
> Hi Anil,
> 
> > In the old times before we had the Internet our computers already had
> > an internal clock. It was just up to us to make sure it shows the
> > correct time. These days we have something much more convenient - and
> > less secure. We can set our clocks through the Internet from time
> servers.
> > the Internet, and it is also one of the longest-running applications
> > on the Internet. It's 30 years' ongoing development and maintenance
> > have been almost exclusively performed by volunteers.
> >
> > this a problem? It turns out it is. Modern TLS connections
> > increasingly rely on the system time as a part of security concepts.
> > This includes certificate expiration, OCSP revocation checks, HSTS
> and
> > HPKP. All of these have security considerations that in one way or
> > another expect the time of your system to be correct.
> 
> I think this problem is not specific to NTP. If you use e.g. DNS
> without security features then you can also easily be spoofed by "bad
> guys".
[[Anil >>]] DNS continues to be a nice target for hackers cannot be taken as reference 
With respect to security features when it comes to backward compatibility and large number of devices.
> keys might have to be replaced by newer ones since the original ones
> can easily be cracked with today's conmputing power.
> 
> whether this justifies a new major version of the protocol, or if we
> can simply call it a new feature of NTPv4?
[[Anil >>]] I am not against NTS but as per Inherent Association Protocol Messages still uses X.509 certificates which is similar to Autokey but not same.
6.1.1.  Preconditions for the Unicast Time Synchronization Exchange insists some means for secure exchange of key input value KIV and the cookie. 
This is going to be a challenge when NTP server which you are using may not be owned by the same service provider. 
So managing certificates and exchanging key information securely and refreshing them on time will be a challenge. 
My point is from IP core networks as there are 1000's of devices running NTP which is basic for many applications.

> > authentication to NTP are a) fairly horrible (ASN.1 etc.) and b)
> > rarely deployed.
> 
[[Anil >>]] HTTPS run over any reliable stream transport protocol. Normally that's TCP, but it could also be SCTP. It is NOT expected to run over UDP, which is an unreliable datagram protocol. 
It uses TCP. It would be difficult to run it on UDP without the guarantee of packets arriving. If the packets don't arrive, the encrypted data will not be decipherable.
What I am looking at is balance b/w scalability and security of NTP. 
Does it makes sense to provide an option for NTP to use TCP or UDP based on need ?

> use a non-well-known port (security by obscurity).
[[Anil >>]] NTP can be run on non standard ports with in a administrator maintained network,
This is what is done in most of the networks, use private port to avoid port scanning and attack.
There are workarounds otherwise NTP wouldn't have survived this far.

> Alternatively they can install their own primary time server driven by
> a hardware refclock.
[[Anil >>]] :) That is not NTP meant for right ?
> 
> > 3. The possibility that the source of the time signal could be
> > spoofed, particularly as communication is over UDP, resulting in an
> > incorrect time being utilized.
> 
> You'd have to do plausibility checks across a number of independent
> time sources, don't accept time steps, etc., to reduce the
> vulnerability against such attacks.
 [[Anil >>]] As I have seen in the real core IP network, there are no real multiple service choice.
The problem is there are 1000's of routers and devices all needs NTP now, To provide hierarchy one of the 
Internal routers are used as NTP server so options are to few. One primary server and other backup server at max. 
> 
> > 4. The possibility that UDP port 123 could be subjected to a DoS
> > attack, therefore preventing time synchronization.
> 
> Again, the alternative would be to use an own independent time server
> on a network protected by a firewall.
[[Anil >>]] This is not a acceptable solution, We need a external time source which is standard but NTP be resilient to attacks
> 
> > 5. The possibility of a remote exploit that could give external
> access
> > to the internal NTP server.
> 
[[Anil >>]] Agreed.
> > optionally be secured but the balance is that encrypting traffic and
> > or verifying checksums is going to slow down the transfer of packets
> reliable if folks keep their private keys really secret, or use some
> 3rd party certifacate authority.
> 
> > particularly common software error is to generate query packets at
> > short (less than five second) intervals until a response is received.
> > When such an implementation finds itself behind a packet filter that
> > refuses to pass the incoming response, this results in a never-ending
> > stream of requests to the NTP server.
> 
> If there are dumb manufacturers who do this, why would you blame the
> NTP protocol?
> 
[[Anil >>]] Anyways this has happened with DLink WLAN routers. 
There must some convincing reason or manageability reason which have forced them to do so.
> > The KoD packets got their name because an early use was to tell
> > clients to stop sending packets that violate server access controls".
> >
> > The new requirements of the NTP protocol do not work retroactively,
> > and old clients and implementations of earlier version of the
> protocol
> > do not recognize KoD and act on it. For the time being there are no
> > good technical means to counteract misuse of NTP servers.
> 
> Coming back to my DNS example, what would you say if some one
> implements a DNS clients which sends the same request to a DNS server
> repeatedly if the DNS server is unable to resolve the requested name?
> Blame the DNS protocol?
> 
[[Anil >>]] I would like NTP to provide a strong platform to do so.
> > For over 20 years ETH Zurich has provided open access to the time
> > server swisstime.ethz.ch for operational time synchronization. Due to
> > excessive bandwidth usage, averaging upwards of 20 GB / day, it has
> > become necessary to direct external usage to public time server pools,
> > such as ch.pool.ntp.org. Misuse, caused mostly by IT-providers
> > synchronizing their client infrastructures, has made unusually high
> > demands on network traffic, thereby causing ETH to take effective
> > measures. As of Fall 2012, the availability of swisstime.ethz.ch has
> > been changed to Closed Access. Since beginning of July 2013, access
> to
> > the server is blocked entirely for the ntp protocol.
> 
> >    5.  If a firmware default server IP address is provided, it MUST
> be
> > a
> >
[[Anil >>]]  I am not for a new NTP protocol unless it is giving an edge over current.
This mail chain is to discuss the same. These are general issues with NTP 
I thought if any of these issues could be addressed by new version then we can go for it. 
> Lange Wand 9, 31812 Bad Pyrmont, Germany Amtsgericht Hannover 17HRA
> 100322 Geschäftsführer/Managing Directors: Günter Meinberg, Werner
> Meinberg, Andre Hartmann, Heiko Gerstung