Subject: Re: [Ntp] WGLC on draft-ietf-alternative-port-01

From: Dieter Sibold <[email-masked]>
Date: Fri, 23 Jul 2021 19:26:13 +0200

I agree with Danny.
- Dieter

On 24 Jun 2021, at 15:52, Danny Mayer wrote:

> On 6/24/21 1:08 AM, Watson Ladd wrote:
>> On Wed, Jun 23, 2021 at 4:30 AM Miroslav Lichvar <[email-masked]> wrote:
>>> On Fri, Jun 11, 2021 at 01:36:03PM +0000, Karen O'Donoghue wrote:
>>>> NTP Working Group,
>>>>
>>>> This email starts a two week working group last call (WGLC) on
>>>> https://datatracker.ietf.org/doc/draft-ietf-ntp-alternative-port/
>>> One thing that I'd like to specifically ask everyone to consider
>>> is the intended future of the alternative port. Do we expect NTP to
>>> fully move there at some point and keep the port 123 only for legacy
>>> implementations? Or should it always be just an alternative in case
>>> the port 123 is not working?
>> I do not think the situation with port 123 is salvageable. There is
>> too much blocking and other manipulation. I think this doc as is is
>> the only way forward.
>
> Using an alternative port will not fly. You need to remember that there are millions upon millions of devices out there that use port 123 for NTP. Don't expect them to change just because you wrote a document to say to use a different port. If you want to use a different port you may as well design a different protocol. The port number is baked in and all firewalls would need to be changed to accommodate this.
>
> I will need to read the document again and decide on my vote. I don't think that the consequences have been thought through.
>
> Danny
>
>
> _______________________________________________
> ntp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ntp

---

From: Dieter Sibold <[email-masked]>
Date: Fri, 23 Jul 2021 19:29:34 +0200

Hi Erik,

Personally, I don’t think that port 123 is completely wasted. I 
support your recommendation to request an early review of the transport 
area.
- Dieter

On 26 Jun 2021, at 5:54, Erik Kline wrote:

> On Thu, Jun 24, 2021 at 6:53 AM Danny Mayer <[email-masked]> 
> wrote:
>
>>
>>> On Wed, Jun 23, 2021 at 4:30 AM Miroslav Lichvar 
>>> <[email-masked]>
>> wrote:
>> Using an alternative port will not fly. You need to remember that 
>> there
>> are millions upon millions of devices out there that use port 123 for
>> NTP. Don't expect them to change just because you wrote a document to
>> say to use a different port. If you want to use a different port you 
>> may
>> as well design a different protocol. The port number is baked in and 
>> all
>> firewalls would need to be changed to accommodate this.
>>
>> I will need to read the document again and decide on my vote. I don't
>> think that the consequences have been thought through.
>>
>
> May I recommend that we request an early Transport Area Review for 
> this
> document?  I'm happy to click the button myself.  They might have some
> thoughts/advice on these issues.

---

From: Danny Mayer <[email-masked]>
Date: Sun, 25 Jul 2021 19:46:28 -0400

I have now come to the conclusion that this should NOT be accepted. 
Based on a conversation I had recently something like 70% of all traffic 
is still NTP V3 so this would not have any effect on them. Millions of 
firewalls would need to be changed. While the idea is generally good, 
it's not practical.

An easier and more practical proposal would be to remove mode 6 and 7 
packets from the existing protocol and require that those types of 
packets and information be done on a separate port or even use TCP.

Danny

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 26 Jul 2021 11:05:29 +0200

On Sun, Jul 25, 2021 at 07:46:28PM -0400, Danny Mayer wrote:
> I have now come to the conclusion that this should NOT be accepted. Based on
> a conversation I had recently something like 70% of all traffic is still NTP
> V3 so this would not have any effect on them. Millions of firewalls would
> need to be changed. While the idea is generally good, it's not practical.

The draft is not specific to NTPv4. NTPv3 clients can be updated to
use the alternative port too. On the public servers I'm running, with
one exception (India), the observed NTPv3 share is below 10% anyway.

> packets from the existing protocol and require that those types of packets
> and information be done on a separate port or even use TCP.

I don't see how would that be better. If you write a new document that
forbids mode 6/7 on port 123, how will that fix the existing devices
that still respond to it?

It's now over 7 years since the large-scale DDoS attacks started. If
everyone fixed configuration of their devices to not respond to the
modes, ISPs wouldn't be using the NTP rate-limiting middleboxes and we
wouldn't have this discussion.

Port 123 seems to be doomed, at least for the near future. The
alternative port gives us a way forward. Yes, the adoption on the
global scale will probably take a long time, but at least people who
are most impacted will be able to do something to fix it (update their
NTP servers and clients).

-- 
Miroslav Lichvar

---

From: "Ulrich Windl" <[email-masked]>
Date: Mon, 26 Jul 2021 11:36:21 +0200

>>> Miroslav Lichvar <[email-masked]> schrieb am 26.07.2021 um 11:05 in
Nachricht <YP562akF+CL/9R5s@localhost>:
> On Sun, Jul 25, 2021 at 07:46:28PM ‑0400, Danny Mayer wrote:

Actually I think removing mode 6 from the protocol is a bad idea:
So (assuming there is a need for monitoring and dynamic configuration) every
implementer will do his/her own protocol.
As a matter of fact crony does not document the internal protocol being used.
Despite of not being compatible with the popular NTP implementation, that would
require to call a process instead of sending out a few network packets.
Obviously when monitoring a bigger landscape this is highly inefficient.
The implementation of NTP in Microsoft Windows is a similar case.

Regards,
Ulrich Windl

> It's now over 7 years since the large‑scale DDoS attacks started. If
> modes, ISPs wouldn't be using the NTP rate‑limiting middleboxes and we
> ‑‑

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 26 Jul 2021 12:05:50 +0200

> implementer will do his/her own protocol.

Why would monitoring and/or configuration have to run on port 123?
That is something completely different than exchanging timestamps for
synchronization of clocks.

There can be a standard protocol for monitoring and configuring, maybe
specific to NTP, but not specific to an NTP implementation (unlike
mode 6).

> As a matter of fact crony does not document the internal protocol being used.

Right, it's not documented. Similarly to mode 6, it is tightly tied to
an implementation. It could be replaced with a standard protocol, if
there was one that covered all the state and configuration.

-- 
Miroslav Lichvar

---

From: Watson Ladd <[email-masked]>
Date: Wed, 28 Jul 2021 20:12:25 -0700

> NTP servers and clients).

We see issues at Cloudflare with packet delivery on port 123. ISP
middleboxes are going to police by length, and an alternative port is
the way forward. There is much less policing on the alternative ports.

Sincerely,
Watson Ladd

--
Astra mortemque praestare gradatim

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 29 Jul 2021 09:08:10 +0200

>>> Watson Ladd <[email-masked]> schrieb am 29.07.2021 um 05:12 in
Nachricht
<[email-masked]>:
>> > I have now come to the conclusion that this should NOT be accepted. Based

> on
>> > a conversation I had recently something like 70% of all traffic is still

> NTP
packets
> the way forward. There is much less policing on the alternative ports.

Actually I'd think teching cloudflare would be better than changing the port.
Otherwise: When do we change the port again?

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 29 Jul 2021 09:44:57 +0200

>>> "Ulrich Windl" <[email-masked]> schrieb am 29.07.2021
um
>>> > I have now come to the conclusion that this should NOT be accepted.
Based
> Actually I'd think teching cloudflare would be better than changing the 

I had meant to write "teaching"...

> port.

---

From: Danny Mayer <[email-masked]>
Date: Thu, 29 Jul 2021 10:47:49 -0400

On 7/29/21 3:08 AM, Ulrich Windl wrote:
> Actually I'd think teaching cloudflare would be better than changing the port.

When NTPv5 comes along it will have a larger packet size, so how is this 
different? In addition there's the problem of extension fields which 
also increase the size of the packet including the NTS-enabled ones. 
It's much more important to figure out how these middleboxes and 
firewalls should handle NTP packets and provide better guidance.

Danny

---

From: Watson Ladd <[email-masked]>
Date: Thu, 29 Jul 2021 08:01:45 -0700

On Thu, Jul 29, 2021, 12:45 AM Ulrich Windl
<[email-masked]> wrote:
<snip>
> I had meant to write "teaching"...

It's not our devices: it's middleboxes in ISP networks. We only find
out from customer pcaps where one in four NTS packets makes it
through. The policing is by length because that is what these boxes
support.

> > Otherwise: When do we change the port again?

Don't screw up and put UDP amplifiers out, and the port will be fine.

---

From: Danny Mayer <[email-masked]>
Date: Thu, 29 Jul 2021 15:12:45 -0400

> through. The policing is by length because that is what these boxes
So how would using an alternative port make a difference?

Danny

---

From: Hal Murray <[email-masked]>
Date: Fri, 30 Jul 2021 22:18:05 -0700

> So how would using an alternative port make a difference?

Because the filters are looking for UDP port 123.  The smart ones do a length 
check and let 48 byte packets througg.  The dumb ones just nuke them all.

The recent discussion has focused on middleware.  It also happens on transit 
links.

I have a pool server in London.  The monitoring station in the US frequently 
kicks it out of the pool because it can't verify that it is responding.

We had significant troubles during the early NTS hackathons.  NTP packets with 
NTS extensions didn't get through.

-- 
These are my opinions.  I hate spam.

---

From: Watson Ladd <[email-masked]>
Date: Sat, 31 Jul 2021 09:59:59 -0700

> So how would using an alternative port make a difference?

The other port is not policed now. We use it, and take care to not
massively deploy amplifiers on it, so it does not get policed.  This
issue is not about opening firewalls that are under people's control.
This is about firewalls on ISP networks to protect the ability of the
network to function, that break and do things like take NIST offline
in the Western US for a month.

Sincerely,
Watson

---

From: Danny Mayer <[email-masked]>
Date: Sat, 31 Jul 2021 18:33:13 -0400

> in the Western US for a month.

If you are not policing privileged ports then you have a problem with 
your firewall.

Danny