Subject: Re: [Ntp] [Tsv-art] Tsvart early review of draft-ietf-ntp-alternative-port-02

From: Hal Murray <[email-masked]>
Date: Sat, 04 Dec 2021 15:12:06 -0800

[email-masked] said:
> FWIW, I don't see this assignment as appropriate. 

Without a new port, it will be close to impossible to widely deploy NTP 
security.

Years ago (2013), NTP was used in a giant DDoS attack.  That was due to a 
bug/oversight that had been around since the early NTP work.  (I've tracked it 
back to 1989.)
  https://www.spamhaus.org/news/article/695/answers-about-recent-ddos
  https://en.wikipedia.org/wiki/Denial-of-service_attack#Amplification
The Wikipedia chart could use another column -- the number of sites available. 
 Almost all Linux or *BSD sites were running ntpd.

The fix was trivial, but there are many essentially unattended sites running 
old versions of ntpd that will never get fixed.

Many many many sites have quietly installed filters in their routers.  Set and 
forget.

A typical filter drops UDP traffic to port 123 with a length other than 48.  
That lets old unauthenticated NTP through but authenticated packets are longer 
and get dropped.

To use authentication on the existing port would require removing those 
filters.  Even if you could track down the right people, they would probably 
drag their feet until most of the sites running old unattended ntpd were fixed.

An alternative would be to implement BCP 38.  How long has that been in 
progress?

-- 
These are my opinions.  I hate spam.

---

From: "[email-masked]" <[email-masked]>
Date: Sat, 4 Dec 2021 18:08:13 -0800

HI, Hal,

Again, with ports team review hat on:

Everything that happened below is equivalent to finding a zero-day or other kind of protocol error in any service.

The fact that the current ’solution’ is to block that port isn’t new either.

We don’t have resources to continually assign ‘fresh’ ports for services and I see no reason this one warrants an exception.

There are other solutions here; if your own recommendations caused packets smaller than 48 to be dropped, then find a way to fragment and reassemble authenticated packets until you find out that larger ones get through.

This is a protocol design problem, not a port assignment problem.

Joe

—
Joe Touch, temporal epistemologist
www.strayalpha.com

> On Dec 4, 2021, at 3:12 PM, Hal Murray <[email-masked]> wrote:
> 
>

---

From: Steven Sommars <[email-masked]>
Date: Sat, 4 Dec 2021 23:00:10 -0600

NTP got a bad reputation as an amplification attack vector circa 2014.  The
offending NTP servers have largely been fixed; today few public servers
will blindly amplify the Mode 7/monlist command (the major problem
source).  The organizations doing the filtering aren't receptive to
suggestions that they alter/remove their filtering schemes.

Those finding the alternate port distasteful could help by reaching out to
the organizations doing the filtering and convincing them that changes are
needed.

I don't like the alternative port idea either, but see no other simple
solution.  Short of designing yet another time transfer protocol what would
you suggest?

On Sat, Dec 4, 2021 at 8:08 PM [email-masked] <[email-masked]>
wrote:

> Everything that happened below is equivalent to finding a zero-day or
> other kind of protocol error in any service.
>
> We don’t have resources to continually assign ‘fresh’ ports for services
> and I see no reason this one warrants an exception.
>
> There are other solutions here; if your own recommendations caused packets
> smaller than 48 to be dropped, then find a way to fragment and reassemble
> authenticated packets until you find out that larger ones get through.
>
> _______________________________________________
> ntp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ntp
>

---

From: "[email-masked]" <[email-masked]>
Date: Sat, 4 Dec 2021 22:10:10 -0800

It’s not the rest of the community’s job to make sure your port is useful - it is yours (the NTP community’s). 

I.e., you gave one viable way forward: fix the filtering.
I gave another - find a way around/through the filtering.

Redundant port assignments are not a “simple” solution - they are an expensive one that you should not have not, IMO, earned access to, if for no other reason than “poor stewardship of what you already have”.

Joe

—

> On Dec 4, 2021, at 9:00 PM, Steven Sommars <[email-masked]> wrote:
> 
> NTP got a bad reputation as an amplification attack vector circa 2014.  The offending NTP servers have largely been fixed; today few public servers will blindly amplify the Mode 7/monlist command (the major problem source).  The organizations doing the filtering aren't receptive to suggestions that they alter/remove their filtering schemes.  
> 
> Those finding the alternate port distasteful could help by reaching out to the organizations doing the filtering and convincing them that changes are needed.
> 
> I don't like the alternative port idea either, but see no other simple solution.  Short of designing yet another time transfer protocol what would you suggest?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Sat, Dec 4, 2021 at 8:08 PM [email-masked] <mailto:[email-masked]> <[email-masked] <mailto:[email-masked]>> wrote:
> HI, Hal,
> 
> www.strayalpha.com <http://www.strayalpha.com/>
> 
>> On Dec 4, 2021, at 3:12 PM, Hal Murray <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> 
>> [email-masked] <mailto:[email-masked]> said:
>>  https://www.spamhaus.org/news/article/695/answers-about-recent-ddos <https://www.spamhaus.org/news/article/695/answers-about-recent-ddos>
>>  https://en.wikipedia.org/wiki/Denial-of-service_attack#Amplification <https://en.wikipedia.org/wiki/Denial-of-service_attack#Amplification>
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/ntp <https://www.ietf.org/mailman/listinfo/ntp>
> Tsv-art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tsv-art

---

From: Danny Mayer <[email-masked]>
Date: Sun, 5 Dec 2021 12:11:22 -0500

> Without a new port, it will be close to impossible to widely deploy NTP
Why would that be? NTP has a port that is widely deployed.
> old versions of ntpd that will never get fixed.

Yes so how is the alternative port going to magically fix this?

> That lets old unauthenticated NTP through but authenticated packets are longer
The same happened with EDNS0. Firewall administrators had to go in and 
adjust their filtering. The same thing needs to  be done for NTP. It's 
EXACTLY what would be needed for an alternate port. You're just adding 
complexity for no real benefit.
> drag their feet until most of the sites running old unattended ntpd were fixed.

They don't need to remove filters as much they need to change them. Are 
you saying that the alternate port would not be filtered?

Dragging feet is the norm. The same happened with EDNS0. This is just 
moving the problem without solving it.

> An alternative would be to implement BCP 38.  How long has that been in
We cannot help people who don't implement best practices. We can only 
point them to the documentation.

Danny

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 5 Dec 2021 15:20:12 -0800

One the one hand I see:

> Years ago (2013), NTP was used in a giant DDoS attack. That was due to a
bug/oversight
> that had been around since the early NTP work.  (I've tracked it  back to
1989.)
> old versions of ntpd that will never get fixed.

And  on the other hand I see:

On Sat, Dec 4, 2021 at 9:00 PM Steven Sommars wrote:
> NTP got a bad reputation as an amplification attack vector circa 2014.
> The offending NTP servers have largely been fixed; today few public
> servers will blindly amplify the Mode 7/monlist command (the major
> problem source). The organizations doing the filtering aren't receptive
> to suggestions that they alter/remove their filtering schemes.

We have two contradictory assertions by proponents of the alternate port.
Which, if either, is correct?

Are the servers largely fixed but the filters still in place out of
inertia, as Mr. Sommers says?

Or do the filters remain because of servers that have not been fixed, as
Mr. Murray says?

And I have another question, which I though had been asked, but which I now
cannot find:

What assurance is there that the alternate port won't be blocked by default?

Mike Heard

---

From: "Ulrich Windl" <[email-masked]>
Date: Mon, 06 Dec 2021 08:59:34 +0100

>>> Hal Murray <[email-masked]> schrieb am 05.12.2021 um 00:12 in Nachricht
<[email-masked]>:

...
> Many many many sites have quietly installed filters in their routers.  Set 
...

Can any RFC do something against that "set and forget" mentality? I guess not.

> That lets old unauthenticated NTP through but authenticated packets are 

Still no RFC can fix this. Anybody having read the NTPv3/NTPv4 RFCs should know better.

...

Regards,
Ulrich

---

From: Eliot Lear <[email-masked]>
Date: Mon, 6 Dec 2021 11:15:55 +0100

I have a question:

Had neither NTP nor NTS existed, and you were requesting a new port for 
a secure time protocol, would you have designed it identically?

Eliot

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 6 Dec 2021 11:00:12 +0100

> Or do the filters remain because of servers that have not been fixed, as

I suspect we won't know for sure as long as NTP is firewalled
everywhere.

The reality is that NTP doesn't work reliably over Internet. In the
community of people running public servers (pool.ntp.org) it's the
number one problem. My impression is that it's getting worse over
time.

With NTS, which uses longer NTP messages it's quite common to see a
client that doesn't get a response to most of its requests.

The major ISPs have middleboxes, which cannot be configured to
specifically block or rate limit the amplifiying mode-6/7 messages.
Better hardware could be designed, but who will pay for that?
Expecting all those existing servers that amplify the traffic to be
fixed and the middleboxes to be disabled doesn't seem to be realistic.

-- 
Miroslav Lichvar

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 6 Dec 2021 14:53:33 +0000

Almost definitely not.  NTS is wrapped around into the quirks of NTP.

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 6 Dec 2021 16:25:36 +0100

> Almost definitely not.  NTS is wrapped around into the quirks of NTP.

Which quirks of NTP? That it is stateless, or that it is UDP?

I doubt any of that would change if a new protocol was designed from
scratch. At least, I'd expect someone to propose the problem to be
fixed in NTPv5.

-- 
Miroslav Lichvar

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 6 Dec 2021 16:11:33 +0000

>    Which quirks of NTP? That it is stateless, or that it is UDP?

For example: Packet sizes being very fixed such that it is feasible to put firewall limits in place. That there is no facility for large packets.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 6 Dec 2021 17:38:56 +0100

Would you merge your authentication method into a single service or 
would you keep it separate?

Eliot

---

From: Hal Murray <[email-masked]>
Date: Mon, 06 Dec 2021 11:07:36 -0800

> would you keep it separate?

I can't quite figure out what you are asking.

NTS is a two step process.  The first step is a key exchange using TLS.
That is complicated by including cookies so the server doesn't need any
per-client storage.

NTP has 2 modes for authentication.  One uses NTS and requires the above
setup dance before exchanging authenticated NTP packets.

The other uses shared keys where the keys are setup using some unspecified
out of band mechanism.  (NIST uses USPS.)  This is impractical for high
volume servers since is requires manual intervention by the server operator
for each client.

Do you mean would a fresh design use separate ports for authenticated
and non-authenticated NTP traffic?  I can't see any reason for that.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 6 Dec 2021 21:57:53 +0100

Hal,

I understand the two step process.  What I'm getting at is that if you 
need a new port, then you might as well take a moment to think about 
whether you want a new protocol to go with it.  View it as an opportunity.

Eliot

---

From: Danny Mayer <[email-masked]>
Date: Mon, 6 Dec 2021 17:25:52 -0500

> number one problem. My impression is that it's getting worse over
That's untrue. It's been working for over 25 years and probably longer. 
The real issue here is that there are firewalls and routers blocking 
packets longer than 48 bytes, yet I've watch autokey do its dance with 
packet lengths much longer. What you are trying to do is circumvent 
these firewall and router rules to get NTP to work with larger packets 
but on a new port and you will still have to set the rules to allow it. 
Like EDNS0 adjusting the existing rules on port 123 is a much better 
solution. Removing or restricting support for mode 6 and 7 packets is 
trivial and requires no additional complexity. You're just trying to 
move the underlying problem from one place to the other.
> client that doesn't get a response to most of its requests.
Fix the firewall rules.
> fixed and the middleboxes to be disabled doesn't seem to be realistic.
No, instead you should be fixing the servers to not reply to mode 6/7 
packets instead. In the NTP Reference implementation it's trivial to 
update the configuration. In other implementations that's a SMOP if it's 
not already there.

Danny

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 6 Dec 2021 21:28:12 -0800

FWIW, NTP has ample capacity to support new versions on the existing port using its version numbers. A new port is not needed for security as TLS use is already supported.

The issue here is firewall configuration, I,e., it is an operational problem, not a protocol design issue.

Joe

—

> On Dec 6, 2021, at 12:57 PM, Eliot Lear <[email-masked]> wrote:
> 
> Hal,
> 
> I understand the two step process.  What I'm getting at is that if you need a new port, then you might as well take a moment to think about whether you want a new protocol to go with it.  View it as an opportunity.
> 
> Eliot
> 
> <OpenPGP_0x87B66B46D9D27A33.asc>

---

From: "Ulrich Windl" <[email-masked]>
Date: Tue, 07 Dec 2021 08:32:03 +0100

>>> Eliot Lear <[email-masked]> schrieb am 06.12.2021 um 17:38 in Nachricht
>> scratch. At least, I'd expect someone to propose the problem to be

Hi!

Actually I think ist the fatal triple:
(dump protocol (NTP), dumb firewall, dumb administrator)

Remove one of the three and things become better. ;-)

Regards,
Ulrich

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 7 Dec 2021 09:29:38 +0100

On Mon, Dec 06, 2021 at 05:25:52PM -0500, Danny Mayer wrote:
> That's untrue. It's been working for over 25 years and probably longer. The
> real issue here is that there are firewalls and routers blocking packets
> longer than 48 bytes, yet I've watch autokey do its dance with packet
> lengths much longer. What you are trying to do is circumvent these firewall
> and router rules to get NTP to work with larger packets but on a new port
> and you will still have to set the rules to allow it.

The problematic middleboxes don't block other ports. They specifically
block or rate limit packets on the port 123 as a mitigation for the
amplification attacks. If a non-amplifying subset of NTP moves to
another port, they will have no reason to block it.

Autokey allows amplification, so it must not be used on the
alternative port. 

BTW, the length-specific blocking starts around 200 bytes, not 48.
Autokey wouldn't be as impacted as NTS. See this great post from
Steven Sommars (who spent a lot of time debugging these issues):

https://weberblog.net/ntp-filtering-delay-blockage-in-the-internet/

> Like EDNS0 adjusting
> the existing rules on port 123 is a much better solution. Removing or
> restricting support for mode 6 and 7 packets is trivial and requires no
> additional complexity.

If it is so easy, why it didn't happen in those 8 years since the
attacks become widespread?

> > client that doesn't get a response to most of its requests.

Some of them are in hardware. They cannot be fixed. They would need to
be replaced with something that blocks all mode 6 and 7 packets, and
packets in modes 1, 2, 3, 4 that contain an Autokey extension field.
Even in a software firewall that would be difficult to do.

> packets instead.

How do you find them, how do you convince the owners to fix/replace
them, and make sure it doesn't happen again?

> In the NTP Reference implementation it's trivial to update
> the configuration.

Maybe it is, but how likely do you think it is that everyone will do
that?

> In other implementations that's a SMOP if it's not
> already there.

AFAIK, no other widely used implementation supports mode 6 or 7, or
Autokey. It's just one implementation that still amplifies traffic in
its default configuration.

-- 
Miroslav Lichvar

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 7 Dec 2021 10:23:10 +0100

I fully agree with Danny.

I find it ridiculous to start using a new port for NTP because some 
admins block the original, well-known port because many years ago there 
was a possibility for DDoS for servers that weren't properly configured.

What if that would be done for any service for which there was a 
security problem in the past? Would a new port e.g. for DNS be proposed 
just because some DNS servers that weren't properly configured could be 
misused?

I wonder what would happen in a big company if time synchronization 
would stop working just because an admin thinks he has to block the NTP 
port or limit the NTP packet size.

I bet that if many users started complaining, the boss of the admin 
would step in. And if that not happens, it's their own problem due to 
silly behavior.

Martin

-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

---

From: Miroslav Lichvar <[email-masked]>
Date: Tue, 7 Dec 2021 11:21:13 +0100

On Tue, Dec 07, 2021 at 10:23:10AM +0100, Martin Burnicki wrote:
> I find it ridiculous to start using a new port for NTP because some admins
> block the original, well-known port because many years ago there was a
> possibility for DDoS for servers that weren't properly configured.

That possibility still exists. It's a security issue in the protocol.
The modes 6 and 7, and Autokey cannot be fixed without breaking
compatibility. They cannot be exposed to Internet. On my public
servers I still see scans looking for the amplification commands. If
everyone suddenly stopped blocking NTP, I think we would see an
explosion in the amplification attacks.

It is not just some admin in a random company deciding to block NTP.
It's major ISPs with millions of customers and huge transit networks
in the Internet. We cannot expect them to be asking individual owners
of misconfigured servers to fix their stuff and keep watching them
indefinitely.

I assume your products don't support NTS yet? I think you would have
have a different view on the subject with customers complaining it
doesn't work reliably.

> What if that would be done for any service for which there was a security
> problem in the past? Would a new port e.g. for DNS be proposed just because
> some DNS servers that weren't properly configured could be misused?

Well, DNS has this issue too. It allows traffic amplification and it's
a big issue that requires a lot of resources to keep it running.

But unlike DNS, NTP cannot fall back to TCP. Fortunately, NTP as a
time synchronization protocol exchanging constant amount of data does
not need any amplification. It can be easily restricted to a
non-amplifying subset.

-- 
Miroslav Lichvar

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 7 Dec 2021 08:31:55 -0800

—

> On Dec 7, 2021, at 2:21 AM, Miroslav Lichvar <[email-masked]> wrote:
> 

Again, IMO, that’s why protocols (including NTP) have version numbers. 

I look forward to a new NTP version that addresses these issues, but it should run on the existing port.

This is the *same* issue every protocol encounters for any vulnerability.

Joe

---

From: Eliot Lear <[email-masked]>
Date: Tue, 7 Dec 2021 20:43:05 +0100

We have the following situation, as I understand it:

There remain servers out there that are open to control packets, and 
those packets are amplified at target NTP servers. Deployments have 
mitigated the attack by placing a firewall rule that matches the frame 
size of a normal NTP query, rather than enumerating the NTP servers that 
they permit.  Do I have the situation correct?

So we have a combination of a protocol weakness, open NTP servers, and a 
common firewall rule set that won't let the right thing happen.  Does 
anyone know how common?

Eliot

On 07.12.21 17:31, [email-masked] wrote:
> www.strayalpha.com <http://www.strayalpha.com>
>
>>> admins

---

From: Hal Murray <[email-masked]>
Date: Tue, 07 Dec 2021 22:51:28 -0800

> Why would that be? NTP has a port that is widely deployed.

Yes, and widely filtered.

> The same happened with EDNS0. Firewall administrators had to
> go in and adjust their filtering.

You said "firewall".  I've been using "filter".
I think of a firewall as something the local administrator controls.

The filters I'm interested in are undocumented.  Both location and
details of what they do are unknown.  They generally are located
"out there" someplace on a system that is not run by the ISP at
either end.

> you saying that the alternate port would not be filtered?

The filters I'm talking about are for UDP port 123 and usually
something about length and/or mode.

If NTP gets a new port it will start out unfiltered.  As long as we
don't screwup, there will be no reason to add filters for the new port.

---

From: Eliot Lear <[email-masked]>
Date: Wed, 8 Dec 2021 08:14:57 +0100

Just to follow this up, my point is that the version bit doesn't matter 
if the packets aren't getting to endpoints.  At that point we need to 
start thinking about what is best for the Internet, and there may well 
be a tradeoff here.  But it really depends on what the operational data 
looks like, in so much as we can get at it.

Eliot

On 07.12.21 20:43, Eliot Lear wrote:
> size of a normal NTP query, rather than enumerating the NTP servers 
> that they permit.  Do I have the situation correct?
>
> So we have a combination of a protocol weakness, open NTP servers, and 
> a common firewall rule set that won't let the right thing happen.  
> Does anyone know how common?
>
> Eliot
>
>
>

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 08 Dec 2021 08:37:30 +0100

>>> Miroslav Lichvar <[email-masked]> schrieb am 07.12.2021 um 09:29 in
Nachricht <Ya8bcmEO04g1TCzB@localhost>:
...
> amplification attacks. If a non‑amplifying subset of NTP moves to

But when noticing that NTP uses a different port, they could also notice how
to fix their servers (regarding mode 6 and mode 7). Autokey shouldn't be a
common problem I guess.

...

Regards,
Ulrich

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 8 Dec 2021 15:24:43 +0100

On Wed, Dec 08, 2021 at 08:14:57AM +0100, Eliot Lear wrote:
> Just to follow this up, my point is that the version bit doesn't matter if
> the packets aren't getting to endpoints.  At that point we need to start
> thinking about what is best for the Internet, and there may well be a
> tradeoff here.  But it really depends on what the operational data looks
> like, in so much as we can get at it.

Cloudflare is probably the largest provider of NTP+NTS. It would be
interesting if they could show how is distributed the length of
NTP+NTS requests received on their servers. IIRC they mentioned this
in one of the NTP WG meetings. As NTS clients are encouraged to not
reuse NTS cookies, they should ask for more cookies if there is a
packet loss, and this should be clearly visible in the distribution of
the request length.

I have some data that show that the filtering exists by measuring
losses at different NTP message lengths. I tested about 2600 public
servers from the pool.ntp.org project spread around the world. Only
servers that respond to packets containing an unknown extension field
were tested. This is a one-way test as the response is always short.
This was done from machines in 3 different countries on different
continents. 10 requests were sent to each server for each message
length. The following table shows from how many of the servers at
least 5 responses were received.

 Len   US     CZ     SG   
  48  98.3%  97.7%  98.6%
 148  98.2%  96.7%  98.2%
 248  90.1%  88.3%  85.0%
 348  90.5%  88.1%  84.7%
 448  89.7%  86.9%  84.1%
 548  96.2%  92.8%  95.5%

The dip between 200 and 500 matches the measurements posted by Steven
Sommars that I linked to earlier. A typical NTP+NTS message length
starts at about 230 bytes. So, depending on your location you could
expect >50% of NTP+NTS requests to about 10% of the public servers to
be lost due their length. After three packets lost in a row, the
request length increases to about 550, where the chances of getting a
response is better again. A response to every fourth request. That
is a common pattern I see on some clients.

-- 
Miroslav Lichvar

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 8 Dec 2021 09:23:37 -0800

Hi, Eliot,

Packets ARE getting to endpoints. 

If the issue is that packets of certain *sizes* aren’t getting there, then NTP should find a way around that using the current port. The fact that the existing port is being filtered is on them. Moving to another port simply invites firewall managers to add that number to their current NTP filters.

Joe

—

> On Dec 7, 2021, at 11:14 PM, Eliot Lear <[email-masked]> wrote:
> 
> Just to follow this up, my point is that the version bit doesn't matter if the packets aren't getting to endpoints.  At that point we need to start thinking about what is best for the Internet, and there may well be a tradeoff here.  But it really depends on what the operational data looks like, in so much as we can get at it.
> 
> Eliot

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 09 Dec 2021 08:24:44 +0100

>>> Eliot Lear <[email-masked]> schrieb am 07.12.2021 um 20:43 in Nachricht
> common firewall rule set that won't let the right thing happen.  Does 

I wonder why this hasn't been discussed (it's obvious that nothing of the
proposals will change any existing NTP deployment automagically):
What if the NTP control protocol were changed to require authentication even
for read operations? (a bit like SNMP's read/write communities)
Then non-authenticated requests simply wouldn't be answered, or be answered by
a fixed-size error packet.

Regards,
Ulrich

---

From: Eliot Lear <[email-masked]>
Date: Thu, 9 Dec 2021 08:51:41 +0100

Hi Joe

On 08.12.21 18:23, [email-masked] wrote:
> If the issue is that packets of certain *sizes* aren’t getting there, 
> then NTP should find a way around that using the current port. The 
> fact that the existing port is being filtered is on them. Moving to 
> another port simply invites firewall managers to add that number to 
> their current NTP filters.

I'm not sold either way.

Yes, as a rule we do not allocate new ports for existing services.  
There are two reasons for that rule: first, it encourages sloppy design 
of the form ("I'll fix it later") and second and more importantly, we 
simply don't have so many ports to go around.

But this is a fundamental service, and the default behavior for clients 
will simply be to not authenticate.  And that would make NTP an 
attractive nuisance if those port filters are likely to stay.  That's a 
big if.

It is simply not reasonable, however, to require that NTP packets 
comport to a particular size in order to evade a combination of broken 
firewall policies and unbending IANA policies

Miroslav, one question I would like to understand is whether those drops 
involve typical packets used to sync clocks or if those are control 
packets.  In other words, are clocks not syncing because of the drops?

Eliot

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 9 Dec 2021 09:26:55 +0100

On Thu, Dec 09, 2021 at 08:51:41AM +0100, Eliot Lear wrote:
> Yes, as a rule we do not allocate new ports for existing services.  There
> are two reasons for that rule: first, it encourages sloppy design of the
> form ("I'll fix it later") and second and more importantly, we simply don't
> have so many ports to go around.

That's fair enough. I wonder if exceptions could be allowed for old
services. It seems the problematic NTP modes were introduced in
RFC1119 from 1989. Could a protocol be pardoned for its mistakes after
30 years?

> It is simply not reasonable, however, to require that NTP packets comport to
> a particular size in order to evade a combination of broken firewall
> policies and unbending IANA policies

I think we could make the most common case of an NTPv4+NTS packet
shorter to get below that length where the observed filtering starts
and add padding to packets that don't fit there. That would make NTS
more reliable (which is what the WG is now most interested in), but it
would not change anything in those cases where NTP is completely
blocked or rate limited.

> packets.  In other words, are clocks not syncing because of the drops?

The intention is to drop the control packets (NTP modes 6 and 7),
which amplify traffic, but it impacts also the time-synchronization
modes (1, 2, 3, 4), which don't amplify traffic (except when Autokey
is enabled, but that is rarely used).

-- 
Miroslav Lichvar

---

From: Hal Murray <[email-masked]>
Date: Thu, 09 Dec 2021 00:46:31 -0800

> packets.  In other words, are clocks not syncing because of the drops?

It depends.  There is no standardization on the filters.

My initial description of a simple length check was overly simplified.

Sometimes, basic non-authenticated NTP requests or responses get dropped.
The NTP pool has a monitoring setup.  It polls each server to verify
that it is responding with reasonable values of time.  Sometimes healthy
servers get dropped because the monitoring packets don't get through.
This is not common, but also not rare.

Sometimes, the filters are dropping mode 6 and 7.  If that was the only
type of filter we wouldn't be having this discussion.

Sometimes, the dropping is length dependent in strange ways.  I had one case
where every 4th NTS packet would get through.  When an NTS exchange fails, the
next request is roughly 100 bytes longer.  That's asking for an extra cookie to
replace the one that would have been in the lost packet.  The request for an
extra cookie is the full length of a cookie to avoid amplification.  If 2
packets in a row are dropped, the 3rd request will have 2 extra cookie requests.
The 4th one will have 3.  That worked. :)  Then it started over again with
short packets which didn't work...

---

From: Watson Ladd <[email-masked]>
Date: Thu, 9 Dec 2021 09:46:10 -0500

> to fix their servers (regarding mode 6 and mode 7). Autokey shouldn't be a

The people who run the middleboxes do not run the servers. They do not
care about the collateral damage. All they care about is keeping up
their network without needing to invest in it.

We are not going to get this changed. We are not going to be able to
hunt down every broken "reference implementation" server out there.

--
Astra mortemque praestare gradatim