Subject: [LOOPS] Packet sequence numbers

From: Carsten Bormann <[email-masked]>
Date: Wed, 16 Oct 2019 22:07:08 +0200

In the process of preparing the next version of the gen-info draft, there are a few items where it probably would be good to get wider input.

One of these is the use of packet sequence numbers.

Gen-info says that the packet sequence number (PSN) counts the LOOPS packets transmitted by the LOOPS ingress, i.e., that retransmissions receive a new PSN.

This removes ACK ambiguity: the situation where original transmission and retransmissions carry the same sequence number and an ACK therefore cannot be unambiguously assigned to one of the transmission never occurs.

Disadvantages may include:

* Without additional information, the original sequence of the packets is no longer visible at the egress.  This means that not enough information is available to perform resequencing at the egress.  (Resequencing is not a feature of LOOPS, but could be easily added by an implementation if overall latency is not an issue and enough information is available at the egress.)

* FEC will not be able to derive a lot of information from the packet sequence number, so will need some additional information, too.

* Monitoring tools that are looking at the PSN will simply see the holes created by the losses on the LOOPS segment and not the successful repair actions.  (But the same is true for FEC, so maybe these monitoring tools are too simple.)

Advantages may include:

* No ACK ambiguity.

* The need for back-filling ACK bitmaps is limited to the time scale of reordering (if any at all).

* Somewhat simplified implementation: NACKs for sequence numbers that already have been retransmitted don’t need a hold-off — the cached packets are simply swept to their new numbers at the ingress and repetitions of old NACKs can be ignored.  On the other hand, some retransmission counter will need to be on each packet to avoid 1-persistence.

One question that comes up about implementation is how likely a “single retransmission” optimization will be: An implementation that only ever retransmits a packet once can simply delete the cache entry after retransmission instead of sweeping it to the new number.  Single retransmission actually fits many usage scenarios well, as it essentially squares the non-delivery reliability for random losses, which may be all that is needed.

Opinions?  Should LOOPS be more like QUIC or like protocols that have been around for a few decades?

Grüße, Carsten

---

From: Liyizhou <[email-masked]>
Date: Thu, 17 Oct 2019 12:27:57 +0000

Hi Carsten,

For packet sequence number (PSN) in a retransmit packet, using the old PSN looks more reasonable to me.

When LOOPS uses psn to link an old (lost) packet to a retransmit one, it helps a lot when developing and debugging code. Very straightforward to know if the retransmission for a specific packet works in a way as designed. And also makes troubleshooting easier when it goes to commercial version in my opinion.
Otherwise, we have to guess, e.g. packet 10 is a retransmit for packet 2, and packet 11 is a retransmit for packet 4, takes more effort for developer/tools.

QUIC uses a new packet number for a retransmit packet. At the same time, stream id and offset field can help to easily identify what have been retransmitted. However LOOPS does not look into the data. So it would be preferable to  make LOOPS header information more self-descriptive on which packet it is retransmitting. 

Certainly using old PSN causes ACK ambiguity. It further causes inaccurate RTT calculation (major) and inability to tell spurious retransmission (minor). I do not think it is a big issue in LOOPS. Timestamp can be carried and echoed back in loops. It solves the major problem of RTT calculation.  At the same time, unlink TCP, LOOPS ingress has no window to maintain. So undo window reduction after spurious retransmission is not even  needed.  

For single retransmission mode, I think it is a good idea. In our experiments,  WAN across countries, most lost packet can be recovered by retransmitting only once. But I suspect if there is any wireless link, single retransmission would not be enough. So probably we need to provide both single retransmission mode and configurable mode (in time or in  number of retransmit times).

Regards,
Yizhou

-----Original Message-----
From: LOOPS [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Thursday, October 17, 2019 4:07 AM
To: [email-masked]
Subject: [LOOPS] Packet sequence numbers

Advantages may include:

* No ACK ambiguity.

Grüße, Carsten

-- 
LOOPS mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/loops

---

From: Michael Welzl <[email-masked]>
Date: Fri, 18 Oct 2019 11:59:05 +0200

Hi,

> On Oct 17, 2019, at 2:27 PM, Liyizhou <[email-masked]> wrote:
> 
> Hi Carsten,
> 

Even though I originally suggested the QUIC method of continuously increasing sequence numbers, I think this assessment of the trade-offs is very good, and now I tend to agree that, for LOOPS, repeating sequence numbers would be better.

> For single retransmission mode, I think it is a good idea. In our experiments,  WAN across countries, most lost packet can be recovered by retransmitting only once. But I suspect if there is any wi…

+1 on the need for more-than-once, for wireless links.

Cheers,
Michael

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 21 Oct 2019 22:45:02 +0200

> Even though I originally suggested the QUIC method of continuously increasing sequence numbers, I think this assessment of the trade-offs is very good, and now I tend to agree that, for LOOPS, repe…

I think that this conclusion is a good basis for continuing the work.
I made the necessary changes (I think) in https://github.com/loops-wg/gen-info/commit/2dd7634

I also added some text about mitigating ACK ambiguity, which is facilitated if the ingress never sets “ACK desired” on a retransmission.

There is currently no detailed text on how to decide to retransmit.  The current spec is mute (a.k.a. “flexible”) on the implementation strategy.  For hole-based retransmission triggering, re-using the PSD needs to include a hold-off for packets that have recently been retransmitted so another copy of block 2 information that just confirms the existing hole doesn’t trigger another retransmission.  For a time-based retransmission scheme (as in RACK), retransmission would simply move forward the packet into the then appropriate slot.

> +1 on the need for more-than-once, for wireless links.

Right.  I think that single retransmission is a nice implementation strategy for some applications, so we should continue to pay attention that we don’t inadvertently make it harder to do than it is now.

Grüße, Carsten

---

From: "Morten V. Pedersen" <[email-masked]>
Date: Tue, 22 Oct 2019 08:18:12 +0200

Hi all,
I'm not an expert on QUIC or such, but what was the motivation for 
continuously increasing sequence numbers there. I imagine they had some 
good reasons to go down that path.

 From a FEC and re-transmission standpoint continuously increasing 
sequence numbers yields simpler implementations from my point of view.

It is relatively easy for the FEC implementation to estimate the packet 
loss, and for a pure re-transmission scheme detection of "holes" is 
trivial without worrying about what data the packet carried i.e. if it 
was payload data or a re-transmission in itself. It can just report back 
and say I did not get packet X whatever it was.

All the best,
Morten

---

From: Michael Welzl <[email-masked]>
Date: Tue, 22 Oct 2019 09:07:16 +0200

Hi all,

It is my understanding (but I might be wrong!) that the motivation was just to avoid retransmission ambiguity when there’s no need to have it - continuously growing sequence numbers make things simpler, and given the length of the field in QUIC, they would support very long transfers, so the disadvantage is minuscule. And, indeed, it generally makes things simpler.

One of the disadvantages mentioned here was related to monitoring tools - that doesn’t apply to QUIC either because its sequence numbers are encrypted.

Cheers,
Michael

> On Oct 22, 2019, at 8:18 AM, Morten V. Pedersen <[email-masked]> wrote:
> 
> Hi all,
> I'm not an expert on QUIC or such, but what was the motivation for continuously increasing sequence numbers there. I imagine they had some good reasons to go down that path.
> 
> From a FEC and re-transmission standpoint continuously increasing sequence numbers yields simpler implementations from my point of view.
> 
> It is relatively easy for the FEC implementation to estimate the packet loss, and for a pure re-transmission scheme detection of "holes" is trivial without worrying about what data the packet carried i.e. if it was payload data or a re-transmission in itself. It can just report back and say I did not get packet X whatever it was.
> 
> All the best,
> Morten
> 
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 22 Oct 2019 11:23:35 +0200

I seem to remember that one other advantage of a strictly monotonous sequence number is that this makes it directly suitable as input for a nonce for the AEAD encryption/authentication, saving bytes by not needing to send one separately.

Grüße, Carsten

---

From: "Morten V. Pedersen" <[email-masked]>
Date: Tue, 22 Oct 2019 12:17:34 +0200

Hi Carsten,
That's interesting - such a number is also used in some FEC schemes e.g. 
RLNC to seed the random number generator (used to create coding 
coefficients). With the same purpose of saving bytes on the wire.

- M