Subject: RE: [rddp] Some questions related to "last segment" of a ddp message

From: [email-masked]
Date: Fri, 23 Jan 2004 13:39:01 -0700

Barry

I believe the only ordering constraint we have in the draft is that the segment with the L bit set must have the highest message offset. We did that because we didn't want the require the receiver to scoreboard and it is therefore necessary for correct operation. It allows the receiver to determine total message length to report based on the information in the last segment.

We have no provisions in the untagged message passing for messages with holes. So, the message sequence you presented would be an incorrect sequence because of the gaps. If you added segments or changed offsets and lengths to fill the gaps, it would be a valid sequence. As it is, the gaps may be detected as an error by some receivers.

It is possible that the draft could use a clearer statement that there should be no holes in a message.

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Barry
Reinhold
Sent: Thursday, January 15, 2004 1:37 PM
To: Hemal Shah
Cc: RDDP
Subject: [rddp] Some questions related to "last segment" of a ddp
message

Hemal,

I would like to check on a couple of issues:

I believe that it is possible to RECEIVE a sequence of DDP segments
using the untagged buffer model as follows: Let me know if this is not
correct.

Segment 1: QN=1, MSN=1, MO=200, L=0 (payload 100) (MO not 0 on first)
Segment 2: QN=1, MSN=1, MO=200, L=0 (payload 100) (duplicate)
Segment 3: QN=1, MSN=1, MO=100, L=0 (payload 100) (backward)
Segment 4: QN=1, MSN=1, MO=000, L=0 (payload 050) (meets 6.3, MO field)
Segment 5: QN=1, MSN=1, MO=400, L=1 (payload 000) (meets 6.1 L bit
field)

In particular, 
1. If no packets have been received w/ a given MSN and QN and a packet
arrives with a non-zero MO this is not an error.
2. The MO fields in intermediate segments are not constrained to be in
order, can have gaps, and can write over buffer space already written
to.

Or from a test perspective, if a receiver returned an error when
receiving the above sequence the receiver would be considered
non-conformant.

I am also a bit confused by the use of the term "last" in the draft.

If the above is considered a valid sequence it is unclear to me how the
receiver can meet the requirements of knowing that all DDP Segments of
the DDP message have been placed before it "delivers" the DDP message to
the ULP. (page 14 under L - Last flag, first bullet)

Is there some method other than the L bit that indicates that all
segments of a DDP message have been placed? (i.e. do implementations
need to keep a scoreboard?) The draft (in clause 7.4, first *, on page
20) references a requirement on the receiver to deliver only when the
last bit is set on the "last DDP Segment". How is the last segment
defined beyond the L bit?

Barry Reinhold
Lamprey Networks
[email-masked]
(603) 868-8411

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: [email-masked]
Date: Fri, 23 Jan 2004 13:53:13 -0700

Barry,

The last segment is defined by the L bit and the segment with the L bit must be the actual last segment in the stream sent to the transport. See the definition of the L bit in 6.2 which says 
	The DDP Segment with the L bit set to 1 MUST be posted to the 
        LLP after all other DDP Segments of the associated DDP Message 
        have been posted to the LLP. For an Untagged DDP Message, the 
        DDP Segment with the L bit set to 1 MUST carry the highest MO. 

The requirements on the transport include that it must allow DDP to determine  that all prior segments have been received (see requirement 5 in section 5). The ordering rules in 7.4 require that all preceding DDP messages MUST have been placed before a message may be delivered. So, segments 1 through 4 in your example MUST be placed before the completion in from segment 5 is delivered.

The receiver doesn't have to scoreboard data placement. It only has to know that the prior data stream has no gaps and that all placements for that prior data stream have been completed.

Regards,
Pat

-----Original Message-----

Hemal,

In particular, 

Barry Reinhold
Lamprey Networks
[email-masked]
(603) 868-8411

---

From: [email-masked]
Date: Tue, 27 Jan 2004 16:35:34 -0700

Barry,

I don't think I wrote anything that conflicts with your statements at 1 and 2 below. I agree that the sender is allowed to send out of order and that the receiver therefore has to tolerate out of order arrival. The only thing the receiver can count on is that the highest MO will come in the last message. 

I'm not clear what you think was excessively liberal in my message.

The transmitter is required to send the whole message. Where I believe the receiver has some freedom is it can:
Trust the transmitter to send the whole message and report message completion based on MO sent in the last segment, or
Scoreboard the message and report an error if there are gaps or if the transmitter sent higher MOs in the segments before the last. 

There aren't any words that prohibit transmitting overlaps so they should probably be treated as not an error.

Regards,
Pat

-----Original Message-----
From: Barry Reinhold [mailto:[email-masked]]
Sent: Saturday, January 24, 2004 11:35 AM
To: THALER,PAT (A-Roseville,ex1); [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [rddp] Some questions related to "last segment" of a ddp
message

Pat,
	Although I don't consider myself a conservative, the
interpretation of the specification here is starting to get too liberal
for my taste. (Sorry, it's primary time here in New Hampshire...)

I do think there is an issue here in terms of what we are expecting when
the sender "SHOULD" send a message in a particular format. If the sender
SHOULD send the message using format A, this implies that the receiver
MUST accept format A and B (assuming a choice between A and B). Thus the
receiver MUST NOT return an error if the message arrives using format
"B".

This is important for interoperability. If a specification allows a
behavior (SHOULD implies allowing a behavior that is not the "desired"
behavior) then the receiver has got to support it, otherwise some pairs
of devices just won't work.

I do not have a strong opinion as to what is the proper choice in terms
of behavior with overlaps/gaps. However, I do think it is important that
the specification be written in a way that the behavior is clear.

As currently written I believe that the draft:

1. Requires that all data of a DDP message must be sent

(Clause 7.2 - "At the Data Source, the DDP layer MUST send all 
   the data contained in the ULP message.")

2. States that the sender is not required to send DDP segments in order

(Clause 7.3 - "At the Data Source, DDP: 

     * MUST transmit DDP Messages in the order they were submitted to 
        the DDP layer, [Note that this is messages, not segments]

     * SHOULD transmit DDP Segments within a DDP Message in increasing 
        MO order for Untagged DDP Messages and in increasing TO order 
        for Tagged DDP Messages.")

The above statements, I believe, imply that the receiving DDP layer MUST
NOT send an error for duplicated data or out of order data. One could
argue that a gap is impossible based on point 1, but it would be a weak,
or indirect, argument.

I would propose that we identify what is desired (do we allow
duplicates/gaps?) and write the document so that the behavior of the
receiver is clear. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, January 23, 2004 3:29 PM
To: [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [rddp] Some questions related to "last segment" of a ddp
message

Paul,

RE: 2) Where are payload bytes 50-99?  300-399?
[prc] The receiver is not expected to check for overlaps, or holes in
the data.  So this is not a valid reason to report errors and there are
no defined errors for this.  Perhaps the draft should explicitly say not
to report errors for these cases.  As long as all the  transmitter's
data was placed and delivered, the receiver is compliant.  In the case
of overlaps, the receiver is not even required to make sure that the
highest TCP sequence numbered segment was the last placed, so overlaps
could lead to unpredictable delivered data.  As a consequence of this,
the transmitter SHOULD send data with no overlaps or holes, although an
application could conceivably be designed to work even if this is not
followed (some kind of test suite?).
[prc]

The receiver is not required to check for overlaps and holes but there
isn't anyplace that disallows it doing so. I believe that this was our
intent when we wrote it and it is consistant with the text in the draft
- 7.2 says at the end that the Data Sink is not required to verify that
the entire message has been received. It doesn't prohibit checking. 

So it would be valid behavior to detect an error because of the missing
bytes. It would also be valid behavior to not detect that error.

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Culley, Paul
Sent: Friday, January 16, 2004 7:33 AM
To: Michael C. Cambria; RDDP
Cc: Barry Reinhold
Subject: RE: [rddp] Some questions related to "last segment" of a ddp
message

A couple of points are included below...

Paul R. Culley
HP Fellow
281-514-5543

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Michael C. Cambria
Sent: Friday, January 16, 2004 8:39 AM
To: RDDP
Cc: Barry Reinhold
Subject: Re: [rddp] Some questions related to "last segment" of a ddp
message

> Segment 2: QN=1, MSN=1, MO=200, L=0 (payload 100) (duplicate) Segment 
> 3: QN=1, MSN=1, MO=100, L=0 (payload 100) (backward) Segment 4: QN=1, 
> MSN=1, MO=000, L=0 (payload 050) (meets 6.3, MO field) Segment 5: 
> QN=1, MSN=1, MO=400, L=1 (payload 000) (meets 6.1 L bit
> field)
> 
> In particular,

My understanding is yes, yes and (re write over buffer space) not
convinced.

On pg 20 of draft-ietf-rddp-ddp-01.txt, "For an Untagged DDP Message, 
the combination of the QN and MSN uniquely identifies a DDP Message." 
Since an MSN doesn't make sense for a DDP _message_, I've read this to 
mean that QN & MSN uniquely identifies a DDP _segment_.
[prc] I suspect some misunderstanding here.  An MSN (Message Sequence
Number)
specifically identifies which _message_ on the particular Queue the
segment
belongs to.  Think of a queue with 5 2000 byte buffers, If TCP can
manage 
1500 bytes or so per segment, each buffer can hold two segments.  The
two segments for the second buffer might be identified by QN=1, MSN=2,
MO=0, L=1500 and QN=1, MSN=2, MO=1500, L=500.  The two segments have the
same MSN, so your statement is not correct.
[prc]

So the way I read the draft, I can't find text that says one cannot send

overlapping DDP segments.

> receiving the above sequence the receiver would be considered 

I'd return an error for several reasons.

1) I know of no legal untagged message on QN=1 that has a payload as 
large as you describe.
[prc] for DDP, all QN values are legal, and only the ULP can place
limits on their size, what you say is true only if you are using RDMA as
the next up ULP. [prc]

2) Where are payload bytes 50-99?  300-399?
[prc] The receiver is not expected to check for overlaps, or holes in
the data.  So this is not a valid reason to report errors and there are
no defined errors for this.  Perhaps the draft should explicitly say not
to report errors for these cases.  As long as all the  transmitter's
data was placed and delivered, the receiver is compliant.  In the case
of overlaps, the receiver is not even required to make sure that the
highest TCP sequence numbered segment was the last placed, so overlaps
could lead to unpredictable delivered data.  As a consequence of this,
the transmitter SHOULD send data with no overlaps or holes, although an
application could conceivably be designed to work even if this is not
followed (some kind of test suite?).
[prc]
If I understand the example correctly, segment 4 provides DDP/RDMA 
message bytes 0-49; segment 3 provides 100-199, segments 1 & 2 provide 
200-300

[prc] one last comment; segment 5 (with zero length) is legal, and must
be delivered.  Further, even if no other data for the message arrived,
the zero length message does consume an untagged buffer.
[prc]

MikeC