Subject: RE: [rddp] WG chair on buffer exhaustion (again)

From: "Kanevsky, Arkady" <[email-masked]>
Date: Fri, 1 Aug 2003 06:42:45 -0700

RC has has parts to its definition. 
One of them states that any failure to deliver message (process posted
WQE) over connection
breaks the connection.

Are you saying that failure to deliver message causes retransmission or
that failure to deliver message causes CQE with ~success but connection
stays?

Arkady Kanevsky                                email: [email-masked]
Network Appliance                              phone: 781-768-5395
375 Totten Pond Rd.                            Fax: 781-895-1195
3rd Floor                                              http:
www.netapp.com
Waltham, MA 02451-2010                  general phone: 781-768-5300

> -----Original Message-----
> From: Mallikarjun C. [mailto:[email-masked]] 
> Sent: Thursday, July 31, 2003 8:54 PM
> To: Kanevsky, Arkady; [email-masked]; [email-masked]
> Subject: Re: [rddp] WG chair on buffer exhaustion (again)
> 
> 
> Arkady,
> 
> Please note the
> 
> "....MUST however ensure that the affected DDP Segment 
>   is eventually recovered ....way that guarantees ULP-transparency"
> 
> language in my suggested text.  So the proposed text does not 
> change anything wrt RDMAP and thus the automatic STag 
> invalidation, and of course the definition of reliable 
> connection stays the same as it was because the delivery 
> semantics are still in-order.
> 
> Regards.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions
> Hewlett-Packard MS 5668 
> Roseville CA 95747
> [email-masked]
> 
> ----- Original Message ----- 
> From: "Kanevsky, Arkady" <[email-masked]>
> To: <[email-masked]>; <[email-masked]>
> Sent: Thursday, July 31, 2003 8:37 AM
> Subject: RE: [rddp] WG chair on buffer exhaustion (again)
> 
> 
> > David,
> > I am trying to comprehand the 2 issues.
> > 
> > You stated that Mallikarjun's issue involves statistical (under-)
> > provisioning of buffers can happen without any Shared
> > Receive queues.
> > 1. What is the definition of reliable connection based on 
> this change?
> > 2. How does UPL knows/control what is the connection behavior:
> > connection broken(termination DDP stream) if no recv buffer is
> > availble vs.
> > Send message dropped on the floor and not delivered, but DDP
> > connection stays?
> > 3. How is the Stag invalidation effected (if at all) with 
> both proposals?
> > 
> > Thanks,
> > Arkady
> > 
> > Arkady Kanevsky                                email: 
> [email-masked]
> > Network Appliance                              phone: 781-768-5395
> > 375 Totten Pond Rd.                            Fax: 781-895-1195
> > 3rd Floor                                              
> http: www.netapp.com
> > Waltham, MA 02451-2010                  general phone: 781-768-5300
> > 
> > 
> > > -----Original Message-----
> > > From: [email-masked] [mailto:[email-masked]]
> > > Sent: Wednesday, July 30, 2003 5:40 PM
> > > To: [email-masked]
> > > Subject: [rddp] WG chair on buffer exhaustion (again)
> > > 
> > > 
> > > With WG Chair hat on, let me step into this debate:
> > > 
> > > > > Section 3.2 to:
> > > > >     " Note, however, that the ULP needs to address 
> flow control
> > > > > issues.
> > > Any
> > > > >       ULP that runs on DDP SHOULD consider the lack of an
> > > associated
> > > receive
> > > > >       buffer to place an inbound Untagged Message of 
> the ULP as 
> > > > > an
> > > exception."
> > > > 
> > > > If the local interface has allows the DDP endpoint to know
> > > the valid
> > > > range of MSNs on a stream (which it will for simple Receive
> > > Queues at
> > > > the minimum) then it definitely SHOULD NOT consider the 
> lack of a
> > > > buffer for an *invalid message* to be "an exception".
> > > 
> > > Two issues are getting conflated here, and need to be 
> kept separate:
> > > - Mallikarjun's issue involves statistical (under-)
> > > provisioning of buffers,
> > > so that an otherwise valid untagged message shows up 
> > > without a buffer
> > > available to receive it (if the implementation had done 
> > > worst-case
> > > provisioning, there would have been a buffer 
> > > available).  I believe
> > > there is WG rough consensus that this is an exceptional 
> > > condition
> > > that does not need to immediately cause the termination 
> > > of the DDP
> > > stream that sent the message, but it SHOULD not occur 
> > > frequently.
> > > Mallikarjun's proposed text is close enough to what's 
> > > needed for this.
> > > This condition could happen even on a simple Receive Queue if
> > > statistical (under-) provisioning is being used.
> > > - Caitlin's issue involves a peer violating a fixed 
> > > per-DDP-stream limit on
> > > the number of untagged messages allowed .  At the 
> > > moment, the exact
> > > need and nature of such limits is still an open issue 
> > > that needs to be
> > > resolved in the context of the security draft.  When we 
> > > have it resolved
> > > there, additional text will (probably) get driven into 
> > > the DDP draft.
> > > If the issue is resolved in the manner Caitlin advocates, then
> > > her proposed text is certainly a candidate for what the 
> > > DDP draft
> > > will need to say, but at the moment, the issue is open, and the
> > > DDP draft should not change what it says about this issue until
> > > it is resolved in the context of the security draft.
> > > 
> > > Also, I've seen some confusing references to iSER and
> > > assumptions about local interface and resource sharing model. 
> > >  Shared Receive Queues have
> > > *not* been approved/adopted by this WG (and can't be until we 
> > > get the resource sharing issue [2nd bullet above] resolved).  
> > > To the extent that iSER requires Shared Receive Queues and 
> > > can't cope with any other local resource model, it's probably 
> > > made an overly restrictive design assumption that it's 
> > > designers would be well-advised to reconsider, keeping in 
> > > mind that the Hilland verbs draft is at best an *example* of 
> > > a local interface to RDDP; it should not be viewed as 
> > > normative functional requirements on that local interface.
> > > 
> > > With luck, that's useful clarification, but feel free to ask
> > > if anything above doesn't make sense ...
> > > 
> > > Thanks,
> > > --David
> > > ----------------------------------------------------
> > > David L. Black, Senior Technologist
> > > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > > +1 (508) 293-7953             FAX: +1 (508) 293-7786
> > > [email-masked]        Mobile: +1 (978) 394-7754
> > > ----------------------------------------------------
> > > 
> > > _______________________________________________
> > > rddp mailing list
> > > [email-masked]
> > > https://www1.ietf.org/mailman/listinfo/rddp
> > > 
> > 
> > _______________________________________________
> > rddp mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/rddp
> > 
> 
> 
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 1 Aug 2003 09:51:18 -0500

On Friday, August 1, 2003, at 08:42 AM, Kanevsky, Arkady wrote:

> WQE) over connection breaks the connection.
>

Any failure will result in terminating the stream (which for
MPA/TCP means breaking the connection). The proposal is to
essentially allow a "conditional placement" interface from
the LLP to DDP.

If a segment/chunk would not be placeable immediately, the
LLP would have the *option* of *delaying* placement. Exactly
what tricks it would use to delay placement are out of scope,
other than there is an expressed preference that forcing the
other side to do an LLP retransmit is NOT preferred and
should be used minimally and only as a last resort. If
the LLP wants to volunteer to hold the buffers for awhile,
that is just fine with DDP.

The LLP merely *wants* to minimize its own buffering.
But it MAY choose to provide that service. DDP, by
contrast, does not buffer.

So whatever delay  mechanism is used, LLP buffering
or LLP retransmission, there is simply no ULP visible
event to be placed on the CQE until placement *and*
delivery is successful or the LLP insists that DDP
accept placement *now* and DDP is forced to terminate
the stream for lack of a buffer.