Subject: [rddp] Re: iSCSI/iWARP drafts and flow control

From: Mike Ko <[email-masked]>
Date: Fri, 15 Aug 2003 18:04:08 -0700

As I indicated to Caitlin in an earlier e-mail, since each iSCSI 
control-type PDU from the target to the initiator is associated with a 
StatSN, the ExpStatSN from the initiator can be used to advance the 
"credit" at the target.

The question is for iSCSI control-type PDUs from the initiator to the 
target, what is the mechanism for the target to advance the "credit" at 
the initiator?  For PDUs where the Immediate flag is not set, the CmdSN 
and MaxCmdSN mechanism in iSCSI are already available and the new "credit" 
scheme is not necessary.  The unsolicited SCSI Data-out PDUs are covered 
by FirstBurstLength and again the new "credit" scheme is not needed.  The 
"credit" scheme is needed for PDUs where the Immediate flag is set or the 
SNACK Request (note that SNACKs are not needed in iSER).  For these PDUs, 
the target may not be able to advance the "credit" at the initiator 
individually.  One possibility is to wait for the initiator to issue an 
iSCSI control-type PDU with a new CmdSN but without the Immediate flag, 
and if the target responds with an iSCSI control-type PDU with ExpCmdSN 
set to CmdSN + 1, then "credit" at the initiator is restored to its full 
negotiated (or by definition) value. 

As Caitlin pointed out, since no one has yet identified a specific need 
for per-session negotiation of these credits, a simple rule enumerated in 
the draft may be sufficient. 

Mike Ko
IBM Almaden Research
[email-masked]

To:     [email-masked]
cc:     Mike Ko <[email-masked]>, [email-masked], [email-masked] 
Subject:        Re: iSCSI/iWARP drafts and flow control

On Friday, August 15, 2003, at 04:12 PM, [email-masked]
wrote:

> Sorry for my delay in responding.
>
> One of the things I've been wondering through this whole thread is
> can't
> we expand iSCSI negotiation to handle these parameters (like how many
> async messages are ok, how many immediate commands, etc.)? That way, to
> the extent that iSCSI/iWARP needs new parameters, we can decide on them
> out easily.
>
> Maybe it's been in everyone's thoughts, but the thread had been talking
> about needing new limits and talking like it's hard to do. Can't we
> just
> negotiate them?
>
> Take care,
>
> Bill

Bill,

There are two separate issues.

The first is establishing what the credit limits are. That could
be negotiated on a per session basis, or it could be established
by the draft as a simple rule. You are correct that negotiating
is not terribly complex. Therefore, if there is *any* benefit to
varying those limits on a per-session basis, then negotiations
should be allowed.

So far, however, I do not believe that anyone has identified
a specific need for per-session negotiation of these credits.

The second issue is the mechanism by which credits are replenished.
The iSER draft authors were opposed to creation of additional
messages to exchange credits. My contention is that under virtually
all circumstances, credit replenishment can be inferred from
existing message exchanges. Under worst case circumstances a
"NOP" might have to be sent that would not have otherwise
been required. No additional message types are required.

I believe the last few prior exchanges may have established
that tracking credits is not as onerous as the draft authors
had originally thought. But it is on that issue, not the
overhead of negotiation during setup, that was their concern.

Caitlin Bestler - [email-masked] - http://asomi.com/

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 15 Aug 2003 21:22:45 -0500

> StatSN, the ExpStatSN from the initiator can be used to advance the

Agreed.

> negotiated (or by definition) value.

My proposal is to apply a "leaky bucket" algorithm. "Extra" credits are
granted for each CmdSN credit granted, up to a maximum. Each extra
send decrements a fixed amount from the credits. In a pinch, Nop
commands can be used to advance the CmdSN.

It allows for bursts, and the credit-scoring can be done entirely
when sending a message that needs an "extra" credit.

> the draft may be sufficient.

Well, their negotiated credits in either case. It's just whether
they are negotiated on a per-session basis by target and
initiator, or negotiated on a mailing list on a per-protocol
basis. If the volume of the mailing list negotiation gets
too high, everyone will quickly agree to the per-session
negotiations. ;-)

---

From: [email-masked]
Date: Mon, 18 Aug 2003 05:24:53 -0700 (PDT)

On Fri, 15 Aug 2003, Caitlin Bestler wrote:

> > As Caitlin pointed out, since no one has yet identified a specific
> > need for per-session negotiation of these credits, a simple rule
> > enumerated in the draft may be sufficient.
>

I'm not saying we can't just pick a number. But if I understand things,
one advantage of per-session negotiation might be that a target with fixed
buffering could offer different command windows to initiators that wanted
different numbers of immediate commands available. For an initiator that
is more comfortable advancing CmdSN (is happier with a lower number of
credits per CmdSN), a target could divy up its buffers to allow a wider
command window.

Take care,

Bill

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 18 Aug 2003 10:34:45 -0500

> credits per CmdSN), a target could divy up its buffers to allow a wider

The cost of per-session negotiation is minor, so I suspect that
almost any solid example of an application or scenario that
would benefit from a non-standard number of credits would
be convincing.

But given that the flow control is imposed on the sending side,
and that these are rare packets, it isn't that easy to come up
with such a scenario.

I'm not opposed to the idea, but if a parameter is going to be
listed as negotiable in the draft there should be at least a
paragraph hinting at why you might want to vary it (even if
it was only on the mailing list and doesn't make it into
the draft).