Subject: [imss] Storage Maintenance (storm) BOF reminder & requests

From: [email-masked]
Date: Wed, 11 Mar 2009 19:28:10 -0400

This is a reminder that the Storage Maintenance BOF will
be held in about 2 weeks at the IETF meetings in San Francisco.
Please plan to attend if you're interested:

THURSDAY, March 26, 2009
Continental 1&2  	TSV  	storm  	 Storage Maintenance BOF

The BOF description is at:
http://www.ietf.org/mail-archive/web/ips/current/msg02669.html

The initial agenda is here:
http://www.ietf.org/mail-archive/web/ips/current/msg02670.html

I'm going to go upload that initial agenda as the BOF agenda,
and it can be bashed at the meeting.

The primary purpose of this BOF is to answer two questions:
(1) What storage maintenance work (IP Storage, Remote Direct
	Data Placement) should be done?
(2) Should an IETF Working Group be formed to undertake that
	work?

Everyone gets to weigh in on these decisions, even those who
can't attend the BOF meeting.  Anyone who thinks that there is
work that should be done, and who cannot come to the BOF meeting
should say so on the IPS or RDDP mailing lists (and it'd be a
good idea for those who can come to do this).  As part of the
email, please indicate how you're interested in helping (author
or co-author of specific drafts, promise to review and comment
on specific drafts).

Here's a summary of the initial draft list of work items:
- iSCSI: Combine RFCs into one document, removing unused features.
- iSCSI: Interoperability report on what has been implemented and
	interoperates in support of Draft Standard status for iSCSI.
- iSCSI: Add backwards-compatible features to support SAM-4.
- iFCP: The Address Translation mode of iFCP needs to be deprecated.
- RDDP MPA: Small startup update for MPI application support.
- iSER: A few minor updates based on InfiniBand experience.

Additional work (e.g., updated/improved iSNS for iSCSI, MIB changes,
updated ipsec security profile [i.e., IKEv2-based]) is possible if
there's interest.

There are (at least) four possible outcomes:
(A) None of this work needs to be done.
(B) There are some small work items that make sense.  Individual
	drafts with a draft shepherd (i.e., David Black) will
	suffice.
(C) A working group is needed to undertake more complex work
	items and reach consensus on design issues.  The WG can
	be "virtual" and operate mostly via the mailing list
	until/unless controversial/contentious issues arise.
(D) There is a lot of complex work that is needed, and a WG
	that will plan to meet at every IETF meeting should be
	formed.

Please note that the IETF "rough consensus" process requires a
working group in practice to be effective.  This makes outcome
(C) look attractive to me, as:
- I'm coming under increasing pressure to limit travel, and
	the next two IETF meetings after San Francisco are not
	in the US.
- I'd rather have the "rough consensus" process available and
	not need it than need it and not have it available.

Setting an example for how to express interest ...

---------------
I think that the iSCSI single RFC and interoperability report are
good ideas, but I want to see a bunch of people expressing interest
in these, as significant effort is involved.  It might make sense
to do the single iSCSI RFC but put off the interoperability report
(the resulting RFC would remain at Proposed Standard rather than
going to Draft Standard), as I'm not hearing about major iSCSI
interoperability issues.

I think the latter four items (SAM-4 for iSCSI, deprecate iFCP
address translation, MPI fix to MPA and iSER fixes) should all
be done.

I plan to author the iFCP address translation deprecation draft,
and review all other drafts.

I think that a virtual WG should be formed that plans to do its
work primarily via the mailing list.  I believe the SAM-4 work
by itself is complex enough to need a working group - I would
expect design issues to turn up at least there and in determining
whether to remove certain iSCSI features, but I'm cautiously
optimistic that the mailing list is sufficient to work these
issues out (and concerned that travel restrictions are likely to
force use of the mailing list).

-----------------

Ok, who wants to go next?

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: [email-masked]
Date: Wed, 11 Mar 2009 19:34:47 -0400

I put everything in there except the meeting time.  Try this:

THURSDAY, March 26, 2009 *** 0900-1130 Morning Session I ***
Continental 1&2  	TSV  	storm  	 Storage Maintenance BOF

Sorry,
--David

> -----Original Message-----
> From: Black, David 
> Sent: Wednesday, March 11, 2009 7:28 PM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]; Black, David
> Subject: Storage Maintenance (storm) BOF reminder & requests
> Importance: High
>

---

From: Stephen Bailey <[email-masked]>
Date: Wed, 11 Mar 2009 18:05:00 -0700

Any chance we could dramatically exceed our brief on this project and 
totally rewrite everything?

:-)
...... Original Message .......
On Wed, 11 Mar 2009 19:28:10 -0400 <[email-masked]> wrote:
>issues out (and concerned that travel restrictions are likely t

---

From: Lars Eggert <[email-masked]>
Date: Thu, 12 Mar 2009 11:36:09 +0200

FYI, the IESG is in the process of figuring out if a more useful  
remote participation mechanism can be set up for SF than our usual  
audio streaming and jabber room. If that can be done, STORM may want  
to make use of it, given that some of your key folks won't be attending.

On 2009-3-12, at 1:28, [email-masked] wrote:

> _______________________________________________
> Ips mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ips

---

From: "Knight, Frederick" <[email-masked]>
Date: Thu, 12 Mar 2009 11:20:11 -0400

Hello David,  I'll be seeing you there in San Francisco.  We have a high
interest in these areas.  As we've discussed I plan to author the
SAM-4/5 feature additions, and contribute heavily to the other items
you've listed.

I too believe your option "C" is the minimum, and we'll have to see if
"D" is needed from time to time.

	Fred Knight

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Wednesday, March 11, 2009 7:28 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: [Ips] Storage Maintenance (storm) BOF reminder & requests
Importance: High

This is a reminder that the Storage Maintenance BOF will be held in
about 2 weeks at the IETF meetings in San Francisco.
Please plan to attend if you're interested:

THURSDAY, March 26, 2009

I'm going to go upload that initial agenda as the BOF agenda, and it can
be bashed at the meeting.

Everyone gets to weigh in on these decisions, even those who can't
attend the BOF meeting.  Anyone who thinks that there is work that
should be done, and who cannot come to the BOF meeting should say so on
the IPS or RDDP mailing lists (and it'd be a good idea for those who can
come to do this).  As part of the email, please indicate how you're
interested in helping (author or co-author of specific drafts, promise
to review and comment on specific drafts).

Please note that the IETF "rough consensus" process requires a working
group in practice to be effective.  This makes outcome
(C) look attractive to me, as:
- I'm coming under increasing pressure to limit travel, and
	the next two IETF meetings after San Francisco are not
	in the US.
- I'd rather have the "rough consensus" process available and
	not need it than need it and not have it available.

---------------
I think that the iSCSI single RFC and interoperability report are good
ideas, but I want to see a bunch of people expressing interest in these,
as significant effort is involved.  It might make sense to do the single
iSCSI RFC but put off the interoperability report (the resulting RFC
would remain at Proposed Standard rather than going to Draft Standard),
as I'm not hearing about major iSCSI interoperability issues.

I think the latter four items (SAM-4 for iSCSI, deprecate iFCP address
translation, MPI fix to MPA and iSER fixes) should all be done.

I plan to author the iFCP address translation deprecation draft, and
review all other drafts.

I think that a virtual WG should be formed that plans to do its work
primarily via the mailing list.  I believe the SAM-4 work by itself is
complex enough to need a working group - I would expect design issues to
turn up at least there and in determining whether to remove certain
iSCSI features, but I'm cautiously optimistic that the mailing list is
sufficient to work these issues out (and concerned that travel
restrictions are likely to force use of the mailing list).

-----------------

Thanks,
--David

---

From: "Mallikarjun C." <[email-masked]>
Date: Sat, 14 Mar 2009 17:47:49 -0700 (PDT)

Hi David,

My recommendation would be to defer the interoperability survey and draft consolidation work for iSCSI.  I do believe however that at least iSCSI SAM-4 compliance work in a backwards compatible fashion is required.  I currently plan to contribute to/author this work, although I unfortunately will not be able to attend the San Francisco meeting in person.

I tend to prefer your option C for the same reasons you cited.  Option B *might be* workable as the next preferred option.  One other suggestion I have is that even with a virtual WG (option C), I believe we will need to move at a relatively slower pace compared to the IPS WG because given the economy, everyone has enough workload on their plates in their companies and has fewer cycles to stay on top of the WG mailing list.  IMHO, this will need to be factored into milestones and such in the planning process.  

Thanks. --
Mallikarjun Chadalapaka 

----- Original Message ----
> From: "[email-masked]" <[email-masked]>
> Sent: Wednesday, March 11, 2009 4:28:10 PM
>  +1...            FAX: +1 (508) 293-7786

---

From: "Robert D. Russell" <[email-masked]>
Date: Sun, 22 Mar 2009 11:41:55 -0400 (EDT)

David:

There are 2 issues I would like to suggest for discussion at the BOF
meeting later this week.  Both have to do with the iSER spec, RFC 5046.

1. At the present time, as far as I know, no existing hardware,
    neither Infiniband nor iWARP, is capable of opening a connection
    in "normal" TCP mode and then transitioning it into zero-copy mode.
    Unfortunately, the iSER spec requires that.
    Can't we just replace that part of the iSER spec?
    Otherwise, all hardware and all implementations are non-standard.

2. The OFED stack is used to access both Infiniband and iWARP hardware.
    This software requires 2 extra 64-bit fields for addressing
    on both Infiniband and iWARP hardware, but these fields
    are not allowed for in the current iSER Header Format.
    Can't we just add those extra fields to the iSER spec?
    If someday some other implementation doesn't need those
    fields, they can be just set to 0 (which is what is implied by
    the current iSER standard anyway).  Again, by not doing this,
    all implementations are non-standard.

In other words, I'm suggesting that we consider replacing the relevant
parts of the current iSER specs with the current OFED specs on these
2 issues.

Thanks for your consideration,
Bob Russell

Note: The following (old) posting by Mike Ko states that the
extra header fields are needed only by IB, not by IETF
(i.e., iWARP), because IB uses nZBVA, whereas iWARP uses ZBVA.

But are there any IETF/iWARP implementations out there that actually
use ZBVA with iWARP RNICs?  (I don't mean software simulations of
the iWARP protocol.)  We have built an iSER implementation that
uses the OFED stack to access both IB and iWARP hardware, and for
both of them we need to use the extra iSER header fields (nZBVA).
Perhaps this is an issue with the design of the OFED stack, which
was built primarily to access IB hardware and therefore reflects
the needs of the IB hardware.  But we found that the only way to
access iWARP hardware via the OFED stack was to used the expanded
(nZBVA) iSER header (and to use a meaningful value in the extra field,
NOT to just set it to zero).

In any case, rather than have 2 different versions of the iSER header,
it would be better to have just one, regardless of the underlying
technology involved (after all, isn't that what a standard is for??).
This is especially relevant when using the OFED stack, because,
as we have demonstrated, software built on top of the OFED stack can
(AND SHOULD!) be able to run with EITHER IB or iWARP hardware,
with NO change to that software.  Having 2 different iSER headers
does NOT make that possible!

> 2008/4/15 Mike Ko <mako at almaden.ibm.com>:
>
> VA is a concept introduced in an Infiniband annex to support iSER. It
> appears in the expanded iSER header for Infiniband use only to support the
> non-Zero Based Virtual Address (non-ZBVA) used in Infiniband vs the ZBVA
> used in IETF.

Mike - could you please put me in contact with someone who has actually
implemented iSER on top of IETF/iWARP hardware NICs using ZBVA?

>
> "The DataDescriptorOut describes the I/O buffer starting with the immediate
> unsolicited data (if any), followed by the non-immediate unsolicited data
> (if any) and solicited data." If non-ZBVA mode is used, then VA points to
> the beginning of this buffer. So in your example, the VA field in the
> expanded iSER header will be zero. Note that for IETF, ZBVA is assumed and
> there is no provision to specify a different VA in the iSER header.

Mike - I believe this VA field in the expanded iSER header is almost
NEVER zero -- it is always an actual virtual address.

>
> Tagged offset (TO) refers to the offset within a tagged buffer in RDMA Write
> and RDMA Read Request Messages. When sending non-immediate unsolicited
> data, Send Message types are used and the TO field is not present. Instead,
> the buffer offset is appropriately represented by the Buffer Offset field in
> the SCSI Data-Out PDU. Note that Tagged Offset is not the same as write VA
> and it does not appear in the iSER header.
>
>Mike

---

From: "Felix Marti" <[email-masked]>
Date: Mon, 23 Mar 2009 09:36:44 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf
Of
> Robert D. Russell
> Sent: Sunday, March 22, 2009 8:42 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: [Junk released by Allow List] Re: [rddp] [Ips] Storage
> Maintenance (storm) BOF reminder & requests
> 
> David:
> 
[felix] Note that iWarp does start life as a 'streaming' connection
(normal) and is then upgraded to iWarp (zero-copy). However, you might
refer to the fact that linux doesn't support this, as the linux network
stack maintainers are unwilling to properly integrate iWarp devices.
 
> rddp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rddp

---

From: Mikkel Hagen <[email-masked]>
Date: Mon, 23 Mar 2009 12:41:48 -0400

My impression is that while it does technically start in streaming
mode, if it receives anything other than a MPA request it will abort
the connection, thus not being usable at all?

Mikkel Hagen
Research and Development
Data Center Bridging/FC/OFA/SAS/SATA/iSCSI/iWARP Consortiums
Phone:1-603-862-5083 Fax:1-603-862-4181
UNH-IOL
121 Technology Drive, Suite 2
Durham, NH 03824

Felix Marti wrote:

-----Original Message-----

Of

Robert D. Russell

David:

There are 2 issues I would like to suggest for discussion at the BOF
meeting later this week. Both have to do with the iSER spec, RFC

5046.

1. At the present time, as far as I know, no existing hardware,
neither Infiniband nor iWARP, is capable of opening a connection
in "normal" TCP mode and then transitioning it into zero-copy

mode.

2. The OFED stack is used to access both Infiniband and iWARP

hardware.

VA is a concept introduced in an Infiniband annex to support iSER.

It

appears in the expanded iSER header for Infiniband use only to

support the

non-Zero Based Virtual Address (non-ZBVA) used in Infiniband vs the

ZBVA

used in IETF.

Mike - could you please put me in contact with someone who has

actually

"The DataDescriptorOut describes the I/O buffer starting with the

immediate

unsolicited data (if any), followed by the non-immediate unsolicited

data

(if any) and solicited data." If non-ZBVA mode is used, then VA

points to

the beginning of this buffer. So in your example, the VA field in

the

expanded iSER header will be zero. Note that for IETF, ZBVA is

assumed and

Tagged offset (TO) refers to the offset within a tagged buffer in

RDMA Write

and RDMA Read Request Messages. When sending non-immediate

unsolicited

data, Send Message types are used and the TO field is not present.

Instead,

the buffer offset is appropriately represented by the Buffer Offset

field in

the SCSI Data-Out PDU. Note that Tagged Offset is not the same as

write VA

Mike

THURSDAY, March 26, 2009

---------------

-----------------

Thanks,
--David

---

From: Bernard Metzler <[email-masked]>
Date: Tue, 24 Mar 2009 07:26:29 -0800

Hi Robert,

an RNIC would be happy to transition an established socket
into RDMA mode if the user wants to. No problem - as far as i know -
to transition from TOE mode to RDMA mode at least. It is a question 
of envrionmental support to allow that. To allow that,
the environment (aka OFED) wold have to integrate late
socket translation, which allows RNIC drivers to officially do
TOE to RDMA mode transition or to take over a socket 
(where the OS would have to expose a well-defined TCP
context reallocation scheme).
Maybe it is not a good idea to change an end-to-end protocol
because a host environment has functional limitations
which may stem from historical transport (IB) limitations or
OS considerations.

many thanks,
bernard.

ZBVA
immediate
data
Instead,
field in
write VA

---

From: "Uri Elzur" <[email-masked]>
Date: Tue, 24 Mar 2009 16:54:18 -0700

Hi David

Unfortunately I won't be able to make it in person to the meeting this week. However, I'd like to extend our support to the following items:

I think that a virtual WG should be formed that plans to do its
work primarily via the mailing list.  This will give us an opportunity to flush out the issues and have a better assessment of the priorities and effort associated with the activities. At that point, we can decide if/when to go into formal WG

thx

Uri  ("Oo-ree") 
W: 949-926-6432 
C:  949-292-6098 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Wednesday, March 11, 2009 4:28 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: [Ips] Storage Maintenance (storm) BOF reminder & requests
Importance: High

THURSDAY, March 26, 2009

---------------

-----------------

Thanks,
--David