Subject: RE: [rddp] Re: Invalidation

From: "Jim Pinkerton" <[email-masked]>
Date: Wed, 1 Oct 2003 20:59:25 -0700

Here's proposed text to address this in the security draft. Note that
the threat actually has nothing to do with whether the STag is
long-lived or not - the key issue is sharing it across multiple
connections.

7.5.5	Remote Invalidation of an STag Shared Across Multiple
Connections

If a Local Peer has enabled an STag for remote access, the Remote Peer
could attempt to invalidate the STag by using the RDMAP Send with
Invalidate or Send with SE and Invalidate Message. If the STag is only
valid on the current connection (NS-NT or NS-RT, S-NT), then the only
side effect is that the Remote Peer can no longer use the STag, thus
there are no security issues.

If the STag is valid across multiple connections, then the Remote Peer
can prevent other connections from using that STag by using the remote
invalidate functionality. 

Thus for trust models that involve shared local STags across multiple
connections connecting to mutually untrusted Remote Peers, the
invalidate STag capability SHOULD be disabled for the shared STag.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Jim
> Pinkerton
> Sent: Wednesday, October 01, 2003 8:42 PM
> To: Caitlin Bestler; [email-masked]
> Cc: David Black
> Subject: RE: [rddp] Re: Invalidation
> 
> 
> David,
> 
> From reading your email, it sounds like you are thinking there is a
> layering violation? Is that your point?
> 
> If so, I'd claim otherwise. Clearly there is an interface above DDP to
> enable a buffer to be mapped to an STag, and just as clearly there is
an
> interface to invalidate that mapping. Thus to me all that is happening
> is RDMAP, as the ULP to DDP, is calling the invalidate mechanism for a
> DDP STag.
> 
> On Caitlin's points below, I agree with the Quirk summary, and also
> agree that we should make it clear that the local peer can choose to
> configure an STag to never be invalidated (regardless of whether an
> invalidate STag is received). This is effectively a DOS attack
otherwise
> (and I'll add it to the security draft).
> 
> 
> 
> Jim
> 
> 
> 
> 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On Behalf Of
> > Caitlin Bestler
> > Sent: Tuesday, September 30, 2003 8:14 AM
> > To: [email-masked]
> > Cc: David Black
> > Subject: [rddp] Re: Invalidation
> >
> >
> > On Monday, September 29, 2003, at 02:12 PM, [email-masked]
wrote:
> >
> > > (2) Invalidation.  We have a lingering issue from way back
> > > 	when that RDMAP is handling invalidation of STags,
> > > 	which are DDP level resources.  This needs some
> > > 	serious examination in this forum.
> >
> > I'm not sure I remember anything that qualifies as an
> > "issue". There are certainly some quirks that relate to
> > the fact that the invalidation is inherently asynchronous
> > because it is processed on the "completion track" and at
> > least theoretically by a different layer (and hence potentially
> > as well).
> >
> > Quirk #1: Invalidation is not a fence. "Later" DDP Segments
> > may be placed prior to the invalidation -- even if received
> > in order. Invalidation offers no guarantees to the sender,
> > only to the receiver. And the receiver is only guaranteed
> > that there will be no further placements *when the notice
> > of the invalidation is delivered*. The only guarantee about
> > what was placed at that point is that *earlier* DDP segments
> > will have been placed. There is no guarantee that *later*
> > DDP segments were *not* placed.
> >
> > Quirk #2: Invalidation may or may not disrupt an in-process
> > RDMA Read. That is, there is no specification on *when* an
> > RDMA Read engine translates STags.
> >
> >
> >
> > The last issue is that the current wording does not allow
> > a local implementation to suppress remote invalidation on
> > specific tags. This essentially prevents exporting STags
> > that are associated with permanent data structures such
> > as Memory Regions (as opposed to windows). That should be
> > fixed.
> >
> > In general, remote invalidation should not be viewed as
> > providing any guarantees to the remote side. It is a
> > service to the local ULP. While it ought to remain
> > mandatory to implement, the local ULP should have
> > the option of not using it for a specific STag.
> >
> >
> >
> > _______________________________________________
> > rddp mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/rddp
> 
> 
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 2 Oct 2003 09:48:30 -0500

> invalidate STag capability SHOULD be disabled for the shared STag.

But of course the ULP MAY have other reasons to disable remote
invalidation that are beyond the ability of transport layer
designers to comprehend. In other words, part of the definition
of "ULP" is that it is allowed to do things for stupid, I mean
application-specific, reasons.

The language proposed sounds fine, and covers the *justifying*
reason well. After all, a Memory Region is just one type of
shared STag.

But we also need permission to do this in the DDP document.
Currently it specifies that after a remote invalidation that
the STag MUST be invalid until it is re-instated by the ULP.

That should be reduced to a SHOULD, with a comments that
local interfaces MAY offer variants such as not recognizing
an invalidation.

The wire-protocol issue is what the response to a blocked
remote invalidate should be:

a) ignore it. The remote end doesn't have to know that the
    STag is still valid, because it won't be attempting to
    use it anyway.
b) treat it as an access violation and terminate the connection.

I favor A, whether thinking as an application designer or as
an RNIC designer.

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 2 Oct 2003 16:51:49 -0500

> invalidate STag capability SHOULD be disabled for the shared STag.

Thinking about this a bit more in the context of the Security draft,
the most critical advise remains that an STag should have no wider
of a scope than the ULP requires.

Hence, the ULP SHOULD have the ability to make an STag valid only
on those DDP Streams that are associated with a specific session,
or are otherwise presumed to have a trusting relationship with
each other.

In other words, if there is a basis to assume that the peers for
multiple remote peers can be trusted to share an STag, they can
probably be trusted to invalidate it.

The major exception that I can think of would be an STag that
was enabled for RDMA Read usage only. The Data Source could be
publishing status data, for example, that any client could
read at its convenience.

In that case, just as none of them are trusted to write to the
data, none would be trusted to invalidate the STag.

I also note that the inability to support dynamically bound
STags across multiple DDP Streams is, in my opinion, a critical
flaw in the RDMAC proposed iWarp verbs. It would also be easily
corrected by replacing the "QP ID" used in validating a bound
STag with a "QP Set ID". By default each QP would belong to its
own QP Set.

On the original topic, a "data publishing STag" is just one
example of many valid *local* reasons to disallow remote
invalidation.