Subject: Re: [Tsvwg] sctpsocket: Returning unsent messages

From: Jonathan Wood <[email-masked]>
Date: Fri, 22 Jun 2001 17:14:03 -0700 (PDT)

OK, I see.

So my next question is: do we only want to return
1. unsent messages only
2. unsent and sent-but-not-ack'd messages
3. unsent and sent-but-not-ack'd and sent-and-SACK'd messages

3 will get you everything in the transmit queue.

It seems to me that since the problem to be solved is to
optimize changeover, 3 would be the best. Thoughts?

In any case, no matter what we choose, I need to add
some text to the draft explaining that there is no
guarantee that the returned messages comprise the set of
all messages not received by the peer application. If
apps need this, they need to add their own mechanisms.

Jonathan

> 
> Some implementations of M2UA on the MGC will want to
> retrieve the unsent messages from a failed SCTP association
> and resend them on an alternate link when doing changeover.
> 
> Of course, some messages will be dropped when doing this,
> but not as many as if retrieval of unsent messages was
> not available.
> 
> MTP requires that the number of dropped messages be as 
> low as possible, not that it be zero.
> 
> Jonathan Wood wrote:
> > 
> > I am unsure as to whether or not returning unsent messages
> > after a shutdown/abort is actually useful, mostly because I
> > am unclear on exactly what problem this is trying to solve.
> > 
> > The only problem I think of that this might try to solve
> > is to enable the sender to know which messages never "made
> > it", so it could, for example, try to resend them to a
> > different server.
> > 
> > However, this problem is not solved by returning unsent
> > messages, since messages that may have been sent may not
> > have been delivered, and even messages that the peer
> > has acknowleged may not have been delivered to or accepted
> > by the application at the other end. In other words, the
> > set of all unsent messages does not necessarily comprise
> > the set of all messages that were not delivered to the peer
> > application.
> > 
> > I would like some more information from the WG on exactly
> > what problem(s) returning unsent messages is trying to
> > solve, and how it is expected to solve them. If it turns out
> > that this functionality is indeed not useful, I propose we
> > drop it from the SCTP sockets extensions draft.
> > 
> > (I am not suggesting, however, that we remove the SCTP_SEND_FAILED
> >  notification for errors like bad stream ID, etc.)
> > 
> > Jonathan

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Jacob Heitz <[email-masked]>
Date: Fri, 22 Jun 2001 18:06:11 -0700

Duplicating messages is 1000 times worse than dropping
messages. M2UA wants to retrieve just those messages that
are in the transmit queue and have never been sent. It
does not want to retrieve the messages in the retransmission
buffer, because there is a risk that those messages may
have actually arrived, in which case they would be
duplicated after retrieval.

To make the distinction, rfc2960 has:

N) Receive unsent message

and

O) Receive unacknowledged message.

Kacheong Poon wrote:
> Note that it is not just drop, there can be duplicate too.  The
> other end may have received the message but the ack never gets
> back.  So I believe the app must have some form of duplicate
> detection mechanism.  Does it also mean that when the changeover
> happens, the app can tell the other end exactly what to resend
> instead of relying on SCTP?  If some messages are not available,
> it still knows exactly how many are missing.
> 
> I think the issue here is if we want to do the extra effort to
> provide an unreliable error reporting mechanism to the apps.
> 
>                                                         K. Poon.
>                                                         [email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Fri, 22 Jun 2001 18:47:28 -0700 (PDT)

> O) Receive unacknowledged message.

So it seems to me that there is no use for receiving unack'ed
message because of the ambiguity.  Note that we do not want
to add things to the socket API which is of no use.

BTW, does it also mean that M2UA has no internal mechanism
to tell how many messages are missing?

							K. Poon.
							[email-masked]

---

From: Jacob Heitz <[email-masked]>
Date: Fri, 22 Jun 2001 19:51:24 -0700

> to tell how many messages are missing?

A part of the MTP changeover procedure queries the remote
end over an alternate link about which messages have been 
received on the failed link. The local MTP can use this 
information to retrieve the correct messages from the
retransmission buffer. 

In the case of M2UA, this is done
at the SG, so is of no use if the SCTP association is down.

However, M2PA does use the changeover information and will
retrieve unacknowledged messages. See 4.2.6 of M2PA-02
for more info.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 22 Jun 2001 22:21:34 -0500

Jonathan,

The M2UA drafts do not require the retrieval mechanism.  In fact,
the current draft does not even require SCTP as a transport.  The
current M2UA draft does not make any recommendations concerning SCTP
retrieval.  AFAIK it does not even mention it.

You are correct in that one solution to the issue is an application
level ack.  Just such a mechanism is inherent in the MTP3
specification and has been proposed for M2UA.  An application level
ack is the only mechanism which wil allow an M2UA system to meet
Q.782 conformance tests (at MTP Level 3).

SCTP retrieval, on the other hand, is part of the current M2PA
draft.  Your concerns regarding messages which are unacknowledged at
the sender, but received and delivered to the receiving app, are an
issue in M2PA.  Using SCTP retrieval alone, again, would cause
failure of Q.782 test cases which require no message loss during
changeover.  (Discarding when communication exists between peers via
an alternative path is unacceptable.)

This situation with M2PA led me to ask whether an SCTP
implementation could be made to withold a Cumulative Ack, yet Sack
TSNs which are contiguous with the cummulative ack point.  Randall
fairly responded that this "trick" might not be treated well by some
applications.  It surely would not advance the goal of
interoperability.

In disussions with Tom George and others, I believe that it is clear
that M2PA's best way of resolving this issue is to provide an M2PA
application level ack outside of SCTP.  Just such an application
level ack has been proposed and I believe will be advanced into the
next M2PA draft.

Because an application level ack is required for both M2PA and M2UA,
which are the lower level UAs in the SIGTRAN SS7 suite, the only UAs
which might benefit from the SCTP retrieval are M3UA and SUA.

In the case of M3UA and SUA, Jacob's superlatives are correct.  When
M3UA fails over between redundant ASPs, it is far better for the
M3UA to discard messages than risk duplication.  To increase the
ability of M3UA to meet the stringent requirements set out in Q.706
for message loss, mis-sequencing and duplication, an application
level ack was proposed but has not been adopted.  M3UA does not
require SCTP retrieval, does not recommend SCTP retrieval, and AFAIK
does not even mention it.  This is largely, I believe, because the
responsibility of synchronization between ASPs during changeover is
placed on an unspecified ASP to ASP state synchronization mechanism.
Because of that, it really wouldn't matter then which messages are
sent in duplicate to an alternate ASP: it is the alternate ASPs
responsibility to perform the atomic state synchronization with its
peers and know which messages have been processed and which have
not, detect duplicates, and maintain synchronization.

What applies to M3UA also applies to SUA, in that redundant ASPs are
tasked with the same peer state synchronization.

Therefore, if one was to use the SCTP retrieval mechanism in M3UA or
SUA failovers to avoid message loss from SG to ASP, the retrieval
mechanism should provide the complete set of messages as you suggest
in (3); in other words, every message that the SCTP transmitter
still has at its disposal to retrieve.  This would allow an
implementation to minimize message loss in M3UA and SUA, with the
responsibility placed on the ASP to know which messages had been
processed by a peer.

A forward Correlation Id proposal has already be advanced to the
M3UA draft which will provide the ASPs with a mechanism for
identifying specific DATA messages between them and allow them to
perform this processing.

It is reasonable to assume that SUA will eventually take this approach
in support of the Broadcast mode for ASPs.

With all that said, I think it is a fair statement to warn the user
of the SCTP retrieval mechanism in the draft that some messages
retrieved may have already been accepted by the peer app.  It also
makes sense that, if retrieved messages are passed as ancillary data
in recvmsg, that the retrieved data be marked with the assigned SSN
(if any) and whether the DATA chunks was from your categories (1),
(2) and (3) with an attribute in the ancillary data.  Retrieval
without these attributes would surely only be tailored to one of
three classes of users, so I would advise against retreival using
read() or recvmsg() (returned as would be normal received data).

--Brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 22 Jun 2001 22:47:36 -0500

Kacheong,

> to tell how many messages are missing?

As per my previous comments on this thread, the ambiguity is
resolved at the ASPs in M3UA using ASP-to-ASP state synchronization.
It is resolved in M2UA and M2PA using the proposed application level
ack (which is also required for SS7 compliance for M2UA and M2PA).
The use of SCTP retrieval in the current M2PA draft has been shown
to be inadequate: expect M2PA-03 to provide a mechanism for an
application level ack in its stead.

The mechanism could still be useful to an M3UA or SUA implementation
which wants to reduce message loss due to failover in Override and
Loadshare modes as described in my other note.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/