Subject: Handling Duplicate Messages

From: Masaaki Hiroya <[email-masked]>
Date: Thu, 22 Jul 1999 13:50:27 +0900

David

The IOTP spec V1.0-04 says as follows:

>   4.5.2.2 Checking/Handling Duplicate Messages
> 
>   If the input message can be identified as potentially a valid input
>   message then check to see if an "identical" input message has been
>   received before. Identical means that all blocks, components,
>   elements, attribute values and element content in the input message
>   are the same.
> 
>   If an identical message has been received before then check to see if
>   the processing of the previous message has completed.
> 
>   If processing has not completed then do nothing as the processing of
>   the previous message will result in a response if required.

I think "If processing has not completed then do nothing" might cause 
some problems.

If timemout occurs in the IOTP client while it is waiting for a response
messege from the server, the session (connnection) is usually disconnected.
In this case, the client cannot receive the response message for the 
original request message.

It the server takes much time to handle the original request message,
the IOTP client will receive an empty http message from the server
when it retransmits the request message due to the timeout event
since the server does nothing during processing the original message.

I think it's better to send a transient error message from the server
to the client for retry after an appropriate period of time
if processing the original request message has not completed.

Besides, it should also clearly be defined that the IOTP client 
should disconnect the previous session before retransmission.
in order to avoid the duplicate messages received by the client.

Any comment?

Masaaki

-----
Masaaki Hiroya
Systems Development Laboratory
Hitachi, Ltd.
email: [email-masked]
tel: +81-44-549-1531
fax: +81-44-549-1640
-----------------------------------------------------------------------
Message addressed to: [email-masked]
Archive available at: http://www.elistx.com/archives/ietf-trade/
To (un)subscribe send a message with "subscribe" or "unsubscribe" in the
body to: [email-masked]

---

From: "Smith, Chris " <[email-masked]>
Date: Thu, 22 Jul 1999 14:33:42 -0400

The difficulty with this approach is that it assumes that a session exists.

If IOTP is using HTTP or HTTPS as a transport layer, then this is certainly
true. However, it is quite possible that a different transport layer may not
have sessions. In such a case, the client cannot ensure that the first reply
will not be received.

Initially, it would seem that the more sensible (but difficult) approach
would be to have the duplicate message wait on the completion of the
original and then pick up the results and return them. However, if the
transport is sessionless, the client will get both replies, and they should
be identical replies. However, these two identical replies should both be
processed (one as an original, one as a duplicate), perhaps resulting in two
continuing threads of control in some clients. Clearly this approach is not
wise, although it should not cause problems if clients retain a tight
context around transactions in progress.

In fact, this problem is always possible if the sessionless reply and the
duplicate request pass each other in transit.

The above solution also has the annoying difficulty that the system stores
up all the duplicates received, and then releases all the replies at once to
a client who has sent in multiple requests. Remember, the client does not
know the request is in progress - they only know that no reply has been
received.

Perhaps the preferred solution would be to indicate exactly what condition
actually exists. Can we specify that a status is sent back, indicating that
the request is still being processed? Such a status prevents the double
reply, while indicating to a client that (if they are certain of no more
replies) they should pause and then resubmit the duplicate in order to
retrieve the reply after processing is complete (if it isn't complete, they
get another status = in process).

...chris

------------------------------------------------------------
 Chris Smith                                +1.416.348.6090
 Royal Bank                       [email-masked]

> -----Original Message-----
> From: Masaaki Hiroya [mailto:[email-masked]]
> Sent: July 22, 1999 12:50 AM
> To: [email-masked]; [email-masked]
> Subject: Handling Duplicate Messages
> 
> 
> 
> David
>

---

From: Masaaki Hiroya <[email-masked]>
Date: Mon, 26 Jul 1999 20:42:23 +0900

> Perhaps the preferred solution would be to indicate exactly what 
> condition actually exists. Can we specify that a status is sent back, 
> indicating that the request is still being processed? Such a status 
> prevents the double reply, while indicating to a client that (if they
> are certain of no more replies) they should pause and then resubmit
> the duplicate in order to retrieve the reply after processing is 
> complete (if it isn't complete, they get another status = in process).

Why do you prefer to send back the status indicating request is in 
progress rather than to send back a transient error indicating request
is in progress?

In order to send back the status indicating request is in progress, 
it is necessary to define a new block to contain the status or 
change the PayExchBlk.  Furthermore, a new status type should be
added such as "MsgExch" status since the payment status is always 
in progress during payment document exchange (until sending back PayRespBlk).

If we use a transient error message, all we have to do is to add a new
error code such as "ReqInProgress".

Is there any problem if the server sends back a transient error message
when request is in progress?

Masaaki

-----
Masaaki Hiroya

---

From: "Smith, Chris " <[email-masked]>
Date: Mon, 26 Jul 1999 10:07:38 -0400

> error code such as "ReqInProgress".

Sorry if I wasn't clear - I wanted a status sent back,
not a Status. Adding the code would resolve the concern.

I did not want the client to receive the same response
for two different situations. One is that there is no
response in progress, and there is a transient error.
The other (this case) is that there is a response in
progress.

We should provide some indication that the previous
request (the abandoned one) was actually received to
prevent the client timing out while in a condition of
assuming the request was never received.

Incidentally, this also provides a potentially valuable 
response for transport layers that must reply, and want 
to state something useful that will keep the client working
on the transaction. Under such circumstances, it would be 
possible to reply with TransientError/ReqInProgress to the
original request. The client would then be aware that the 
work is in progress, and will come back periodically to get 
the response.

(An example of this might be an e-mail based connection, 
where the server side has determined that manual intervention 
is required. The server can indicate ReqInProgress, and then 
set MinRetrySecs to a very high number of seconds (perhaps 
86400 seconds) indicating that the client can come back for 
the reply tomorrow.)

---

From: [email-masked]
Date: Wed, 28 Jul 1999 03:40:21 +0100

Chris/Masaaki
 
Thanks for thinking this through. Propose the following changes to add a new
type of Transient Error as Masaaki suggested. Details below.
 
David
 
Added new Error Code to indicate that "Message is being processed" called
"MsgBeingProc". "ReqInProgress" which Masaaki thought of seems to suggest
that it can only apply to a request when in fact it could also work with an
exchange message such as a Payment Exchange. Its description is ...

"Message being processed. This error code is only used with a Severity of
Transient Error. It indicates that the previous message, which may be an
exchange message or a request message, is being processed and, if no
response is received by the time indicated by the MinRetrySecs attribute,
then the original message should be resent."

I also noticed that there isn't an error code to indicate that a "Server (or
other system) Is Busy" so I've added another code called SystemBusy with a
description of ...

"SystemBusy. This error code is only used with a Severity of Transient
Error. It indicates that the system that received a message is currently too
busy to handle the message. If no response is received by the time indicated
by the MinRetrySecs attribute, then the original message should be resent.
This code should be used when to indicate that a server (e.g. HTTP) 

[Note] If the server/system handling the Transport Mechanism (e.g. HTTP) is
busy then a Transport Specific error message should be used instead of an
IOTP Error message. This code should be used in association with IOTP
servers/systems or other servers/systems to which the IOTP server is
connected.
[Note End]"

Added to section 4.5.2.2 Checking/Handling Duplicate Messages ...

	"... If processing has not completed then generate an Error
Component with a Severity of Transient Error and an Error Code of to
indicate the message is being processed and send it back to the sender of
the Input Message requesting that the original message be resent after an
appropriate period of time."

... and clarified section 4.5.2.3 Processing Non-Duplicate Message (CHECKING
A SERVER IS AVAILABLE) to say ...

	"... If the server is too busy, then it should generate an Error
Component with a Severity of Transient Error and an Error Code of SystemBusy
and send it back to the sender of the Input Message requesting that the
original message be resent after an appropriate period of time."

... and in PROCESS THE ERROR FREE INPUT MESSAGE changed ... 

*	if the Input message contains an Error Block with a Transient Error
then wait for the required time then resend the previous message

... to ...

*	if the Input message contains an Error Block with a Transient Error
then wait for the required time then resend the previous message, if a
response to the earlier message has not been received

... Also updated ServerLogic.txt and ConsumerLogic.txt

  -----Original Message-----
From: Smith, Chris [mailto:[email-masked]]
Sent: Monday, July 26, 1999 7:08 AM
To: 'Masaaki Hiroya'; Smith, Chris ; [email-masked];
[email-masked]
Subject: RE: Handling Duplicate Messages

**********************************************************************************************

This Email and any attached files are confidential and may also be privileged. 
If you are not the intended recipient, please notify the postmaster using email 
address [email-masked] or call +44 171 557 5000 and ask for the 
IT Helpdesk.  You should not copy this email and any attached files, use them 
for any purpose or disclose the contents to any other person; all copies of the 
Email and associated files in your possession should be destroyed.

Mondex International Limited
47-53 Cannon Street
London EC4M 5SQ
United Kingdom
Registered No: 3122085, England

Phone:          +44 171 557 5000
Fax:            +44 171 557 5200
Email:          [email-masked]
WebSite:        http://www.mondexinternational.com

*********************************************************************************************

---

From: Masaaki Hiroya <[email-masked]>
Date: Thu, 29 Jul 1999 22:27:22 +0900

David

I agree with you.
I think "MsgBeingProc" is better, too.
Besides, I agree to using ErrorCode="SystemBusy" when the server is busy.

Masaaki

-----
Masaaki Hiroya