Subject: HTTP Multipart Response, is this reasonable?

From: "J. David MacDonald" <[email-masked]>
Date: Mon, 20 Mar 2000 00:02:59 -0500

Can someone comment on the use of the following structure for EDI-INT usage.
Specifically, is the Multipart/mixed (see below) content-type permissible in
EDI-INT.

SCENARIO.

Client submits a transaction via HTTP.  Neither the EDI response transaction
nor the MDN response is available immediately (the inbound transaction from
the client is queued to a back-end system for processing.)

The back-end system decrypts/verifies the inbound transaction and generates
an MDN for it.  It also forwards the EDI content to an application subsystem
for processing.

The MDN and eventually the EDI response go out to a queue for pickup by the
client.

The client signs on an hour later to retrieve the results.  Rather than do
individual HTTP request/response pairs (one for each transaction sitting in
the queue), is it reasonable to do just a single HTTP request, and have the
server return as the HTTP response all transactions wrapped in a multipart
structure.

EDI-INT-AS2-06 alludes to this but comes up short on a specific example:
"In general, both HTTP servers and HTTP clients handling the message
templates of AS1 should be prepared to process these basic EDIINT data
formats when they are embedded within MIME multiparts".

EXAMPLE OF THE HTTP MULTIPART RESPONSE:
(1st part is the MDN)
(2nd part is the EDI)

Date: Mon, 20 Mar 2000 02:41:50 GMT
Server: IBM_HTTP_Server/1.3.6.2 Apache/1.3.7-dev (Win32)
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Connection: close
Mime-Version: 1.0
Content-Length: 10390
Content-Type: Multipart/mixed;
boundary="------------Boundary-EdiAndMdnTransactions"

--------------Boundary-EdiAndMdnTransactions
Mime-Version: 1.0
From: J. David MacDonald <[email-masked]>
Message-Id: <[email-masked]>
Date: Sun, 19 Mar 2000 21:41:38 -0500 (Eastern Standard Time)
Content-Type: Application/pkcs7-mime
Content-Length: 5194

[encrypted content here]
--------------Boundary-EdiAndMdnTransactions
Mime-Version: 1.0
From: J. David MacDonald <[email-masked]>
Date: Sun, 19 Mar 2000 21:41:35 -0500 (Eastern Standard Time)
Message-Id: <[email-masked]>
disposition-notification-options: signed-receipt-protocol=required,
pkcs7-signature; signed-receipt-micalg=required, sha1
disposition-notification-to: J. David MacDonald
<[email-masked]>
receipt-delivery-option: http://12345OntarioInc.com/UriForAsyncronousMdn
Content-Type: Application/pkcs7-mime
Content-Length: 4250

[encrypted content here]
--------------Boundary-EdiAndMdnTransactions--

Regards,
Dave MacDonald

E-mail: [email-masked]
Phone: (613) 825-6183

---

From: "Carl Hage" <[email-masked]>
Date: Mon, 20 Mar 2000 11:46:24 -0800

From:           	"J. David MacDonald" <[email-masked]>
Date sent:      	Mon, 20 Mar 2000 00:02:59 -0500

A response message and MDN can be combined, but not under this 
scenario. A client cannot "sign on" later for pickup [except as a POP3
client for SMTP-based delivery]. Transactions are only sent via push 
technology, i.e. either email to the original sender, or an HTTP post to 
the orignal sender's http server. (A sender cannot poll the original 
recipient's http server.)

Normally, the MDN response is generated immediately. (Like a P.O. 
sent by FedEx or certified mail, where a reciept is signed at delivery, 
not when processed.) 

A deferred receipt can be prearranged using the
"Receipt-delivery-option" header, sent by the client.

Normally, a multipart EDI message and MDN is to support real-time 
EDI systems which can generate a response transaction immediately 
(within a few seconds).

> SCENARIO.
> 
> Client submits a transaction via HTTP.  Neither the EDI response
> transaction nor the MDN response is available immediately
...
> The client signs on an hour later to retrieve the results.Rather than 
do individual HTTP request/response pairs (one for each transaction 
sitting in the queue), is it reasonable to do just a single HTTP request, 
and have the server return as the HTTP response all transactions 
wrapped in a multipart structure. 

--------------------------------------------------------------------------
Carl Hage                                              C. Hage Associates
<mailto:[email-masked]> Voice/Fax: 1-408-244-8410      1180 Reed Ave #51
<http://www.chage.com/chage/>                          Sunnyvale, CA 94086

---

From: "Gregory W. Vesper" <[email-masked]>
Date: Mon, 20 Mar 2000 14:50:28 -0700

Please bear in mind, customers who are using HTTP for large files, say in
excess of 10MB, will require asynchronous transfer of MDNs as opposed to
MDNs on the same connection.  Async MDNs allow you to support full EDIINT
across any transport.  We currently support EDIINT capability over: SMTP,
FTP, HTTP, HTTPS, MQSeries.

There's no reason MDNs should be hard wired to a specific transport.  I
agree on support for synchronous MDNs for HTTP(S), so long as async is an
option..
Cheers,
-Greg.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Carl Hage
> Sent: Monday, March 20, 2000 12:46 PM
> To: J. David MacDonald
> Cc: [email-masked]
> Subject: Re: HTTP Multipart Response, is this reasonable?
>
>

---

From: "Carl Hage" <[email-masked]>
Date: Mon, 20 Mar 2000 15:10:34 -0800

From:           	"Gregory W. Vesper" <[email-masked]>
Date sent:      	Mon, 20 Mar 2000 14:50:28 -0700

> MDNs on the same connection. 

Why would the message size affect use of async vs sync (in the http 
post reply) MDNs?

(I assume the MDN wouldn't include a copy of the original-- just the 
MIC)

It seems to me that separate transmission of an MDN would be mainly 
needed if a forwarding gateway was involved, and final delivery can't be 
confirmed until the forwarding is completed.

Note the way I read the spec, the choice of reply method is determined 
by the sender not the recipient. Did I miss something, or would there 
be a problem using HTTP if the sender requested an MDN using the 
'default' method, but the http is a forwarding agent and can't issue 
immediate replies?
--------------------------------------------------------------------------
Carl Hage                                              C. Hage Associates
<mailto:[email-masked]> Voice/Fax: 1-408-244-8410      1180 Reed Ave #51
<http://www.chage.com/chage/>                          Sunnyvale, CA 94086

---

From: "Gregory W. Vesper" <[email-masked]>
Date: Mon, 20 Mar 2000 16:32:41 -0700

> Why would the message size affect use of async vs sync (in the http

For a 10+MB file, by the time you decrypt, validate signatures, and generate
the MDN, you may have timed out your HTTP connection.

> (I assume the MDN wouldn't include a copy of the original-- just the

The MDN would be the the same as AS1 MDNs.

> confirmed until the forwarding is completed.

That's another potential scenario yes, but I think the response timeout
scenario will be more common.

> 'default' method, but the http is a forwarding agent and can't issue

The MDN reply method is determined by the sender - I suppose this could be
made a suggestion rather than a mandate with a fallback to a default
behaviour.  Inasmuch as current EDIINT MDNs are async over SMTP, this might
be a nice default.

Cheers,
-Greg.

---

From: "J. David MacDonald" <[email-masked]>
Date: Tue, 21 Mar 2000 21:52:45 -0500

Perhaps I phrased the scenario incorrectly and made it too complex for what
I'm trying to determine.
Lets set aside the MDN issue for now.

MODIFIED SCENARIO.

Client submits a 2 EDI transactions via HTTP.  The EDI response is not
available immediately.

Client submits another transaction an hour later (I originally referred to
this as a sign-on ie. the client establishes an HTTP sesson, POSTs an
EDI-INT transaction, waits for a response...).  By this time, the EDI
responses from the 1st transmission are available and are sent back in the
HTTP response in a multipart format.  Acceptable?

> scenario.
Under what scenario is this permissible: only real-time?

> A client cannot "sign on" later for pickup [except as a POP3
> the original sender's http server. (A sender cannot poll the original

This would be an unnecessary burder to smaller clients who wish to
communicate via HTTP over dial-up. It  assumes that all clients set up an
HTTP server infrastructure, dedicated connections, etc.

EDI-INT is supposed to be protocol-neutral (as much as possible).  The
statement infers that there is one set of rules/techniques for SMTP and
another for HTTP.  Was this intended?  Shouldn't a HTTP Request/Response
session be similar to an SMTP/POP3 Send/Receive session to achieve that
neutrality?

After a re-read of the specification, I cannot locate any references that
state that if I want to return an EDI response to the client via HTTP, then
I must POST it the client's server.  I do see many references (albeit, most
of them real-time oriented) where it does seem to be acceptable to return an
EDI response as a part of the HTTP response.  Does this mean: one set of
rules for real-time and yet another (or worse, set up a separate technology
infrastructure to implement another protocol) for deferred processing?

Dave

---

From: "Carl Hage" <[email-masked]>
Date: Wed, 22 Mar 2000 11:54:26 -0800

From:  "J. David MacDonald" <[email-masked]>
Date sent:      	Tue, 21 Mar 2000 21:52:45 -0500

> HTTP response in a multipart format.  Acceptable?

Normally, a transaction is sent immediately when generated, not 
queued for possible attachment to some other response. (The spec 
doesn't prohibit this, but it would make no sense.)

Trading partners using the HTTP method must both have http servers, 
not just clients. Transactions are always pushed from the sender to the 
recipient's http server, with the exception that an immediate reply could 
be returned as a response to the post operation.

I suppose it would be possible for one trading partner to use http and 
one to use smtp. (It doesn't make much sense, though since it's 
simpler to just support SMTP-only.)

The sender of an http posted transaction specifies how the MDN is 
returned, one of 3 ways:
1. In the http post response
2. Deferred, via http
3. Deferred, via SMTP

The recipient's http server must support all methods unless there is a 
specific agreement on a subset.

> Under what scenario is this permissible: only real-time?

A deferred MDN could be combined with a response EDI transaction 
posted to the sender's http server, provided the sender specified the 
MDN to be returned separately to the http URL, same as the URL to 
receive transactions.

The real-time scenario is also possible.
...
> HTTP server infrastructure, dedicated connections, etc.

Clients with dialup should use the SMTP method. SMTP (ASN1) 
supports real-time direct EDI almost equivalent to HTTP as well as 
supporting store-and-forward messaging.
 
> session be similar to an SMTP/POP3 Send/Receive session to achieve that

There is no HTTP/POP3, unless a special gateway is implemented. If it 
were truely comparable, then the MDN would always be sent in a 
separate message to the senders http server. However, the http post 
requires a reply, and in most cases it's most efficient to return the 
reciept in the reponse. Allowing deferred receipts actually makes the 
HTTP method much more complex.

The SMTP method was used in EDIINT AS1 because it is universal. 
My own personal opinion is that an http method is unnecessary and 
undesirable-- but lots of people disagree with that (no need to argue).

[Note that in the case of POP3, the recipient's server (ISP) does the 
queuing, whereas in the case of polled http, it's the sender's system 
that would be doing the queuing. The EDIINT specs don't use a 
queuing except as part of retry of a failed transmission.]

> state that if I want to return an EDI response to the client via HTTP,
> then I must POST it the client's server.

There is no explicit prohibition, however, the recipient MAY post a 
transaction so the sender MUST have an http server. [Unless the weird 
http/smtp combo is used.]

  I do see many references
> (albeit, most of them real-time oriented) where it does seem to be
> acceptable to return an EDI response as a part of the HTTP response.  Does
> this mean: one set of rules for real-time and yet another (or worse, set
> up a separate technology infrastructure to implement another protocol) for
> deferred processing?

Actually, this is the case for http. The normal case is to have 
transactions immediately confirmed with a receipt. A full 
implementation would require the response method to be checked and 
then all three ways would need to be supported, with slightly different 
infrastructure.

It might be the case that a trading partner agreement would prohibit 
certain cases in order to simplify the software.