Subject: Re: [Sip] Comments on draft-ietf-sip-body-handling-02

From: Eric Burger <[email-masked]>
Date: Wed, 1 Oct 2008 08:40:11 -0400

Agreed. And, echoing what Keith said, what Dale has done is prove  
(from a Computer Science perspective) that complete evaluation of  
unfettered MIME is *possible*. It is up to the implementor to figure  
out whether they want to take short-cuts (and risk barfing on  
legitimate messages) or do the whole thing.

The point is doing the Full Monte *is* practical on small, memory and  
CPU constrained devices. So the argument "We can't do the real thing  
because it hurts" is provably not legitimate.

On Sep 29, 2008, at 11:09 PM, [email-masked] wrote:

>   From: Paul Kyzivat <[email-masked]>
>
>   I agree it would be straightforward to define a rule that says one  
> must
>   recursively unwrap all multipart/mixed bodies in order to find other
>   parts to process. But we ought to think if we really want to do  
> that.
>
> What alternative rule do you propose as the meaning of
> multipart/mixed?
>
>   Potentially this could introduce a lot of extra processing.
>
> That's sort of like saying that "x = 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0"
> involves a lot of extra processing.  Yes, it does.  But it meaning is
> clear, once we define what "+" means.  We could try to outlaw that
> expression and "expressions like it", but there's no sensible boundary
> to enforce -- Do we ban expressions with more than five addends?  How
> about more than one addend which is "0"?  Any possible solution to
> this problem is worse than the problem itself.
>
> What I am advocating is that we explicitly abandon the attempt to
> define only the situations that intuitively make sense, and instead
> define a set of simple, well-defined rules that make clear what is
> valid and what is not, and what the valid cases mean.  Just like we've
> done with arithmetic expressions.  Otherwise we'll never escape the
> swamp of trying to discern which ones make intuitive sense.
>
>   From: Paul Kyzivat <[email-masked]>
>
>> RFC 3459 uses multiparts in a somewhat different way that SIP does.
>> In SIP, the I-D suggests that components of a multipart/alternative
>> should have handling=optional and that the processor should ignore  
>> the
>> handling parameter:
>>
>> 	The UA SHOULD also set the 'handling' parameter of all the
>> 	body part within the 'multipart/alternative' to 'optional'
>> 	(the receiver will process the body parts based on the
>> 	handling parameter of the 'multipart/alternative' body; the
>> 	receiver will ignore the handling parameters of the body
>> 	parts).
>>
>> 	The receiver SHOULD ignore the handling parameters of the body
>> 	parts within the 'multipart/ alternative'.
>
>   Right. But the SHOULD without qualification suggests that somebody  
> might
>   do otherwise, and we don't know under what circumstances that might
>   happen. So if somebody sets all the parts to handling=required, it  
> seems
>   that the handling of the unselected parts will still be ignored.
>
> Yes, I see the problem there.  I'd replace that last SHOULD with MUST.
> Actually, I argue that MUST is implied, as the RFCs define how
> multipart/alternative is to be processed, and that algorithm does not
> look at the handling parameters of components of
> multipart/alternatives.
>
>   Suppose I want to send a MESSAGE, and I require that some form of  
> the
>   body be processed. I will offer it in two forms: text/enhanced and
>   text/html. How do I encode that?
>
> 	   C-T: multipart/alternative
> 	   C-D: render;handling=required
> 		   C-T: text/enhanced
> 		   C-D: render;handling=optional
> 		   ===
> 		   C-T: text/html
> 		   C-D: render;handling=optional
>   or
> 	   C-T: multipart/alternative
> 	   C-D: render;handling=required
> 		   C-T: text/enhanced
> 		   C-D: render;handling=required
> 		   ===
> 		   C-T: text/html
> 		   C-D: render;handling=required
>
> C-T: multipart/alternative
> C-D: render;handling=required
> 	C-T: text/enhanced
> 	===
> 	C-T: text/html
>
> 'render' is inherited from the parent, and the handling of the
> components is ignored.  We could also take it that MESSAGE implies
> that the body is 'required', so the handling specification of the
> parent is also optional.
>
>   And if I don't support either of those text types, what error do I
>   return? 415? In the first case it seems a little odd to return an  
> error
>   if I selected a part that is optional.
>
> 415.  The I-D is clear that 415 is how you respond to messages which
> are not processable due to a failure of these MIME tests.
>
>   And if I didn't require either part to be understood, would I just
>   change to handling=optional for the multipart/alternative itself?
>
> Intuitively, I'd say that MESSAGE demands that the entire body be
> processable, so handling=required is implicit for the entire body.
> But assuming that we did not make such a deduction from the SIP
> context of the message, if the body had handling=optional, it would
> have the effect that the request could succeed even if the requester
> did not understand the body.  (Hmmm...  That's another clause of the
> algorithm I overlooked.)
>
> (Note that there are cases where understanding the body is optional.
> E.g., a sipfrag of the request in a failure response.)
>
> Dale
> _______________________________________________
> Sip mailing list  https://www.ietf.org/mailman/listinfo/sip
> This list is for NEW development of the core SIP Protocol
> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments on the application of sip

_______________________________________________
Sip mailing list  https://www.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 01 Oct 2008 12:51:16 -0400

Ok, ok, UNCLE!!!

I'd still like to know if email clients need to do this two pass 
approach - first searching for all the references to determine if all 
the handling requirements can be met, and then actually doing the 
processing of the referenced parts?

Also, it *is* legal for proxies to process bodies. But its obviously not 
required (by the protocol) for them to do so. Do we have a clue how the 
handling parameter applies to proxies? I assume in effect they can just 
ignore it and consider handling to be optional for all parts. Also, I 
presume a proxy that cares about body parts might want to investigate 
all the alternatives of a multipart/alternative, since it can't know 
which one the recipient will choose.

	Thanks,
	Paul

Eric Burger wrote:
> Agreed. And, echoing what Keith said, what Dale has done is prove (from 
> a Computer Science perspective) that complete evaluation of unfettered 
> MIME is *possible*. It is up to the implementor to figure out whether 
> they want to take short-cuts (and risk barfing on legitimate messages) 
> or do the whole thing.
> 
>>   I agree it would be straightforward to define a rule that says one must
>>   parts to process. But we ought to think if we really want to do that.
>>
>>> should have handling=optional and that the processor should ignore the
>>> handling parameter:
>>>
>>   Suppose I want to send a MESSAGE, and I require that some form of the
>>   return? 415? In the first case it seems a little odd to return an error
> ------------------------------------------------------------------------
>

---

From: Eric Burger <[email-masked]>
Date: Fri, 3 Oct 2008 14:20:11 -0400

I'm trolling my e-mail contacts (pardon the pun) on how they do MIME  
decoding.

On the second paragraph: if you are a proxy that is actually doing  
something with the body, *and you are going to hack the body*, then I  
would offer you've created something that most likely will be some  
kind of B2BUA.

In the e-mail world, the ;handling property was created for gateways  
(B2BUA's), so the gateway could know whether dropping a body part,  
because the connected network/protocol could not handle that body  
part, would render the message useless. For example, if I am using an  
old version of Outlook, and I send you a Word file, the Word body part  
is required for the relay, but the application/tnef (proprietary  
Outlook stuff) is optional and the gateway can silently drop the part.

On Oct 1, 2008, at 12:51 PM, Paul Kyzivat wrote:

> approach - first searching for all the references to determine if  
> all the handling requirements can be met, and then actually doing  
> the processing of the referenced parts?
>
> Also, it *is* legal for proxies to process bodies. But its obviously  
> not required (by the protocol) for them to do so. Do we have a clue  
> how the handling parameter applies to proxies? I assume in effect  
> they can just ignore it and consider handling to be optional for all  
> parts. Also, I presume a proxy that cares about body parts might  
> want to investigate all the alternatives of a multipart/alternative,  
> since it can't know which one the recipient will choose.
>
> 	Thanks,
> 	Paul
>
> Eric Burger wrote:
>> unfettered MIME is *possible*. It is up to the implementor to  
>> figure out whether they want to take short-cuts (and risk barfing  
>> on legitimate messages) or do the whole thing.
>> The point is doing the Full Monte *is* practical on small, memory  
>> and CPU constrained devices. So the argument "We can't do the real  
>> thing because it hurts" is provably not legitimate.
>>>  Right. But the SHOULD without qualification suggests that  
>>> somebody might
>>>  happen. So if somebody sets all the parts to handling=required,  
>>> it seems

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 03 Oct 2008 14:42:59 -0400

Eric Burger wrote:
> would offer you've created something that most likely will be some kind 
> of B2BUA.

Thats a different case. A B2BUA *is* a UA, so I suppose these rules 
about handling apply to it. (But I know it will get more confusing when 
considering B2BUAs that are more "proxy-like" in their handling of the 
body - just passing it through and perhaps peeking into it.

I was concerned with true proxies. E.g. I know there are proxies that 
look for SDP bodies and use them as input to policy decisions for 
control of gates in the media path. Similarly, I can imagine a proxy 
that scans bodies of MESSAGE requests, recording them and perhaps 
alerting when certain keywords are noticed.

I'm not certain what relevance the handling parameter has to such a 
proxy. Perhaps none.

> because the connected network/protocol could not handle that body part, 
> would render the message useless. For example, if I am using an old 
> version of Outlook, and I send you a Word file, the Word body part is 
> required for the relay, but the application/tnef (proprietary Outlook 
> stuff) is optional and the gateway can silently drop the part.

That is quite different than what we are setting up here. We are in the 
process of saying that if there is some part with handling=required, and 
you don't know how to process it, then you should fail the request, 
before you have processed any of it.

That provides nice, clear, unambiguous semantics. But it may not be 
realistic. I don't have the sense that is how mail works. But then mail 
is different because there is typically no error message to the sender.

Its unclear to me whether there is much that can be shared of mime 
processing between a sip UA and a mail client/UI. Maybe not much, so 
that differences in processing logic are ok. But maybe some significant 
part can be shared, especially when exchanging info that is to be 
"rendered", such as with MESSAGE. If mime libraries can be shared, there 
is reason to keep the processing semantics the same.

	Thanks,
	Paul

>> how the handling parameter applies to proxies? I assume in effect they 
>> can just ignore it and consider handling to be optional for all parts. 
>> Also, I presume a proxy that cares about body parts might want to 
>> investigate all the alternatives of a multipart/alternative, since it 
>> can't know which one the recipient will choose.
>>
>>     Thanks,
>>     Paul
>>
>> Eric Burger wrote:

---

From: [email-masked]
Date: Sun, 5 Oct 2008 22:07:09 -0400 (EDT)

From: Eric Burger <[email-masked]>

Though the real limitation to "doing the real thing" is programming
effort (or more exactly, finding good enough programming talent) than
the CPU/memory costs in the end-systems.  The downside of Moore's Law
is that it becomes increasingly uneconomical to pay a programmer to
spend the extra effort to write efficient code.

I once worked for a company that crammed a fully RFC 2616-compliant
HTTP server into a Z80.  Now embedded systems will give you many
megabytes without blinking.

Dale

---

From: [email-masked]
Date: Sun, 5 Oct 2008 22:08:12 -0400 (EDT)

Well, if you want to be able to process all valid MIME multipart
bodies, it seems to be unavoidable.  As far as I can tell, almost all
of the algorithm is prescribed by the RFCs.  The only specification
I've added is the subtle question of what strategy will be used to
choose from multipart/alternatives.  My first discussion some months
ago, I used the strategy "Can you choose so that the constraints are
satisfied?", which led to NP-completeness.  This version, which is
probably closer to what is intended by the RFCs, is "Make the choice
for each multipart/alternative without regard to whether it is adverse
to multiparts that contain you."

As far as I can tell, proxies aren't allowed to reject messages
because they don't like their bodies, although I'm sure that in
specialized applications, they will.  But the interpretation of
handling parameters by such proxies will be determined by what the
"specialized application" is attempting to do; it's not in the scope
of RFCs.

Dale

---

From: [email-masked]
Date: Mon, 6 Oct 2008 22:42:07 -0400 (EDT)

Ah!  This is an interesting difference, as the concept is that the
handling property is "global", that is, if a part labeled "required"
is necessary for successful processing of the entire message.  (As
opposed to the "local" concept, where it is only neccessary for the
successful processing of the containing multipart, whose processing
might not be necessary for processing the entire message.)

Of course, with one-level nesting, the two are equivalent.

Dale