Subject: [Uri-review] Request for review

From: Andrey Shur <[email-masked]>
Date: Thu, 11 May 2006 14:51:04 -0700

This mail is sending to request the submission of the "pack" Uri scheme registration template (attached) for review.

Regards
- Andrey Shur

_______________________________________________
Uri-review mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/uri-review

---

From: "Mark Baker" <[email-masked]>
Date: Sat, 20 May 2006 16:40:12 +0200

Andrew,

When I try to learn more about "open packaging", I'm confronted with a
request to accept a license that I'm hesitant to accept.  Therefore, I
have not read that document.

But if I assume it's just some kind of packaging format, like
multipart/zip/etc.., I have a serious concern about the need for this
new URI scheme.

If I have a package identified by this URI;

http://www.mysite.com/my.package

Then why can't I identify the "a/b/foo.xml" part contained therein, by this URI?

http://www.mysite.com/my.package/a/b/foo.xml

It seems to me to meet the requirements that are explicitly mentioned
in your registration template. e.g. being able to use the URI of the
package as a base URI for parts.

There's other concerns too, about the encoding a URI inside the
authority component of another URI, but the concern described above
trumps it so I'll refrain from getting into that for now.

Cheers,

BTW, thanks for following the procedure described in RFC 4395.  I
think this is our first registration request under the new guidelines,
so it's nice to see it in good shape.

Mark.

On 5/11/06, Andrey Shur <[email-masked]> wrote:
>
>
>
>
> This mail is sending to request the submission of the "pack" Uri scheme
> registration template (attached) for review.
>
>
>
> Regards
>
> - Andrey Shur
>
>

---

From: "Daniel R. Tobias" <[email-masked]>
Date: Sat, 20 May 2006 10:46:44 -0400

On 11 May 2006 at 14:51, Andrey Shur wrote:
[long line re-wrapped in accordance with RFC 2822]
> This mail is sending to request the submission of the "pack" Uri
> scheme registration template (attached) for review. 

Was there any reason to put this plain-ASCII template in a 
proprietary-format, 50 kilobyte, MS Word attachment instead of as 
plain text?  Was there any reason to send the single-line cover 
letter as a multipart MIME message with a heap of grotesque MS Office 
HTML code (when, in fact, no special formatting was done to the 
message)?  Micro$oft strikes again... I wouldn't trust that company's 
"engineers" to have any part in the Internet standards process.

-- 
== Dan ==
Dan's Mail Format Site: http://mailformat.dan.info/
Dan's Web Tips: http://webtips.dan.info/
Dan's Domain Site: http://domains.dan.info/

---

From: "Gregg Brown" <[email-masked]>
Date: Sat, 20 May 2006 09:46:15 -0700

Mark,

I'll let Andrey answer the technical concerns, but I would like you to
be able to view the specification. The license is just a review license,
and is intended support letting you, and others like you look at the
spec. Please have a quick look at the license text and feel free to ask
any specific questions.

Gregg Brown  |  Lead Program Manager  |  Digital Documents  |  (206) 295
0592 (cell)  |  (425) 707 5892 (desk) 
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Baker
Sent: Saturday, May 20, 2006 7:40 AM
To: Andrey Shur
Cc: [email-masked]; Ted Hardie; Jerry Dunietz; Gregg Brown; John
Calhoon (LCA)
Subject: Re: [Uri-review] Request for review

Andrew,

Then why can't I identify the "a/b/foo.xml" part contained therein, by
this URI?

Cheers,

Mark.

> This mail is sending to request the submission of the "pack" Uri
scheme

---

From: "Mark Baker" <[email-masked]>
Date: Sat, 20 May 2006 21:34:35 +0200

Andrey,

First, my sincere apologies for botching your name earlier.  I didn't
look closely enough, obviously ...

On 5/20/06, Andrey Shur <[email-masked]> wrote:
> Mark,
>
> Thank you for the question. The primary reason why we did not use the
> approach you describe is the requirement for relative references within a
> package parts to refer to parts within the same package.
> Our registration template says:
>
>    The purposes of the "pack" URI scheme are:
>
>    1. To identify a part resource within a package that conforms to
>       Open Packaging Conventions [4].
>    2. To enable the use of a part's URI as a base URI for resolving
>       relative references to parts within the same package.
>
> Let's assume that in your example part identified by the Uri
> http://www.mysite.com/my.package/a/b/foo.xaml
> holds relative reference /bar.xml.
> Being resolved against the base Uri of the part it gives us
> http://www.mysite.com/bar.xml which is outside of the package.

Ok, thanks.  But why do you need to refer to a part using "/bar.xml"?
What purpose does that serve?  Why does "bar.xml" not suffice?

Perhaps the packaging spec could shed some light on this, and I'm sure
you're just doing what your lawyers suggested by using a license, but
I'm not about to accept its terms without running it by my employer's
lawyers, lest I get my employer in trouble by accepting (even though
I'm not representing them here).  And I'm not sure about the IETF
rules about this stuff, but I believe it to be (at least) considered
bad form.

Mark.

---

From: Andrey Shur <[email-masked]>
Date: Sat, 20 May 2006 10:51:38 -0700

Mark,

Thank you for the question. The primary reason why we did not use the approach you describe is the requirement for relative references within a package parts to refer to parts within the same package.
Our registration template says:

Let's assume that in your example part identified by the Uri http://www.mysite.com/my.package/a/b/foo.xaml
holds relative reference /bar.xml.
Being resolved against the base Uri of the part it gives us http://www.mysite.com/bar.xml which is outside of the package.

Note that "pack:" Uri scheme prevents from such cases, unless the relative reference in a part is in the rarely-used network-path reference form.

Of course, users may need to resolve relative references in the package parts to resources outside of the package. For these cases the explicit definition of an appropriate base Uri is required. You can find the example on that in section 1.3.3 of the "Open Packaging Conventions" 0.85, if you choose to accept the license look through the document.

Thank you
- Andrey

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Baker
Sent: Saturday, May 20, 2006 7:40 AM
To: Andrey Shur
Cc: [email-masked]; Ted Hardie; Jerry Dunietz; Gregg Brown; John Calhoon (LCA)
Subject: Re: [Uri-review] Request for review

Andrew,

Cheers,

Mark.

---

From: Andrey Shur <[email-masked]>
Date: Sat, 20 May 2006 13:20:25 -0700

Mark,

you ask why do we need to refer to a part using /bar.xml?

We don't need to use exactly this form of relative reference to refer to a part. Our goal was to guarantee that any relative reference in a package part refers to a part within the same package (unless, as I mentioned before, relative reference is in the network-path form, or base Uri for the reference is explicitly defined

in a way that leads to opposite). Having said this, in the example below "/bar.xml" can be replaced by e.g. "../../../bar.xml" with the same reasoning.

As for your concerns about the <authority> component of the "pack:" Uri being the encoded Uri of the package, it is indeed an unusual (although we are sure - consistent) approach. I would expect the serious discussion on that in the context of registration procedure. To begin with let me say that the grammar for pack: <authority> component completely fits the RFC 3986.

Thanks

- Andrey

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Baker
Sent: Saturday, May 20, 2006 12:35 PM
To: Andrey Shur
Cc: [email-masked]; Ted Hardie; Jerry Dunietz; Gregg Brown; John Calhoon (LCA)
Subject: Re: [Uri-review] Request for review

Andrey,

bad form.

Mark.

---

From: "Mark Baker" <[email-masked]>
Date: Mon, 22 May 2006 22:13:05 +0200

Andrey,

On 5/20/06, Andrey Shur <[email-masked]> wrote:
> We don't need to use exactly this form of relative reference to refer to a
> part. Our goal was to guarantee that any relative reference in a package
> part refers to a part within the same package

Ok, thanks.  What purpose does this goal serve?

Mark.

---

From: Andrey Shur <[email-masked]>
Date: Mon, 22 May 2006 19:08:00 -0700

Mark,

Thank you for the question. I can see two parts of it that should be answered:

First, why do we consider "any relative reference"? Actually we expect users of the packaging technology to store manifold content in the package parts. Certain kinds of the content may hold relative references. We don't want to limit our customers to a particular form of relative references.

Second, why do we have a goal to "keep relative references within the package"? This is all about security. An application performing regular (default) resolution of relative references should not access resources outside of the package.
If an application needs to access external resources, it must be explicitly defined in the content by specifying an appropriate base Uri or by using network-path form of the relative reference.

- Andrey

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Baker
Sent: Monday, May 22, 2006 1:13 PM
To: Andrey Shur
Cc: John Calhoon (LCA); Ted Hardie; [email-masked]; Jerry Dunietz; Gregg Brown
Subject: Re: [Uri-review] Request for review

Andrey,

Mark.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 23 May 2006 12:03:32 +0900

Hello Andrey,

At 11:08 06/05/23, Andrey Shur wrote:
>First, why do we consider "any relative reference"? Actually we expect 
>users of the packaging technology to store manifold content in the package parts.

Of course.

>Certain kinds of the content may hold relative references.

Yes of course. But you don't have any guarantee that the
content referenced will be part of the package. So if a
user wants to create a 'self-contained' (meaning no
external relative references) package, either the user
(manually, or with dedicated scripts,...) or the packaging
software has to check all the references anyway.
Such a check is only marginally simplified by the
fact that there is a way to rewrite these relative
references so that any in-package relative reference
can start with a single slash, and out-of package
relative refences cannot be converted.

>We don't want to limit our customers to a particular form of relative 
>references.

If you want to make use of the specific properties of your
scheme, you have to rewrite relative references anyhow, in
which case it doesn't make much of a difference.

Let's say the user uses something like ../../foo.ext.
If this is in-package, you can leave it as that. If it
is out-of-package, and you want to restrict such links,
you have to raise an error (or do something else according
to user preferences).

Let's say the user uses something like /path/foo.ext.
Again, this could be in-package or out of package. You have
to check, and if necessary rewrite it.

>Second, why do we have a goal to "keep relative references within the 
>package"? This is all about security. An application performing regular 
>(default) resolution of relative references should not access resources 
>outside of the package.

But the syntax you came up with doesn't guarantee this, as you
showed with the ../../../foo.ext example.

Trying to get security with the syntax, when you actually don't have
that security and therefore need to check things anyway on the
receiving end and/or on the production end doesn't make much sense
to me.

Also, the additional potential security threads created by
an additional escaping layer should be considered.

>If an application needs to access external resources, it must be 
>explicitly defined in the content by specifying an appropriate base Uri

So basically all the security precautions can be circumvented
by using a base URI? Again, security has to be checked on
resolution. Trying to build it into the syntax doesn't work.

>or by using network-path form of the relative reference.

Using a network-path reference means to use the same scheme as
used previously, which means the pack: scheme. As said in
RFC 3986, network-path references are rare anyway.
You don't mention absolute URIs. Are they allowed or not?

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Andrey Shur <[email-masked]>
Date: Tue, 23 May 2006 09:30:12 -0700

Hi Martin,

thank you for the feedback. You have raised the number of points, but before going through them let me describe how relative references in a part's content are resolved according to the Open Packaging Conventions (OPC) technology (which "pack:" Uri scheme is a part of).

Let's take the package http://www.mysite.com/my.package. The package holds a number of parts; the part named /bar.xml holds relative references "../../foo.ext" and "/path/foo.ext".

OPC defines the base Uri of the part named /bar.xml as pack://http%3c,,www.mysite.com,my.package/bar.xml. Note that OPC only defines the base Uri of a part, the resolution of relative references follows the algorithm defined in RFC 3986 section 5.

If content of the part /bar.xml does not hold explicit instructions defining another base Uri (e.g. xml:base attribute), relative references are resolved against the base Uri of the part.

In our example:

Relative reference ../../foo.ext resolved against the base Uri = pack://http%3c,,www.mysite.com,my.package/bar.xml gives us pack://http%3c,,www.mysite.com,my.package/foo.ext. The obtained Uri identifies part named /foo.ext within the given package.

Relative reference /path/foo.ext resolved against the base Uri = pack://http%3c,,www.mysite.com,my.package/bar.xml gives us pack://http%3c,,www.mysite.com,my.package/path/foo.ext. The obtained Uri identifies part named /path/foo.ext within the given package.

We can see from these two examples that any relative reference (let's put network-path aside) being resolved against the base Uri of the part gives us the "pack:" Uri that identifies a part within the same package.

Regards,
Andrey Shur

-----Original Message-----
From: Martin Duerst [mailto:[email-masked]]
Sent: Monday, May 22, 2006 8:04 PM
To: Andrey Shur; Mark Baker
Cc: John Calhoon (LCA); [email-masked]; Ted Hardie; [email-masked]; Jerry Dunietz; Gregg Brown
Subject: RE: [Uri-review] Request for review

Hello Andrey,

Of course.

Regards,    Martin.

---

From: "Mark Baker" <[email-masked]>
Date: Tue, 23 May 2006 22:24:32 +0200

> to check, and if necessary rewrite it.

AIUI, that's a guaranteed in-package reference.

But I agree with the gist of your argument, Martin; it's where I was
going with my questioning, in particular the value of syntactic safety
(low) vs. the cost of deploying a new URI scheme (high).

Mark.

---

From: Martin Duerst <[email-masked]>
Date: Wed, 24 May 2006 19:28:44 +0900

At 01:30 06/05/24, Andrey Shur wrote:
 >Hi Martin,
 >
 >thank you for the feedback. You have raised the number of points, but
 >before going through them let me describe how relative references in a
 >part's content are resolved according to the Open Packaging Conventions
 >(OPC) technology (which "pack:" Uri scheme is a part of).

[Examples removed]

 >We can see from these two examples that any relative reference (let's put
 >network-path aside) being resolved against the base Uri of the part gives
 >us the "pack:" Uri that identifies a part within the same package.

Well, that's true as long as you stay within your packageing
technology. But what I was talking about (sorry I wasn't explicit
enough) was conversion from e.g. a directory full of files and
directories and so on to a package and again backwards.

Relative references serve various purposes. One of the is to
be able to use short URIs for 'close' things. But another is to
be able to create collections of stuff where internal references
don't have to be changed if the location of the overall thing
changes. The "location of the overall thing" doesn't only mean
in the same scheme, but also across schemes (http: and ftp:
would be the example where this is easiest to do/show).

If you re-read my previous mail from that viewpoint, I hope you
understand my point better.

Regards,     Martin.

---

From: "Jerry Dunietz" <[email-masked]>
Date: Thu, 25 May 2006 22:49:52 -0700

Forgive me for being late to join this conversation.  Thanks for all of 
the constructive discussion!  I'm going to chime in with some long mail,

picking up on some comments made by both Martin Duerst and Mark Baker.  

I'm covering three topics here:

I.   Relative references and leading slash
II.  Martin Duerst's scenarios involving a "directory of files"
III. Refining the goals for the "pack:" scheme

I. Relative references and leading slash

Just to confirm, I hope that it is now clear that there is no
requirement 
for the relative references within a package to begin with a leading
slash.  
Assume that we take some hierarchically-"close" resources on the web,
and 
we copy their representations (unmodified) into a package.  If the
package 
preserves enough of the "close" naming hierarchy surrounding the
resources, 
then the original relative-references will now resolve to the 
representation-copies within the package.  Achieving this behavior is
one 
of the goals we had in specifying the "pack:" scheme.

II.  Martin Duerst's scenarios involving a "directory of files"

Martin Duerst wrote:

> But what I was talking about (sorry I wasn't explicit

I want to take the time to expound on this topic at some length, just to

make sure that we clearly understand one another.  (I think that I
understand 
the concerns, and I believe that the implications are acceptable.)

It is true that if one of our resource-representations contained a
relative 
reference which pointed outside of the "close" naming hierarchy that has

been preserved in the package, then such a relative reference will no
longer 
resolve within the package in a manner that is consistent with the way
in 
which it was resolved against the original HTTP server.  Most likely,
the 
relative-reference will fail to resolve.  However, it is conceivable
that 
the relative reference will now be interpreted as pointing to some other

resource-representation within the package.  So, for example, if I have 
on a web-server:

http://example.com/foo.xml
http://example.com/a/foo.xml
http://example.com/a/b/foo.xml
http://example.com/a/b/bar.xml

Let us assume that http://example.com/a/b/bar.xml contains within it 
three relative references, with the following forms:

"foo.xml"
"../foo.xml"
"/foo.xml"

Against the HTTP server, the these three reference will resolve to 
*different* resources.

Now, suppose we choose to gather everything under
"http://example.com/a/b" 
into a package.

There are three different naming choices I might make in putting
together 
my package.

1)

/foo.xml
/bar.xml

2)

/b/foo.xml
/b/bar.xml

3)

/a/b/foo.xml
/a/b/bar.xml

If I take the first naming approach, the relative references of the 
form "../foo.xml" and "/foo.xml" behave very differently then they did 
against the HTTP server.  They both successfully resolve, but produce 
very different results than they originally did.  The behavior of 
"../foo.xml" is a direct result of the requirements of RFC 3986 -- 
the excess leading ".." segments (in the context of the package
authority) 
effectively disappear.  All three references now resolve to the same 
resource, whereas they resolved to three different resources originally.

If I take the second or third naming approach, the references of the 
form "../foo.xml" and "/foo.xml" will not be resolvable, but at least 
they will not resolve to totally different resources (or copies of 
a resource-representations) then they did in the original environment.  
However, if I take the second approach, there are other relative 
references which will (successfully) resolve differently then they 
would have originally.

By taking the third approach, I guarantee, without processing the 
contents of every copied resource-representation, that no formerly-
functioning relative reference will *successfully* resolve differently 
then it did in the original environment.

If I take instead the stronger goal of assuring that no formerly-
functioning relative-reference is broken, then I must either propagate 
every resource from the original web-server (authority) into the
package, 
or I must filter all of the references within all of the resources 
being propagated into the package.

So far, I've only addressed what happens when I propagate a bunch 
of resources into a package.  I haven't discussed what happens if 
I take a copy of the package, and then "explode" its contents to point 
deep within the naming hierarchy of my own private filesystem or web-
server.  If any reference contained excess leading ".." segments on 
the original HTTP server, then it is true such a reference might have 
successfully resolved in the original environment and also within the 
package environment, but it might (successfully) resolve differently 
in the new environment.

Having taken the time to write all of the above down, I don't actually 
consider the behavior I've described above to be problematic enough 
to outweigh the advantages of what we've designed.  I recognize that 
others might disagree.  But I do want to make sure that I fully 
understand the behavior that might be seen as problematic.  Have 
I captured your concerns (perhaps in too great detail)?  Hopefully, 
we can agree on all matters of fact, but we might have disagreements 
on matters of judgment.

III. Refining the goals for the "pack:" scheme

Let me list a few goals here, some of which were not clearly called out 
in the original submission.  I believe that this set of goals leads us 
to the design we've proposed here.  I think it important that we clearly

distinguish between discussions questioning the goals below from 
discussion questioning the means for achieving these goals.  (Both are 
interesting discussions, but let's me clear about when we are discussing

each.)

1)  Relative references within a package should "work" correctly.

2)  Given a package on an HTTP server somewhere, we should be able to 
construct an absolute reference to a particular part nested in the 
package, made from outside of the package.

3)  We would like the absolute reference described in #2 to function 
without requiring the deployment of any new code on the HTTP server 
holding the package.  (Mark Baker's suggestion of 
"http://www.mysite.com/my.package/a/b/foo.xaml" does not satisfy 
this goal.)

4)  We would like to deliver behavior above without changing existing 
code that relies upon off-the-shelf client-side libraries for dealing 
with URIs.  (That is, we assume that client-side library or installed
code would be updated to support the "pack:" scheme, but the 
code that calls the library requires no change.)

So -- are concerns being expressed with respect to the desirability of 
the new set of goals I describe here, or are they with the means for 
achieving those goals?  (I do think that it would be reasonable for us 
to flesh out the goals stated in the original submission.)

-- Jerry Dunietz