Subject: [OAUTH-WG] Defining a maximum token length?

From: David Recordon <[email-masked]>
Date: Tue, 9 Mar 2010 15:50:32 -0800

Ideally we'd limit the length of access and refresh tokens as well as
client keys and secrets to no more than 255 characters (a one byte
varchar in MySQL).  Is this an issue for anyone?

The OAuth 1.0 protocol specifically states:
Clients should avoid making assumptions about the size of tokens and
other server-generated values, which are left undefined by this
specification.

That seems like a poor idea when it comes to implementability of the
technology.  Why did OAuth 1.0 make that decision?

--David

---

From: Chuck Mortimore <[email-masked]>
Date: Tue, 9 Mar 2010 15:57:59 -0800

No issue, and there is certainly precedent. SAML 2.0 specifies the following about persistent name identifiers, which would be similar is use and formfactor:

Persistent name identifier values MUST NOT exceed a length of 256 characters

-cmort

On 3/9/10 3:50 PM, "David Recordon" <[email-masked]> wrote:

--David
_______________________________________________
OAuth mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/oauth

---

From: Marius Scurtescu <[email-masked]>
Date: Tue, 9 Mar 2010 16:23:23 -0800

On Tue, Mar 9, 2010 at 3:50 PM, David Recordon <[email-masked]> wrote:
> varchar in MySQL).

Add verification codes to the list as well.

> Is this an issue for anyone?

Not sure if anyone really wants to do this, but long tokens would
allow you to implement a stateless authorization server. A refresh
token can encrypt all the information needed to issue an access toke,
similar for a verification code. Such a server could either not deal
with revocations and replays, or track state only for revoked refresh
tokens and used verification codes.

That being said, I don't see a problem with limiting the lengths.

Marius

---

From: David Recordon <[email-masked]>
Date: Tue, 9 Mar 2010 16:44:19 -0800

I believe that Google wishes to encode information within access
tokens so that they can be verified in a stateless manner.  Brian, how
many characters do you need?

---

From: Dick Hardt <[email-masked]>
Date: Tue, 9 Mar 2010 17:14:47 -0800

> That being said, I don't see a problem with limiting the lengths.

I would not want to limit them anymore than they need to be.
When editing OAuth WRAP, we looked into size issues. Current limits are HTTP header size limitations, which are 4-8K total.

Given the ability to put all the claims needed into the Access Token, I can see Access Tokens being 1-2K and being really useful.

-- Dick

---

From: Ethan Jewett <[email-masked]>
Date: Tue, 9 Mar 2010 21:24:59 -0500

Agreed. I've heard tell of Yahoo access tokens with encoded
information weighing in at up to 800 characters. I don't see anything
necessarily wrong with this and I don't think there's much reason to
limit it in the spec. It could incur a significant bandwidth cost, but
since the provider is going to shoulder most of this cost the provider
in a good position to make the tradeoff calculation.

I think it would make sense to advise client library and application
programmers to provide for the possibility of and storage of large
tokens. We should probably reference examples of tokens seen in the
wild and mention the technical limitations on token length from the
HTTP protocol (with Dick outlines). I'm not sure where in the spec
this would go, but it sounds like a good thing to include.

Ethan

---

From: Dick Hardt <[email-masked]>
Date: Tue, 9 Mar 2010 18:33:09 -0800

> this would go, but it sounds like a good thing to include.

In the WRAP spec, we had a section entitled "Parameter Considerations".

-- Dick

---

From: David Recordon <[email-masked]>
Date: Tue, 9 Mar 2010 19:16:40 -0800

The challenge is that client developers (who we really want to make
OAuth dead simple for) will be forced to use less optimal storage for
tokens (blobs versus shorter and indexable types such as varchars).
They also won't be able to build any assumptions around token length
into their database design.  Any DBA cringes when they see the blob
type for multiple columns within a table (access token and refresh
token per user).

Some OAuth servers will have short tokens which a client might
integrate with first and decide that a varchar(255) is reasonable to
hold tokens.  They'll then run across a server with longer tokens, not
realize it, and be confused why their client isn't working when it's
due to their database silently truncating tokens after 255 characters.

--David

---

From: Dick Hardt <[email-masked]>
Date: Tue, 9 Mar 2010 19:25:03 -0800

I understand the desire to set a max length that can easily fit into a DB.
There are lots of other items I think the developer is storing that can be
long as well, like URLs -- so I don't see it as a huge issue.

I do see the need to make it clear that it can be a few K or something like
that so that people don't assume it is shorter than it might be.

-- Dick

On Tue, Mar 9, 2010 at 7:16 PM, David Recordon <[email-masked]> wrote:

> >> When editing OAuth WRAP, we looked into size issues. Current limits are
> HTTP header size limitations, which are 4-8K total.
> >>
> >> Given the ability to put all the claims needed into the Access Token, I
> can see Access Tokens being 1-2K and being really useful.
> >>
> >> -- Dick

---

From: Marius Scurtescu <[email-masked]>
Date: Tue, 9 Mar 2010 19:25:02 -0800

> type for multiple columns within a table (access token and refresh

Why would you index the refresh token column? I think it is enough to
lookup by user id and just retrieve the token.

Also, quite likely that you don't need to store the access token in the
SQL database at all and just make it a session attribute. Not sure what
is reasonable for session data, a large set slows down replication, but
a typical access token should be OK I think.

> due to their database silently truncating tokens after 255 characters.

Well, silently truncating data is a different issue altogether.

Marius

---

From: David Recordon <[email-masked]>
Date: Tue, 9 Mar 2010 19:50:33 -0800

> that so that people don't assume it is shorter than it might be.

Why would the URLs be stored in a database?  Wouldn't they be in
configuration files given that OAuth doesn't support discovery of new
servers?  Yes, developers do store URLs in databases but it's
generally not very optimized.

> lookup by user id and just retrieve the token.

Agreed, you shouldn't need to assuming that you only have one access
token per server/user combination.  Ignoring indicies, in MySQL a
varchar up to 255 characters takes one byte whereas a blob (or a
varchar over 255 characters) is the number of bytes for the string
plus two extra bytes.

In MySQL allowing tokens over 255 characters triples your storage
requirement in the best case. We're not talking about a lot of data
here, but it is worth noting that this design decision has a direct
impact on implementors.

> a typical access token should be OK I think.

That's true if all access tokens expire frequently.  We plan to also
offer long lived access tokens.

> Well, silently truncating data is a different issue altogether.

Sure, but standards can't just work in theory. :)

---

From: Dick Hardt <[email-masked]>
Date: Tue, 9 Mar 2010 20:02:18 -0800

On 2010-03-09, at 7:50 PM, David Recordon wrote:

> Why would the URLs be stored in a database?  

Same reason an Access Token would be. Was that a trick question?
My point was that there are other arbitrary pieces of data that developers store and need to retrieve.

> Wouldn't they be in
> servers?

yet

---

From: Torsten Lodderstedt <[email-masked]>
Date: Wed, 10 Mar 2010 05:44:20 +0100

Hi David,

I wouldn't restrict the length of a tokens to 255 characters, especially 
for access tokens.

Our proprietary token service today uses a token type combination 
similar to OAUTH refresh and access tokens. Based on our experiences I 
would assume the following characteristics of a large-scale OAuth 2.0 
implementation.

Access tokens: Autonomous (w/o callback to authorization server) token 
validation and access control on the service side requires to put all 
relevant user/consumer data into the access token. So access token would 
contain the following data: User Id, Consumer id, context(s), validity, 
authentication methods, user attributes, user permissions, digital 
signature, token id

For our token format, access token length would vary between 200 and 700 
Bytes.

Refresh tokens: in order to refuse refreshing requests for outdated 
tokens early and w/o database access, an authorization server would 
probably put a lot of data into refresh tokens: User Id, Consumer id, 
context(s), validity, authentication methods, digital signature

For our token format, refresh tokens length would be about 200 Bytes. 
 From my perspective, refresh tokens are a typical candidates for 
long-therm database storage on the consumer side. So limiting the length 
would be acceptable, if there is a real benefit. Length limitations 
could also be beneficially for the authorization server since it needs 
to lookup refresh tokens often.

Regarding request payload length: For GET request there is the typical 
header length limitation. POST requests may contain an virtually 
unlimited size of data, even SAML assertion of > 10 Kb could be passed 
w/o any problem :-)

Limiting the length of the client key is ok.

The length of client secrets is somehow related to the algorithms used 
for consumer authentication. So a 255-chars restriction would probably 
raise issues when using public key crypthography.

regards,
Torsten.

---

From: Luke Shepard <[email-masked]>
Date: Tue, 9 Mar 2010 21:10:16 -0800

Whoa- are we seriously saying we need more than 255 characters to encode a token? (By the way, that's 10^396 combinations, with letters and numbers.)

Having short tokens makes the whole protocol much simpler, more approachable, easy to use for developers. I will push hard to keep them short and sweet.

For example, here's what would be needed in a naive, stateless Facebook access token:
 - 32-char api_key (client identifier)
 - 42-char session key
 - 20-char user id (max)
 - 20-char signature

All that comes to 114 characters max, which I still consider to be way too long. With a few additional optimizations (cut the signature length, base convert the app ID), it becomes closer to 70 characters, and I haven't even started trying to compress it or anything yet.

Here's what that URL would look like:

	http://example.com/oauth_endpoint?wrap_access_token=2m2zpk2w|2.5ybrE_bzYCp6A7_pK5PDVA__.3600.1268200800-2901279|aCe23FSrNM.

For comparison, here's a URL with a 255-character access token:

	http://example.com/oauth_endpoint?9295730172112|2.5ybrE_bzYCp6A7_pK5PDVA__.3600.1268200800-2901279|zpssaSxclo9eWs7Jw1Hga3FSrN9295730172112|2.5ybrE_bzYCp6A7_pK5PDVA__.3600.1268200800-2901279|zpssaSxclo9eWs7Jw1Hga3FSrNM9295730172112|2.5ybrE_bzYCp6A7_pK5PDVA__.3600.1268200800-2901279|zpcloMM'

Isn't that ugly? At least in my email window, the first URL fits on one line while the other stretches to three. This is starting to remind me of some OpenID URLs I've seen.

Torsten just replied:

>  So access token would 
> signature, token id ... length would vary between 200 and 700 Bytes.

Can you give a representative example of that? I find it hard to believe that even all of those attributes couldn't fit into a much shorter token, if smartly sized.

Thanks,
Luke

---

From: Brian Eaton <[email-masked]>
Date: Tue, 9 Mar 2010 21:15:38 -0800

> many characters do you need?

I think it would be OK to make request tokens short.  They are
service-specific passwords, and can/should be treated as such.

For authorization servers: store cryptographically strong one-way
hashes.  No need to salt the hashes, because they are very
high-entropy already.

For clients: store them in a DB that is locked down as tight as you
can get it.  Plan on changing your client secret when the DBA leaves
the company.

(hrm, I just saw that Torsten has a use case for refresh token ==
encrypted blob.  I am deeply suspicious.)

For most deployments, access tokens are basically like session
cookies.  Some deployments use self-verifying cookies with keys stored
server-side, other people store sessions in databases or in-memory.
There are lots of fun trade-offs here.  The choices you make have
fundamental impacts on every facet of your systems, ranging from DNS,
to load balancers, to frontends, to backend servers.  The choice
ripples out to impact scalability, security, latency, and availability
*everywhere*.

It would be a mistake for the spec to say that access tokens have to
be short - that one decision impacts way too many other things.

The non-opaque access token use cases (SWT, claims in access tokens)
are interesting, and for those use cases access tokens can't be short
either.

Cheers,
Brian

---

From: Torsten Lodderstedt <[email-masked]>
Date: Wed, 10 Mar 2010 06:25:54 +0100

Hi,
> Whoa- are we seriously saying we need more than 255 characters to encode a token? (By the way, that's 10^396 combinations, with letters and numbers.)
yes. As I said, in our approach we put permissions and attributes into 
the tokens.
> Having short tokens makes the whole protocol much simpler, more approachable, easy to use for developers. I will push hard to keep them short and sweet.
why make short tokens the protocol simpler?

regards,
Torsten.

---

From: David Waite <[email-masked]>
Date: Tue, 9 Mar 2010 23:33:24 -0700

> type for multiple columns within a table (access token and refresh

To throw another opinion on the fire - access tokens shouldn't be required to be over an order of magnitude smaller than browser cookies or HTTP headers, and nearly an order of magnitude smaller than a URL. In fact, 256 characters is only barely long enough to hold the longest allowed domain name.

To parallel browser cookies, HTTP headers and URLs, there are accepted 'minimum maximums' out there - which the minimum size that user agents are expected to support, and the maximum size the server will assume be supported by an arbitrary agent. I don't believe all of these are formalized, but they all exist.

> due to their database silently truncating tokens after 255 characters.

Sounds like they should consider picking a different database vendor.

-DW

---

From: Luke Shepard <[email-masked]>
Date: Tue, 9 Mar 2010 23:02:00 -0800

I'd still like to see someone construct an example access token that is longer than 255 characters that would be reasonably used. If there are real, legitimate use cases that REQUIRE more than that many characters, then let's hear them. I don't think that appealing to "it might be useful" is a good enough argument.

>  nearly an order of magnitude smaller than a URL

I'd hope so, given that it is supposed to be a single URL parameter, and it has to share that URL length with other parameters to actual API calls.

> I don't believe all of these are formalized, but they all exist.

The lack of formalized URL length limits has been a headache for us, since we end up testing each browser and coding to the most restrictive. I'd like to avoid the need for OAuth clients to do the same.

---

From: Brian Eaton <[email-masked]>
Date: Tue, 9 Mar 2010 23:35:20 -0800

On Tue, Mar 9, 2010 at 11:02 PM, Luke Shepard <[email-masked]> wrote:
> I'd still like to see someone construct an example access token that is
> longer than 255 characters that would be reasonably used. If there
> are real, legitimate use cases that REQUIRE more than that many
> characters, then let's hear them. I don't think that appealing to
> "it might be useful" is a good enough argument.

Cached group memberships and other user attributes are what typically
blow out the cookie size in enterprise environments.

If you browse around the web for a bit you'll see various sites that
set very large cookies after users log in.  They are caching state in
the cookie.  It's all fair game for API tokens as well.

Cheers,
Brian

---

From: Justin Smith <[email-masked]>
Date: Wed, 10 Mar 2010 07:58:34 +0000

Along those lines, here's an access token (SWT w/o URL encoding) that has some role and attribute data. I think it is representative of how customers are using the OAuth WRAP implementation in AppFabric.

Role=user,superuser,administrator&Action=create,retrieve,update,delete&CustomerID=123456789&Issuer=https://acsinteropdemo.accesscontrol.windows.net/&Audience=http://acsinteropdemo.appspot.com/orders&ExpiresOn=1268207444&HMACSHA256=0p1PPgCcox7uRw1ETtUTlpwBgfGAF3UhTFaHUPaprik=

--justin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Eaton
Sent: Tuesday, March 09, 2010 11:35 PM
To: Luke Shepard
Cc: OAuth WG
Subject: Re: [OAUTH-WG] Defining a maximum token length?

Cheers,
Brian

---

From: John Kemp <[email-masked]>
Date: Wed, 10 Mar 2010 10:42:03 -0500

> varchar in MySQL).  Is this an issue for anyone?

Yes. 

Why would we want to encode such a specific implementation decision into the OAuth standard (where I'm sure there are lots of folks who don't use MySQL to log tokens)?

> technology.

Can you explain why this is a poor idea? 

It seems to me that such a statement leaves it open to implementations to create tokens that make sense for their usage and which work with their clients and partners. 

>  Why did OAuth 1.0 make that decision?

One reason I can imagine is to make it possible to encode information into the token itself so that the token can be "self-contained" (mentioned also by others on this list). 

A good reason to limit token length would be if there were issues in the client implementation which meant that tokens might get truncated by particular implementations - but I would guess that these would again be implementation-specific, and not something which should be standardized generally. 

Regards,

- johnk

---

From: Moritz Maisel <[email-masked]>
Date: Wed, 10 Mar 2010 17:16:57 +0100

> One reason I can imagine is to make it possible to encode information into the token itself so that the token can be "self-contained" (mentioned also by others on this list). 

Though thats an interesting option, compatibility of implementations
might be easier to achieve by strict specifications like "maximum of 256
characters".

Just to get an idea about the situation: Is the mentioned
"self-contained token" a common scenario / popular demand that needs to
be covered?

Regards,
Moritz

-- 
sipgate GmbH - Gladbacher Str. 74 - 40219 Düsseldorf
HRB Düsseldorf 39841 - Geschäftsführer: Thilo Salmon, Tim Mois
Steuernummer: 106/5724/7147, Umsatzsteuer-ID: DE219349391

www.sipgate.de - www.sipgate.at - www.sipgate.co.uk

---

From: John Kemp <[email-masked]>
Date: Wed, 10 Mar 2010 11:30:34 -0500

> might be easier to achieve by strict specifications like "maximum of 256

Could you explain why you need to standardize a maximum token length, or why you would want to standardize on implementations of a token store rather than a token-exchange protocol here? 

> "self-contained token" a common scenario / popular demand that needs to

Yes it is. Several people on this list have already mentioned it. 

Regards,

- johnk

---

From: Paul Lindner <[email-masked]>
Date: Wed, 10 Mar 2010 12:47:27 -0800

Standards have size limits to overcome operational issues all the time.  For an extreme example look at the backflips that MIME goes through to insure that mail can be delivered by even the most hostile relay.  (Including systems using EBCDIC, and systems that mangle payloads!)

If there are no bounds on the size of a token the end result is that some parts of the protocol move from MAY to MUST.  For example, many firewall/proxy services in the wild will truncate URLs and even HTTP headers  much shorter than the 4k normally assumed.    Ergo, POST support is now a MUST.

---

From: John Kemp <[email-masked]>
Date: Wed, 10 Mar 2010 17:23:59 -0500

On Mar 10, 2010, at 3:47 PM, Paul Lindner wrote:

> Standards have size limits to overcome operational issues all the time.

Standards usually standardize on the things necessary for interoperability, and token size isn't a necessity for interoperability unless we make it one. No-one has explained the benefit of doing that yet. 

>  For an extreme example look at the backflips that MIME goes through to insure that mail can be delivered by even the most hostile relay.  (Including systems using EBCDIC, and systems that mangle payloads!)
> 

And in environments without such intermediaries, tokens of much larger sizes may be sent without problems. 

I agree that implementation considerations such as URL length and so on may be relevant in some environments, but I would oppose some mandatory limit in OAuth on token size. 

It's fine, however, to say that if we rely on HTTP as a substrate that we should be governed by any relevant and documented standard HTTP (or other relevant specifications) limits, and provide implementation advice to suggest that in some environments where implementations don't follow the standards, URLs may be truncated by *some implementations* (for example). 

Regards,

- johnk

---

From: Paul Lindner <[email-masked]>
Date: Wed, 10 Mar 2010 14:47:42 -0800

RFC 2822 <http://www.rfc-editor.org/rfc/rfc2822.txt> provides an apropos
example in sections 2.1.1:

There are two limits that this standard places on the number of
characters in a line. Each line of characters MUST be no more than
998 characters, and SHOULD be no more than 78 characters, excluding
the CRLF.

RFC 2821 <http://www.rfc-editor.org/rfc/rfc2821.txt> has a whole section on
size limits in section 5.4.3.1.

Good reading....

OAuth  has it's own transport related issues, so our MUST limit would
probably be determined by HTTP or other transport considerations.  As a
community we can hopefully come to an agreement on what SHOULD value makes
sense...

I also suggest that error handling around this condition (and everything
else) be nailed down so things don't just break in mysterious ways.
 Remember to SMTP a 500 error is 'line too long'..

On Wed, Mar 10, 2010 at 2:23 PM, John Kemp <[email-masked]> wrote:

> Standards usually standardize on the things necessary for interoperability,
> and token size isn't a necessity for interoperability unless we make it one.
> No-one has explained the benefit of doing that yet.
>
> >  For an extreme example look at the backflips that MIME goes through to
> insure that mail can be delivered by even the most hostile relay.
>  (Including systems using EBCDIC, and systems that mangle payloads!)
> >
> > If there are no bounds on the size of a token the end result is that some
> parts of the protocol move from MAY to MUST.  For example, many
> firewall/proxy services in the wild will truncate URLs and even HTTP headers
>  much shorter than the 4k normally assumed.    Ergo, POST support is now a
> MUST.
>
> I agree that implementation considerations such as URL length and so on may
> be relevant in some environments, but I would oppose some mandatory limit in
> OAuth on token size.
>
> It's fine, however, to say that if we rely on HTTP as a substrate that we
> should be governed by any relevant and documented standard HTTP (or other
> relevant specifications) limits, and provide implementation advice to
> suggest that in some environments where implementations don't follow the
> standards, URLs may be truncated by *some implementations* (for example).
>
> Regards,
>
> - johnk
>
> >
> >>>> One reason I can imagine is to make it possible to encode information
> into the token itself so that the token can be "self-contained" (mentioned
> also by others on this list).
> >>>>
> >>>
> >> Could you explain why you need to standardize a maximum token length, or
> why you would want to standardize on implementations of a token store rather
> than a token-exchange protocol here?
> >>
> >>>