Subject: [ietf-822] Question about line folding and maximum line length

From: Bryan Frimin <[email-masked]>
Date: Thu, 06 May 2021 15:52:08 +0200

Hello everyone,

I am implementing an e-mail client. I have to implement the SMTP
protocol, starting with the internet message format defined by RFC
5322[1].

After reading the RFC, I found two points that are not clear to me.

1. The maximum line length.
RFC 5322 section 2.1.1[2] describes two possible line length limits. The
historical one, 78 characters, and the new one, 998 characters. Is the
historical limit still relevant for an email client in 2021, or can I
just use the new one?

2. The long line folding mechanism.
RFC 5322 section 2.2.3[3] defines a folding mechanism for long header
fields. But it does define whether it is possible to fold a header field
name longer than the limit (yes it is an edge case, but it could
happen). And folding can only be done if a lexical token or a space is
present. So what about values without any separator ?

Regards,

[1] https://tools.ietf.org/html/rfc5322
[2] https://tools.ietf.org/html/rfc5322#section-2.1.1
[3] https://tools.ietf.org/html/rfc5322#section-2.2.3

-- 
Bryan Frimin
https://www.frimin.fr
[email-masked]

---

From: Paul Smith <[email-masked]>
Date: Thu, 6 May 2021 15:33:18 +0100

> historical limit still relevant for an email client in 2021, or can I

998 is a MUST (to avoid possibly breaking something, although it's 
unlikely nowadays)

78 is a SHOULD (to make it easier for users to read).

The 78 character limit is not 'historical'; it's a recommendation, not a 
requirement. In fact in RFC 822, there are no line length limits defined 
at all, just a suggestion not to make them too long. The limits were 
first introduced in RFC 2822 as far as I can see, and that uses the same 
wording as RFC 5322

When reading RFCs it's very important to know the difference between 
MUST and SHOULD (in capitals) as they have a strict meaning.

So, you should try to make lines no longer than 78 characters, but if 
they're longer (but no longer than 998) it's not going to break anything.

> present. So what about values without any separator ?

Don't fold them if you can't. I don't think you'd fold a header field 
name (we don't) - as you say it's an edge-case, and I've never come 
across it.

It is very possible to have header lines where the data does not contain 
any spaces and is longer than 78 characters. That's exactly why the 78 
character line-length limit is a SHOULD and not a MUST.

Obviously if you've got a header field name longer than 998 characters 
that you would HAVE to fold but you can't, then you've got a problem - 
in more ways than one.

-- 
Paul
Paul Smith Computer Services
[email-masked] - 01484 855800

-- 

Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

---

From: "John Levine" <[email-masked]>
Date: 6 May 2021 11:57:15 -0400

>they're longer (but no longer than 998) it's not going to break anything.

Right.  If you're supporting EAI with unencoded UTF-8, which you should,
the 998 is octets, not characters which matters since UTF-8 code points
vary from one octet to four.

>that you would HAVE to fold but you can't, then you've got a problem - 

No kidding.  In practice it appears that systems that add giant private
headers (Yahoo, er, Oath, er, Veriz, er, Apollo, we're looking at you)
tend to make them blocks of opaque hex or base64 that ignore spaces so
they can fold them as needed.

R's,
John

---

From: Pete Resnick <[email-masked]>
Date: Thu, 06 May 2021 12:54:35 -0500

On 6 May 2021, at 9:33, Paul Smith wrote:

> The 78 character limit is not 'historical'; it's a recommendation, not 
> a requirement.

Well, to be clear, a "SHOULD" according to RFC 2119 is basically "MUST 
unless you know what you're doing". But otherwise, Paul is exactly 
right. As 2.1.1 says:

    The more conservative 78 character recommendation is to accommodate
    the many implementations of user interfaces that display these
    messages which may truncate, or disastrously wrap, the display of
    more than 78 characters per line, in spite of the fact that such
    implementations are non-conformant to the intent of this
    specification (and that of [RFC5321] if they actually cause
    information to be lost).  Again, even though this limitation is put
    on messages, it is incumbent upon implementations that display
    messages to handle an arbitrarily large number of characters in a
    line (certainly at least up to the 998 character limit) for the sake
    of robustness.

So, you really ought to accommodate arbitrarily receiving long lines, 
but 998 at a minimum, and on sending limit yourself to 78 if you can, 
but 998 at maximum.

> In fact in RFC 822, there are no line length limits defined at all, 
> just a suggestion not to make them too long. The limits were first 
> introduced in RFC 2822 as far as I can see, and that uses the same 
> wording as RFC 5322

Yep. When 2822 was being written, there were implementations that 
downright blew up on anything over 998. I imagine there still are.

> MUST and SHOULD (in capitals) as they have a strict meaning.

https://www.rfc-editor.org/rfc/rfc2119.txt

> they're longer (but no longer than 998) it's not going to break 

Yep.

>> happen).

Header field names definitely can't be folded. We probably should have 
put an explicit limit of 77 (to leave space for the colon), but we 
didn't. See https://www.rfc-editor.org/errata/eid5918. In the cleanup 
we're currently working on, we added in a comment in the ABNF in section 
3.6.8:

    field-name      =   1*ftext            ; Limit to 77 characters to
                                           ;  stay within 78 char-per-
                                           ;  line recommendation

We haven't seen any actual header field names in the wild that are 
greater than 77 in length, so decided that changing the ABNF to 
"1*77ftext" wasn't worth possibly screwing something else up, given that 
we're trying to advance this to full Standard, and that the comment in 
the ABNF would be sufficient.

(If you'd like to follow along with the current cleanup effort, please 
see https://datatracker.ietf.org/wg/emailcore/about/.

>> And folding can only be done if a lexical token or a space is
> It is very possible to have header lines where the data does not 
> contain any spaces and is longer than 78 characters. That's exactly 
> why the 78 character line-length limit is a SHOULD and not a MUST.
>

Yeah, you also have a similar problem if you have a header field value 
(or a line in the body of a message for that matter) with an unbroken 
chain of greater than 998 characters. There are ways in MIME to solve 
that problem (RFC 2047 for header field values, RFC 2045 for bodies). If 
you can't use one of those, best answer is for your MUA or MSA to tell 
the user "Sorry, can't do that". (At the SMTP -- i.e., MTA -- level, you 
shouldn't be trying to check syntax and re-fold lines; if you can't 
handle the mess you were sent, bounce the message. If you send it to the 
next one down the line and it bounces, so be it.)

pr

-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: Hector Santos <[email-masked]>
Date: Thu, 06 May 2021 20:33:13 -0400

> historical limit still relevant for an email client in 2021, or can I

For SMTP (RFC5321 et al) Internet Mail (RFC5322 et al ) 
transportation, MTA (client/server),  998+EOL is the requirement. EOL 
is <CR><LF> (DOS-based easy to remember). If a line is ending as 
Unix-based <LF> or MAC-based <CR>, then I/O translation must be taken 
into account.

For DISPLAY purposes, the MUA, Mail User Agents, SMTP should at least 
handle old school 80 columns, that meant 78+EOL.  It was always tricky 
to deal with column soft vs hard wraparounds ideas.

> present. So what about values without any separator ?

Serious? A Header Field Bame longer than the line limit?   That would 
instantly cause breakage somewhere. So I don't know off hand if the 
doc makes a mention of the Header Field Name limit, but if not, this 
is the group to make that clarification in the ABNF.

The reality is that there are remote hosting limits that are external 
to RFC5822, perhaps even predated RFC5322 header fields, especially 
the top header fields from the Host/User Mail Reader/Writer MUA 
standpoint and for all telecommunications;

From:
To:
Subject:
Date:

All others are network or some meta-related thing,  perhaps, but there 
are legacy and established systems that has limits based on display 
designs. Much shorter than 998,  How will any device display long 
fields over 80 or extended graphically like 132 that the early smarter 
terminals offered?

Overall, there is no standard on displays, yet, because there are so 
many different ways to do it. Smart Watches?

-- 
Hector Santos,
https://secure.santronics.com
https://twitter.com/hectorsantos