Subject: Coming changes to materials uploads

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 7 Mar 2018 17:47:48 +0100

Hi,

During the last few IETF meetings, there have been a few cases of agenda
and minutes uploads that have not worked well, for various reasons.  Some
have unintentionally used frames, and failed to include the frame contents;
some have used iframes, which pulls the actual content from elsewhere, which
means it won't actually be saved on the IETF servers and archived.  There
has also been issues relating to styling and use of javascript.  This shows,
of course, that malicious uploads (even if unintentional) are possible.

Considering this, it seems that a good and general approach would be to
do what is often called 'sanitization' of uploaded html content.

This will reduce the permitted html tags, attributes, and styles to those
configured for the sanitizer.

In addition to the security benefits, and confidence in actually capturing
and archiving the intended session agenda and materials, this will set the
stage for one-stop-shop session pages on the datatracker.  These would 
combine on one page all the information germane to a session:  The actual
agenda text, links to slides, drafts, audio and video streams, etherpad,
and whatnot, similar to what has been provided earlier on tools.ietf.org.
This is difficult to do if uploaded html comes with widely varying styles.

However, the cost of this is that if you upload agendas and minutes in HTML
format, you will need to check the results after upload, to make sure that
the agenda and minutes still captures your intent after the sanitization.

The integration of sanitization in the materials upload code is ready for
release, but I wanted to send this note first, to give you a heads-up and
avoid surprises; and also in order to invite you to give feedback on the
actual html tags which are left intact by the sanitizer.  We will start out
with a set which is wider than the sanitizer default, and are prepared to
adjust over time, as needed.  You can see the current acceptable elements
by checking the source file:
https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/ietf/utils/html.py

Best regards,

	Henrik

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 9 Mar 2018 08:52:37 -0700

I would be happy to upload markdown but I’m not very keen on having to use HTML. It’s like PERL, nice to write you own, awful to have to edit someone else. 

> On Mar 7, 2018, at 9:47 AM, Henrik Levkowetz <[email-masked]> wrote:
> 
> Hi,
> 
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Fri, 9 Mar 2018 16:08:29 +0000

I didn't read this as mandating HTML:

> However, the cost of this is that *if you upload agendas and minutes in HTML*
> format, you will need to check the results after upload, to make sure that...

Al

> -----Original Message-----
> From: WGChairs [mailto:[email-masked]] On Behalf Of Cullen
> Jennings
> Sent: Friday, March 9, 2018 10:53 AM
> To: Henrik Levkowetz <[email-masked]>
> Cc: WG Chairs <[email-masked]>
> Subject: Re: Coming changes to materials uploads
> 
> 
>  I would be happy to upload markdown but I’m not very keen on having to
> use HTML. It’s like PERL, nice to write you own, awful to have to edit
> someone else.
> 
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__trac.tools.ietf.org_tools_ietfdb_browser_trunk_ietf_utils_html.py&d=Dw
> IFaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=W2hIr4jYXyIw2r8qVDX9qMyGAkTWq
> qZHc10Vn-kBDg4&s=D7a2EQ8c8flvHFvon6MmaINpSd_UOV7cV5gRh2QEoWE&e=
> >
> >
> > Best regards,
> >
> > 	Henrik
> >
> >
> >

---

From: Yaron Sheffer <[email-masked]>
Date: Fri, 9 Mar 2018 08:09:55 -0800

+1. I've been using Markdown to generate HTML for the agenda, I'd much 
rather upload Markdown directly.

On the security side, note that all Markdown also allows raw HTML. So 
we'd still be stuck with sanitization.

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 9 Mar 2018 09:14:27 -0700

> On Mar 9, 2018, at 9:08 AM, MORTON, ALFRED C (AL) <[email-masked]> wrote:
> 

No worry then - I took this the wrong way.

---

From: Russ Housley <[email-masked]>
Date: Fri, 9 Mar 2018 11:17:16 -0500

You can still use plain text.  Some WG Chairs want to use HTML for agenda and minutes.  This change allows them to do so safely.

Russ

> On Mar 9, 2018, at 10:52 AM, Cullen Jennings <[email-masked]> wrote:
> 
>

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 9 Mar 2018 17:24:11 +0100

Hi Cullen,

>  I would be happy to upload markdown but I’m not very keen on having to use HTML. It’s like PERL, nice to write you own, awful to have to edit someone else. 

This change doesn't require you to upload html, not at all.  Upload plain
text or markdown as before if you wish.  But _if_ you upload html, it will
now be sanitized, not provided as-is.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 9 Mar 2018 17:27:50 +0100

Hi Yaron,

> rather upload Markdown directly.

Ack.  My thoughts on that is to eventually process markdown when presenting
uploaded materials (at least in some contexts).  When doing that, I think
we still need to make the original raw upload available, too.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 9 Mar 2018 17:28:42 +0100

> You can still use plain text.  Some WG Chairs want to use HTML for agenda and minutes.  This change allows them to do so safely.

Just so.

	Henrik

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Mon, 12 Mar 2018 17:37:26 +0100

hi Henrik, all,

I'll note that the sanitization seems to remove inline CSS (i.e. <head><style> contents) in pandoc-generated HTML, not just linked CSS. This seems like overkill to me.

In any case, it makes the output look mid-1990s enough that it's probably better just to upload the markdown source as if it were text.

Cheers,

Brian

> On 9 Mar 2018, at 17:24, Henrik Levkowetz <[email-masked]> wrote:
> 
> Hi Cullen,
>

---

From: Sean Turner <[email-masked]>
Date: Mon, 12 Mar 2018 16:54:44 +0000

> On Mar 9, 2018, at 15:52, Cullen Jennings <[email-masked]> wrote:
> 
> I would be happy to upload markdown

+1

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 12 Mar 2018 18:13:05 +0100

> I'll note that the sanitization seems to remove inline CSS (i.e. <head><style> contents) in pandoc-generated HTML, not just linked CSS. This seems like overkill to me.

Yep, sanitizing on tags only is heavily broken.

(Use

https://www.ietf.org/proceedings/96/agenda/agenda-96-core

as an example of an agenda that would be borked by this.

And, yes, that was markdown before we had to upload it; we were forced to use HTML because the materials server wasn’t UTF-8 ready for plaintext at the time.)

Grüße, Carsten

---

From: Adam Roach <[email-masked]>
Date: Mon, 12 Mar 2018 12:31:32 -0500

> as an example of an agenda that would be borked by this.

Ditto for minutes: 
https://www.ietf.org/proceedings/95/minutes/minutes-95-sipcore

/a

---

From: Fred Baker <[email-masked]>
Date: Mon, 12 Mar 2018 13:49:24 -0400

Here's an example of one that wouldn't, or at least I don't think it would:
    https://datatracker.ietf.org/meeting/101/materials/agenda-101-v6ops.html

> On Mar 12, 2018, at 1:13 PM, Carsten Bormann <[email-masked]> wrote:
>

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 12 Mar 2018 19:49:22 +0100

Hi Brian,

> I'll note that the sanitization seems to remove inline CSS (i.e.
> <head><style> contents) in pandoc-generated HTML, not just linked
> CSS.

This is currently the case, yes.

> This seems like overkill to me.

Yes, maybe.  My thought was that when we get to the point of embedding
the agenda content in one page, together with all other information
relevant to a session, the style in use on the datatracker would be
applied, and give some consistency to the look-and-feel.

> In any case, it makes the output look mid-1990s enough that it's
> probably better just to upload the markdown source as if it were
> text.

I expect to do markdown conversion for display eventually, which should
make this even more attractive.

I'm open to tweaking the sanitization (as long as we don't make it
meaningless by making it unsafe).  Neither of the two sanitization
libs I've tested for this is willing to handle <head/> and <body/>
separately, so a bit of extra processing is going to be needed in
order to preserve <style/> in <head/>, but that's doable.  There is
also a third sanitization option which is slower than the other two,
which might be able to deal better with this.  I'll investigate that.

I observe that this still leaves a bit of tension between the desirability
of permitting arbitrary styling of agendas when seen as stand-alone pages,
and the desirability of producing one-stop pages with all session related
information in one place, with consistent styling.

But if it is the will of the chairs to preserve css in <head><style>,
at least for agendas viewed as stand-alone pages, I'll work on that.

Best regards,

	Henrik