Subject: [rfc-i] Meta decorations in generated HTML

From: "John R Levine" <[email-masked]>
Date: 24 May 2022 20:37:29 -0400

We have two xml2rfc tickets asking to add more meta tags in the rendered 
HTML of RFCs.

Ticket 757 is mine https://github.com/ietf-tools/xml2rfc/issues/757

It asks for the meta tags that Google Scholar uses. I know these are the 
right tags because I added them to the HTML-ized versions of older RFCs 
which are now at long last showing up in Google Scholar.

Tickets 776 and 777 are from Mark Nottingham 
https://github.com/ietf-tools/xml2rfc/issues/777

It asks for Open Graph metadata (see https://ogp.me/) which makes links 
display nicely on Facebook, Twitter, some Wordpress sites, and probably 
other places.  The should work also for the HTML rendered versions of 
I-D's.

Are there other meta tags that people would find useful?  Do we agree that 
we want the OG tags?  We already agreed on the Google Scholar tags about a 
decade ago but it took a while to get around to it.

If we like the OG tags, should I arrange to add them to the HTMLized 
versions of pre-XML RFCs?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

_______________________________________________
rfc-interest mailing list
[email-masked]
https://mailman.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Lars Eggert <[email-masked]>
Date: Wed, 25 May 2022 06:32:21 +0300

Yes and yes, personally 

-- 
Sent from a mobile device; please excuse typos.

> On May 25, 2022, at 03:38, John R Levine <[email-masked]> wrote:
> 
> Do we agree that we want the OG tags?  We already agreed on the Google Scholar tags about a decade ago but it took a while to get around to it.
> 
> If we like the OG tags, should I arrange to add them to the HTMLized versions of pre-XML RFCs?

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 May 2022 06:14:13 +0200

Am 25.05.2022 um 02:37 schrieb John R Levine:
> ...

Are we sure that we need all of them? (It's quite a long list...)

Best regards, Julian

---

From: "John Levine" <[email-masked]>
Date: 25 May 2022 16:38:25 -0400

>Are we sure that we need all of them? (It's quite a long list...)

I'm not sure what "need" means here.  These are tags that Scholar says it uses,
and it's the ones we're putting on the pre-XML RFCs that Scholar is now indexing.

R's,
John

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 May 2022 07:43:56 +0200

> and it's the ones we're putting on the pre-XML RFCs that Scholar is now indexing.

They duplicate a lot of information that is elsewhere, so I think it
would be good to minimize the amount of duplication.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 May 2022 08:30:42 +0200

> would be good to minimize the amount of duplication.

But then, they are buried in inscrutable HTML boilerplate anyway, so I think the need to minimize is somewhat weak.
The need to get indexed prevails, at least from my point of view.
(That doesn’t mean dumping in the kitchen sink, but only what we know that works.)

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 May 2022 14:09:50 +0200

> The need to get indexed prevails, at least from my point of view.

Absolutely. It would just be nice to understand what Google Scholar
actually *needs* (their intransparency is really mind boggling).

Best regards, Julian

---

From: "John R Levine" <[email-masked]>
Date: 26 May 2022 10:18:07 -0400

>> But then, they are buried in inscrutable HTML boilerplate anyway, so I 
>> think the need to minimize is somewhat weak.

We're talking about seven small pieces of bibliographic metadata that are 
added mechanically to the HTML.  They're all the ones listed on the 
Scholar web site that Google uses to decide where to index the pages they 
include.  While I suppose they might still include us if we left some of 
them out, I don't see why we would want to make it harder to index our 
RFCs accurately.

There was one undocumented problem, that they can't handle tags like
", Ed." in the author name which we found out by talking to people who run 
Scholar.  I also added an XML site index that includes all of the versions 
of the RFCs to make spidering easier.

I would have thought it was self-evident why we have redundant 
bibliographic tags in the HTML (not the XML).  After three decades of 
disorganized evolution, different indexes use different tags.  None of 
them are large or hard to create, so if we add them all, we help get our 
documents indexed better.

Regards,

---

From: Larry Masinter <[email-masked]>
Date: Thu, 26 May 2022 08:59:07 -0700

Just to point out that other systems (including Zotero, which I'm familiar
with) use the Google Scholar tags. Would it  be worth putting together an
Informational(?) RFC on metadata available/expected for RFCs?

-- 
https://LarryMasinter.net  https://interlisp.org

---

From: Michael Richardson <[email-masked]>
Date: Fri, 27 May 2022 10:07:27 -0400

John R Levine <[email-masked]> wrote:
    > them out, I don't see why we would want to make it harder to index our RFCs
    > accurately.

I haven't looked at all into what's in the tags, except by view-source.
I compared
view-source:https://www.rfc-editor.org/rfc/rfc6698.html (which was an open tab)
to:
view-source:https://www.rfc-editor.org/rfc/rfc8995.html

and in the former, I see lots of interesting <meta name="citation_...
stuff, but in the later, I do not.

In the former, I do see an almost good self-reference:
   <meta name="citation_pdf_url" content="https://www.rfc-editor.org/rfc/pdfrfc/rfc6698.txt.pdf"/>

I looked this up because I was thinking that some of our indexing of AUTH48
drafts might be dealt with if the indexers could be told where the canonical
URL of the document is.

    > I would have thought it was self-evident why we have redundant bibliographic
    > tags in the HTML (not the XML).  After three decades of disorganized
    > evolution, different indexes use different tags.  None of them are large or
    > hard to create, so if we add them all, we help get our documents indexed
    > better.

I agree.  :-)

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: "John R Levine" <[email-masked]>
Date: 27 May 2022 10:55:55 -0400

> stuff, but in the later, I do not.

Right.  For the pre-XML RFCs, the HTML version is generated on the fly by 
the RFC Editor's web server so it was easy to adjust the script to add the 
tags.  But now the HTML is a static version created by xml2rfc, so it will 
have to add the tags.  This also means we need to re-render the existing 
RFCs to get the tags but I expect to do that for other reasons.

>   <meta name="citation_pdf_url" content="https://www.rfc-editor.org/rfc/pdfrfc/rfc6698.txt.pdf"/>

That's a link to the PDF version of an HTML document.  Scholar says they 
use it, so I add it.

> drafts might be dealt with if the indexers could be told where the canonical

The way to keep AUTH48 pages out of search engines is to tell them not to 
index them.  Earlier this month I helped the RPC update the web site so 
spiders will skip the authors' directory.  The AUTH48 pages that were 
already indexed will eventually age out of the search engines.

Regards,

---

From: Michael Richardson <[email-masked]>
Date: Fri, 27 May 2022 11:52:20 -0400

> spiders will skip the authors' directory.  The AUTH48 pages that were already
    > indexed will eventually age out of the search engines.

Yes, that particular mole has been whacked, but I'm just thinking about other
ways/places that RFCs might show up in non-canonical places.    I might post
a copy on a web site devoted to that technology for instance, and it would be
good if it re-inforced rfc-editor.org as the canonical source, rather than
diluted it.

---

From: "John R Levine" <[email-masked]>
Date: 27 May 2022 12:58:57 -0400

>    > index them.

> good if it re-inforced rfc-editor.org as the canonical source, rather than

I suppose we could put "don't index me" tags into the HTML and edit them 
out on the copies on the rfc-editor web site but that seems too fragile.

The normal way you get high in the search rankings is to have high quality 
links pointing at them.  If all of the places in the datatracker and the 
other IETF web sites pointed at the copies RFCs on the rfc-editor web site 
rather than at other places that would help but that also doesn't seem 
likely.

Regards,

---

From: "Salz, Rich" <rsalz=[email-masked]>
Date: Fri, 27 May 2022 18:16:39 +0000

> I might post

Since the RPC knows what the RFC number will be, having the tools add
	<meta name="source" content="https://rfc-editor/rfc/420/">
Seems pretty simple.  I do not know if it is worthwhile or not.  I am sure that the "source" isn't ideal :)

---

From: "John R Levine" <[email-masked]>
Date: 27 May 2022 14:38:17 -0400

> Seems pretty simple.  I do not know if it is worthwhile or not.  I am sure that the "source" isn't ideal :)

We could put in RFC 6596 canonical web links but I don't know if anything 
pays attention to them.

R's,
John

---

From: Eliot Lear <[email-masked]>
Date: Sat, 28 May 2022 08:17:29 +0200

Hi John,

Since this looks like it can nearly all be automated, it mostly looks 
good to me.  Two points:

  * The independent stream doesn't have a logo, and neither does the
    nascent Editorial stream.  We should probably have something (I'm
    referring to PR#777).
  * Once we start adding metadata tags like this, it'd be good to find
    some way to track the policy sources so that we can track changing
    requirements.

Eliot

On 25.05.22 02:37, John R Levine wrote:
> We have two xml2rfc tickets asking to add more meta tags in the 
> rendered HTML of RFCs.
>
> It asks for the meta tags that Google Scholar uses. I know these are 
> the right tags because I added them to the HTML-ized versions of older 
> RFCs which are now at long last showing up in Google Scholar.
>
> It asks for Open Graph metadata (see https://ogp.me/) which makes 
> links display nicely on Facebook, Twitter, some Wordpress sites, and 
> probably other places.  The should work also for the HTML rendered 
> versions of I-D's.
>
> Are there other meta tags that people would find useful?  Do we agree 
> that we want the OG tags?  We already agreed on the Google Scholar 
> tags about a decade ago but it took a while to get around to it.
>