Subject: I-D Action: draft-koster-rep-02.txt

From: [email-masked]
Date: Tue, 09 Jun 2020 13:41:30 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : Robots Exclusion Protocol
        Authors         : Martijn Koster
                          Gary Illyes
                          Henner Zeller
                          Lizzi Harvey
	Filename        : draft-koster-rep-02.txt
	Pages           : 10
	Date            : 2020-06-09

Abstract:
   This document standardizes and extends the "Robots Exclusion
   Protocol" <http://www.robotstxt.org/> method originally defined by
   Martijn Koster in 1996 for service owners to control how content
   served by their services may be accessed, if at all, by automatic
   clients known as crawlers.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-koster-rep/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-koster-rep-02
https://datatracker.ietf.org/doc/html/draft-koster-rep-02

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-koster-rep-02

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/