Subject: I-D Action: draft-koster-rep-06.txt

From: [email-masked]
Date: Sun, 07 Nov 2021 04:28:19 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : Robots Exclusion Protocol
        Authors         : Martijn Koster
                          Gary Illyes
                          Henner Zeller
                          Lizzi Harvey
	Filename        : draft-koster-rep-06.txt
	Pages           : 10
	Date            : 2021-11-07

Abstract:
   This document specifies and extends the "Robots Exclusion Protocol"
   method originally defined by Martijn Koster in 1996 for service
   owners to control how content served by their services may be
   accessed, if at all, by automatic clients known as crawlers.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-koster-rep/

There is also an htmlized version available at:
https://datatracker.ietf.org/doc/html/draft-koster-rep-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-koster-rep-06

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/