Subject: automated id submission tool munging the abstract?

From: Eliot Lear <[email-masked]>
Date: Wed, 20 Jan 2010 16:19:23 +0100

Hi everyone,

I've submitted two drafts recently, and in both cases, the submission
tool runs over the end of the abstract and into Status of this memo.  Is
that intentional?  The two drafts in question are draft-lear-lisp-nerd
and draft-lear-ietf-sasl-openid.  This punts the thing to a manual
submission.

Eliot

---

From: Bernie Hoeneisen <[email-masked]>
Date: Wed, 20 Jan 2010 16:24:23 +0100 (CET)

Hi Eliot

I run into the same issue recently and I was choosing to correct this 
leading to manual submission. The reason for this problem lies in the new 
ordering of the I-D sections (the Abstract jumped over the boilerplate 
text).

cheers,
  Bernie

---

From: Tony Hansen <[email-masked]>
Date: Wed, 20 Jan 2010 10:38:28 -0500

Which leads to the question as to why the xml isn't being consulted to 
extract the abstract, instead of using heuristics on the text version?

	Tony Hansen
	[email-masked]

Bernie Hoeneisen wrote:
> leading to manual submission. The reason for this problem lies in the 
> new ordering of the I-D sections (the Abstract jumped over the 
> boilerplate text).
> 
> cheers,
>  Bernie
>

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 20 Jan 2010 10:50:20 -0500

So, are we now required to use the xml tools even though there has been no
announcement to that effect?

I continue to use nroff for all my draft and have always had this problem
because the submission tools has always, as far as I know, failed to have
the seeming obvious heuristic of being able to detect the end of abstract by
some number (2, 3, 4?) of blank lines.

Donald
=============================
Donald E. Eastlake 3rd   +1-508-634-2066 (home)
155 Beaver Street
Milford, MA 01757 USA
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 20 Jan 2010 10:57:57 -0500

> announcement to that effect?

I guess we'd better hope not, since xml2rfc is now unsupported and is
already drifting from being useful without postprocessing.

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Tom Taylor <[email-masked]>
Date: Wed, 20 Jan 2010 11:04:53 -0500

On that, I can give some good news. A few of us have taken on the job of 
support. We're just getting started, but the necessary directory and tracking 
tools have been put in place.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 20 Jan 2010 10:06:14 -0600

Hi, Donald,

I would actually be delighted to head off to the lang of XML, but since not everyone is, we might respin Tony's question as "why not extract the tagged information from the XML *if the XML is available*?"

"some number of blank lines" is a less robust heuristic than <abstract> stuff </abstract> ;-) - keeping in mind that the original complaint was information that is clearly tagged in the XML being reordered, so that it's not where the heuristic-based application was expecting to see it - and that's not a problem if you extract from the XML.

Continuing to extract from TXT if no XML is available would not work any worse than extracting from TXT does for everyone today, of course.

Thanks,

Spencer

  ----- Original Message ----- 
  From: Donald Eastlake 
  To: Tony Hansen 
  Cc: Working Group Chairs ; Henrik Levkowetz 
  Sent: Wednesday, January 20, 2010 9:50 AM
  Subject: Re: automated id submission tool munging the abstract?

  So, are we now required to use the xml tools even though there has been no announcement to that effect?

  I continue to use nroff for all my draft and have always had this problem because the submission tools has always, as far as I know, failed to have the seeming obvious heuristic of being able to detect the end of abstract by some number (2, 3, 4?) of blank lines.

  Donald

    Which leads to the question as to why the xml isn't being consulted to extract the abstract, instead of using heuristics on the text version?

           Tony Hansen
           [email-masked]

    Bernie Hoeneisen wrote:

      Hi Eliot

      I run into the same issue recently and I was choosing to correct this leading to manual submission. The reason for this problem lies in the new ordering of the I-D sections (the Abstract jumped over the boilerplate text).

      cheers,
       Bernie

        Hi everyone,

        Eliot

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 20 Jan 2010 17:13:26 +0100

Hi Eliot,

> and draft-lear-ietf-sasl-openid.  This punts the thing to a manual

It's not intentional, the tool is old, broken, and the intention is to
replace it as soon as possible with a rewritten version.  That said, this
known bug seems to have become so prevalent with the changed placement of the
abstract that we may have to deal with the pain of trying to fix the old
code in this respect.

I'll coordinate with Glen/AMS about this.

Regards,

	Henrik

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 20 Jan 2010 11:19:36 -0500

On Wed, Jan 20, 2010 at 11:04:53AM -0500, Tom Taylor wrote:
> support. We're just getting started, but the necessary directory and 
> tracking tools have been put in place.

Fantastic news!  Thanks!

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Tony Hansen <[email-masked]>
Date: Wed, 20 Jan 2010 11:34:16 -0500

Not at all. But if one is present, then it can be used.

I've seen more problems with the heuristics than just this one example. 
When a source file is present that uses semantic markup, then we should 
be able to use it to extract exact text.

If the nroff source files used more semantic-oriented markup, then they 
could be used as well. Unfortunately, the nroff source I've written and 
seen tends to be just as opaque semantically as the text output.

	Tony Hansen
	[email-masked]

Donald Eastlake wrote:
> I continue to use nroff for all my draft and have always had this 
> problem because the submission tools has always, as far as I know, 
> failed to have the seeming obvious heuristic of being able to detect the 
> end of abstract by some number (2, 3, 4?) of blank lines.
> 
> 
> 
> On Wed, Jan 20, 2010 at 10:38 AM, Tony Hansen <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Which leads to the question as to why the xml isn't being consulted
>     to extract the abstract, instead of using heuristics on the text
>     version?
> 
>            Tony Hansen
>            [email-masked] <mailto:[email-masked]>
> 
> 
>     Bernie Hoeneisen wrote:
> 
>         Hi Eliot
> 
>         I run into the same issue recently and I was choosing to correct
>         this leading to manual submission. The reason for this problem
>         lies in the new ordering of the I-D sections (the Abstract
>         jumped over the boilerplate text).
> 
>         cheers,
>          Bernie
>

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 21 Jan 2010 10:54:12 +0100

An update:

> I'll coordinate with Glen/AMS about this.

I've patched the old code, and Glen has installed it on datatracker.ietf.org.
It seems to do a much better job of extracting the abstract.

Best,

	Henrik

---

From: Jari Arkko <[email-masked]>
Date: Thu, 21 Jan 2010 11:21:40 +0100

Henrik,

> It seems to do a much better job of extracting the abstract.

Thank you.

Jari

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 21 Jan 2010 09:18:11 -0800

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Spencer Dawkins
> Sent: Wednesday, January 20, 2010 8:06 AM
> To: Donald Eastlake; Tony Hansen
> Cc: Working Group Chairs; Henrik Levkowetz
> I would actually be delighted to head off to the lang of XML, 
> but since not everyone is, we might respin Tony's question as 
> "why not extract the tagged information from the XML *if the 
> XML is available*?"
>  
> "some number of blank lines" is a less robust heuristic than 
> <abstract> stuff </abstract> ;-) - keeping in mind that the 
> original complaint was information that is clearly tagged in 
> the XML being reordered, so that it's not where the 
> heuristic-based application was expecting to see it - and 
> that's not a problem if you extract from the XML.
>  
> Continuing to extract from TXT if no XML is available would 
> not work any worse than extracting from TXT does for everyone 
> today, of course.

I have been using two sed scripts to extract abstracts for my
own purposes.  They worked with the old ordering and the new
ordering, without modification.  Maybe someone will find them 
useful.  Example command line:

   cat FILENAME | sed -f abstract.sed | sed -f tighten.sed

abstract.sed and tighten.sed are in
ftp://ftpeng.cisco.com/dwing/abstract.zip

-d

> 	From: Donald Eastlake <mailto:[email-masked]>  
> 	To: Tony Hansen <mailto:[email-masked]>  
> 	Cc: Working Group Chairs <mailto:[email-masked]>  ; 
> Henrik Levkowetz <mailto:[email-masked]>  
> 	So, are we now required to use the xml tools even 
> though there has been no announcement to that effect? 
> 
> 	I continue to use nroff for all my draft and have 
> always had this problem because the submission tools has 
> always, as far as I know, failed to have the seeming obvious 
> heuristic of being able to detect the end of abstract by some 
> number (2, 3, 4?) of blank lines.
> 
> 	Donald
> 		Which leads to the question as to why the xml 
> isn't being consulted to extract the abstract, instead of 
> using heuristics on the text version?
> 		
> 		       Tony Hansen
> 		       [email-masked] 
> 
> 
> 		Bernie Hoeneisen wrote:
> 		
> 
> 			Hi Eliot
> 			
> 			I run into the same issue recently and 
> I was choosing to correct this leading to manual submission. 
> The reason for this problem lies in the new ordering of the 
> I-D sections (the Abstract jumped over the boilerplate text).
> 			
> 			cheers,
> 			 Bernie
> 			
> 				I've submitted two drafts 
> recently, and in both cases, the submission
> 				tool runs over the end of the 
> abstract and into Status of this memo.  Is
> 				that intentional?  The two 
> drafts in question are draft-lear-lisp-nerd
> 				and 
> draft-lear-ietf-sasl-openid.  This punts the thing to a manual
> 				submission.
> 				
> 				Eliot
> 				
> 				
> 
> 
>