Subject: [alto] Adding explanation for E_SYNTAX error

From: Wendy Roome <[email-masked]>
Date: Thu, 10 Oct 2013 12:50:41 -0400

As current;y defined, E_SYNTAX gives no indication as to where the error
is or what was wrong. So how about adding the following additional fields
for E_SYNTAX:

"line"  Optional line number (first line is 1)
"character"  Optional character in line (first character is 1)
"reason"  Optional error message, which may or may not include line and
character.

This raises the question of what language to use for "reason". At the risk
of coming across as an English chauvinist, I suggest that it be in
English. After all, the RFC is in English, and the field names are
English. So it's pretty unlikely that server and client programmers won't
understand enough English to deal with a simple error message. And I don't
see a client library displaying that message to the end user; instead the
lib would log it for analysis by the developers.

And if a server isn't able to give an English error message, use "line"
and "character".

	- Wendy Roome

---

From: "Y. Richard Yang" <[email-masked]>
Date: Fri, 11 Oct 2013 20:54:57 -0400

Hi Wendy,

Interesting suggestion. I assume that the fields will be optional, right? A
consideration is that the syntax errors typically are caught by a JSON
parser. I know that you know quite a few parsers. Will the existing parsers
give such information (e.g., line, character)?

To others: if you have used multiple parsers, your experience can be quite
helpful here.

Thanks!

Richard

On Thu, Oct 10, 2013 at 12:50 PM, Wendy Roome <[email-masked]>wrote:

> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto
>

---

From: Richard Alimi <[email-masked]>
Date: Sat, 12 Oct 2013 10:43:47 -0700

We used to have a 'reason' field, but it was suggested in the the apps-area
review that it be removed because of the concern over internationalization.
 Given that it's WGLC I would prefer not to revisit that.

I don't have a problem with adding optional line and character fields,
given that they can be conveyed as integers.  I don't think it should be
required, since not all parsers may provide that information.

Rich

On Fri, Oct 11, 2013 at 5:54 PM, Y. Richard Yang <[email-masked]> wrote:

> Hi Wendy,
>
> Interesting suggestion. I assume that the fields will be optional, right?
> A consideration is that the syntax errors typically are caught by a JSON

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 14 Oct 2013 00:14:41 -0400

I am fine with adding two optional fields. Regarding an optional
"character" field, the suggested semantics is the character position at the
line. To avoid ambiguity, how about name them "line-number" and
"character-number" so that their JSONNumber type is not surprising?

Richard

On Sat, Oct 12, 2013 at 1:43 PM, Richard Alimi <[email-masked]> wrote:

> We used to have a 'reason' field, but it was suggested in the the
> apps-area review that it be removed because of the concern over
> internationalization.  Given that it's WGLC I would prefer not to revisit
> that.
>
>> To others: if you have used multiple parsers, your experience can be
>> quite helpful here.
>>
>> Thanks!
>>
>> Richard
>>
>>
>> On Thu, Oct 10, 2013 at 12:50 PM, Wendy Roome <[email-masked]
>> > wrote:
>>

---

From: Richard Alimi <[email-masked]>
Date: Sun, 13 Oct 2013 21:46:02 -0700

Actually, how about just the character position in the input stream?  That
would avoid any complexities of different line-ending styles.  It could
simply be named "position".

On Sun, Oct 13, 2013 at 9:14 PM, Y. Richard Yang <[email-masked]> wrote:

>>> Interesting suggestion. I assume that the fields will be optional,
>>> right? A consideration is that the syntax errors typically are caught by a
>>> JSON parser. I know that you know quite a few parsers. Will the existing
>>> parsers give such information (e.g., line, character)?
>>>
>>> On Thu, Oct 10, 2013 at 12:50 PM, Wendy Roome <
>>> [email-masked]> wrote:
>>>

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 14 Oct 2013 00:54:40 -0400

Interesting thought. It appears a json parser, from what I used, will more
likely give only the line number. For example, try
http://jsonlint.com/

It typically gives a line number, and then some message on what is
expected. Hence, line # might be more likely available?

Richard

On Monday, October 14, 2013, Richard Alimi wrote:

> On Sun, Oct 13, 2013 at 9:14 PM, Y. Richard Yang <[email-masked]<javascript:_e({}, 'cvml', [email-masked]');>
> > wrote:
>
>> On Sat, Oct 12, 2013 at 1:43 PM, Richard Alimi <[email-masked]<javascript:_e({}, 'cvml', [email-masked]');>
>> > wrote:
>>
>>> On Fri, Oct 11, 2013 at 5:54 PM, Y. Richard Yang <[email-masked]<javascript:_e({}, 'cvml', [email-masked]');>
>>> > wrote:
>>>
>>>> Hi Wendy,
>>>>
>>>> [email-masked] <javascript:_e({}, 'cvml',
>>>> [email-masked]');>> wrote:
>>>>
>>>>> [email-masked] <javascript:_e({}, 'cvml', [email-masked]');>
>>>> [email-masked] <javascript:_e({}, 'cvml', [email-masked]');>

---

From: Wendy Roome <[email-masked]>
Date: Mon, 14 Oct 2013 11:25:22 -0400

Yes, I'm aware that we dropped "reason" a while back.

But we dropped it because the original "reason" covered all sorts of errors
-- in particular, transient errors on the server. The problem was that an
ALTO library might want to display that "reason" to the end user. That was
clearly inappropriate.

However, my proposal is different. This only happens because of a
programming error in the client library. An ALTO library would never display
the "reason" error message to the end-user. Instead, the ALTO lib would log
the "reason" somewhere, for later examination by the developers. The ALTO
lib would tell the user "Opps, internal error, send us the log file" , in
the user's language.

In short, most JSON parsers generate useful error messages for syntax
errors. It seems like a shame to discard information and tell a developer
"Something's wrong. Guess where."

Back in the 1970's, the original unix text editor, "ed", had one error
message: "?".  I'd like to think we can do better than that.

- Wendy Roome

From:  Richard Alimi <[email-masked]>
Date:  Sat, October 12, 2013 13:43
To:  "Y. Richard Yang" <[email-masked]>
Cc:  Wendy Roome <[email-masked]>, IETF ALTO <[email-masked]>
Subject:  Re: [alto] Adding explanation for E_SYNTAX error

I don't have a problem with adding optional line and character fields, given
that they can be conveyed as integers.  I don't think it should be required,
since not all parsers may provide that information.

Rich

On Fri, Oct 11, 2013 at 5:54 PM, Y. Richard Yang <[email-masked]> wrote:
> consideration is that the syntax errors typically are caught by a JSON parser.
> I know that you know quite a few parsers. Will the existing parsers give such
> information (e.g., line, character)?
>

---

From: Wendy Roome <[email-masked]>
Date: Mon, 14 Oct 2013 12:41:46 -0400

The JSON lib I'm currently using gives reasonably useful syntax error
messages, complete with line number and character offset in line. Of course,
as with any syntax error, that's approximate.

However, that position info is buried inside the library, and I'd have to do
a lot of surgery to the library to get line & offset as separate fields.
It's possible, of course -- it's just annoying.

So how about dropping the separate line & character fields, and simplifying
my original proposal to this:

"For an E_SYNTAX error, the ALTO server MAY provide the optional string
field "syntax-error" with a human-readable description of the syntax error.
If possible, this field should include position information such as line
number and offset within line. If nothing else, "syntax-error" could have
just the position. The format of "syntax-error" field is free-form. The
contents of that field should be given to the ALTO client developer, not the
end-user. If a syntax error occurs in a production environment, the ALTO
client should  log the "syntax-error" in an internal log file, tell the
end-user that there was an error communicating with the ALTO server, and
suggest the user forward that log file to the developers."

- Wendy Roome

From:  "Y. Richard Yang" <[email-masked]>
Date:  Mon, October 14, 2013 00:54
To:  Richard Alimi <[email-masked]>
Cc:  "Y. Richard Yang" <[email-masked]>, Wendy Roome
<[email-masked]>, IETF ALTO <[email-masked]>
Subject:  Re: [alto] Adding explanation for E_SYNTAX error

It typically gives a line number, and then some message on what is expected.
Hence, line # might be more likely available?

Richard

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 14 Oct 2013 13:16:32 -0400

Hi Wendy,

On Mon, Oct 14, 2013 at 12:41 PM, Wendy Roome <[email-masked]>wrote:

> messages, complete with line number and character offset in line. Of
> course, as with any syntax error, that's approximate.
>
> However, that position info is buried inside the library, and I'd have to
> do a lot of surgery to the library to get line & offset as separate fields.
I had similar experiences as well.

> So how about dropping the separate line & character fields, and
> simplifying my original proposal to this:
>
> contents of that field should be given to the ALTO client developer, not
> the end-user. If a syntax error occurs in a production environment, the
> ALTO client should  log the "syntax-error" in an internal log file, tell
> the end-user that there was an error communicating with the ALTO
> server, and suggest the user forward that log file to the developers."
>
> A free form string field may turn out to be very useful, but lose
information. For example, try
{
    "meta": {
        "vtag :

on jsonlint.com, and one gets the error message:

Parse error on line 2:
...  "meta": {        "vtag :
----------------------^
Expecting 'STRING', '}'

It will be hard to convey this formatting, but actually the text gives much
information to a developer. A problem is that this is essentially the
"reason" field as Richard A. mentioned, and then we have the
internationalization problem....

Richard

- Wendy Roome
> Cc: "Y. Richard Yang" <[email-masked]>, Wendy Roome <
> [email-masked]>, IETF ALTO <[email-masked]>
>

---

From: Wendy Roome <[email-masked]>
Date: Mon, 14 Oct 2013 14:51:06 -0400

The ALTO java lib I downloaded a while back gives errors like:
      Parsing error: Expected a ':' after a key at 10 [character 1 line 3]
"10" is offset in file.

As for internationalization problems, my proposed "syntax-error" field is
very different from the "reason" field proposed a year or so ago.

The original "reason" field gave the explanation for all types of errors.
The problem is that the ALTO client would like to display that to the end
user to explain what went wrong. But if that were in a language the end user
didn't understand, it was useless.

However "syntax-error" has a very limited audience. The contents only need
to make sense to the ALTO client developer. Just tell the end user, "Oops,
internal error."

As for internationalization problems, why aren't the field names also a
problem? They're all in english.  And HTTP seems to survive even though it
has english header names and status codes. Ditto for HTML, SIP, etc, etc.

- Wendy Roome

From:  "Y. Richard Yang" <[email-masked]>
Date:  Mon, October 14, 2013 13:16
To:  Wendy Roome <[email-masked]>
Cc:  Richard Alimi <[email-masked]>, IETF ALTO <[email-masked]>
Subject:  Re: [alto] Adding explanation for E_SYNTAX error

A problem is that this is essentially the "reason" field as Richard A.
mentioned, and then we have the internationalization problem....

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 14 Oct 2013 16:02:37 -0400

Hi Wendy,

On Mon, Oct 14, 2013 at 2:51 PM, Wendy Roome <[email-masked]>wrote:

> Now I see where your design comes from. We used some different parsers and
hence different design.

> user to explain what went wrong. But if that were in a language the end
> user didn't understand, it was useless.
>
> has english header names and status codes. Ditto for HTML, SIP, etc, etc.
Distinguishing messages between those to developers and those to end users
is important. To avoid issues with the review (the problem with "reason" as
Richard A. mentioned), let's add a sentence to emphasize that the
syntax-error text is technical, for developer debugging purposes.

Thanks.

Richard

---

From: Wendy Roome <[email-masked]>
Date: Mon, 14 Oct 2013 16:29:39 -0400

Sounds perfect. And by all means, name the field "syntax-error". The name
"reason" would just be a red flag. I must have been out-to-lunch when I
suggested that name originally.

- Wendy

Date:  Mon, October 14, 2013 16:02

Thanks.

Richard