Subject: [precis] algorithm order (esp. HasCompat)

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 26 Jun 2013 21:15:01 -0600

I've made good progress on some Python code for generating the PRECIS
codepoint table. While working on the HasCompat part of the code today,
it struck me that our current algorithm for calculation of the derived
property might be wrong:

   If .cp. .in. Exceptions Then Exceptions(cp);
   Else If .cp. .in. BackwardCompatible Then BackwardCompatible(cp);
   Else If .cp. .in. Unassigned Then UNASSIGNED;
   Else If .cp. .in. ASCII7 Then PVALID;
   Else If .cp. .in. JoinControl Then CONTEXTJ;
   Else If .cp. .in. PrecisIgnorableProperties Then DISALLOWED;
   Else If .cp. .in. Controls Then DISALLOWED;
   Else If .cp. .in. OldHangulJamo Then DISALLOWED;
   Else If .cp. .in. LetterDigits Then PVALID;
   Else If .cp. .in. OtherLetterDigits Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Spaces Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Symbols Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Punctuation Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. HasCompat Then SAFE_DIS or FREE_PVAL;
   Else DISALLOWED;

In particular, I don't think we can check the HasCompat rule last,
because that will allow too many codepoints to be accepted in the
IdentifierClass. Instead, I think it needs to be before LetterDigits in
the order of operations. Here are a few examples to back up my claim.

Consider the first codepoint that has a compatibility equivalent:

00AA;FEMININE ORDINAL INDICATOR;Ll;0;L;<super> 0061;;;;N;;;;;

If LetterDigits is checked before HasCompat, this character (which is a
superscript version of LATIN SMALL LETTER A) would be allowed in the
Identifier Class.

Or consider 00BC:

00BC;VULGAR FRACTION ONE QUARTER;No;0;ON;<fraction> 0031 2044
0034;;;1/4;N;FRACTION ONE QUARTER;;;;

If OtherLetterDigits is checked before HasCompat, this character (¼)
would be allowed in the IdentifierClass.

There are many more examples. Unless I'm horribly mistaken, the order of
checking in the algorithm needs to be corrected. I propose:

   If .cp. .in. Exceptions Then Exceptions(cp);
   Else If .cp. .in. BackwardCompatible Then BackwardCompatible(cp);
   Else If .cp. .in. Unassigned Then UNASSIGNED;
   Else If .cp. .in. ASCII7 Then PVALID;
   Else If .cp. .in. JoinControl Then CONTEXTJ;
   Else If .cp. .in. PrecisIgnorableProperties Then DISALLOWED;
   Else If .cp. .in. Controls Then DISALLOWED;
   Else If .cp. .in. OldHangulJamo Then DISALLOWED;
   Else If .cp. .in. HasCompat Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. LetterDigits Then PVALID;
   Else If .cp. .in. OtherLetterDigits Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Spaces Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Symbols Then SAFE_DIS or FREE_PVAL;
   Else If .cp. .in. Punctuation Then SAFE_DIS or FREE_PVAL;
   Else DISALLOWED;

(And yes, "SAFE_DIS" needs to be changed to "ID_DIS"...)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/