Subject: Re: [ire] DNRD CSV Draft

From: Gustavo Lozano <[email-masked]>
Date: Thu, 1 Nov 2012 17:16:26 -0700

Chris, all,

We value your suggestions regarding the new gTLD program;
please send your suggestions, questions or comments regarding the escrow
requirements
present in the AGB or the registry transition process to [email-masked].

The objective of the dndrd-objects-mapping draft is to
create a specification where the common elements and objects of domain
registries are listed. If you believe that some element, attribute or
object is
missing please provide feedback on the list.

In order to advance in the development of this
specification, please provide feedback regarding the objects, elements and
attributes described in the dndrd-objects-mapping-draft.

The main update of dndrd-objects-mapping version 01 is
regarding IDN variant handling. ICANN have been working on the IDN variants
project for some time and knowledge gathered from this project has been
incorporated into the draft. The proposed approach in the draft supports
variants of type explicit (listed as objects) and implicit (computed based
on a
LGR).

Remember that this specification is being developed not only
for new gTLDs and it might be used by ccTLDs or current gTLDs to escrow
its SRS
data.

Regards,
Gustavo Lozano

On 10/31/12 9:55 PM, "Chris Wright" <[email-masked]> wrote:

>Thanks Francisco,
>
>I 100% agree with what your saying, from the perspective of that is what
>the agreement currently says.
>
>However I am arguing that the requirements from that agreement are silly
>and flawed, and that if we the technical community go back to ICANN and
>tell them as such we can get that addressed.
>
>It makes no sense to enforce a technical process like this in a legal
>agreement, including down to the level of how to name files. All ICANN
>needs to care about is how they get the data from the escrow provider.
>
>The validation currently asked for by the legal agreement, which as you
>rightly point our is trivial to do, is also almost pointless, schema
>validation proves nothing except that I submitted schema valid XML, I
>could fill schema valid XML with anything - it will not meet the
>objectives of ensuring data is true and accurate based on what is actually
>in my registry.
>
>Im fine with the escrow providing having to write custom stuff to validate
>data, they will need to to do 'real' validation, however again I wouldn't
>go to that level from ICANN perspective, I would specify a set of things
>the escrow provider needs to validate, and leave the 'how' to them.
>
>If we are serious about validation we should validate things like,
>
>Ensure every contact object referenced by a domain object is present
>Ensure every host object reference by a domain object is present
>Ensure that every sponsor referenced by domains, contacts and host is
>present as a registrar object
>Ensure every contact country code matches a country code on the ISO list
>
>On and on...
>Etc.. - we probably want to apply some checks to object properties as well
>(e.g. Check email address are valid (syntactically))
>
>Perhaps statistical analysis of object counts could be done, compared to
>previous counts and anomalies investigated (if one day I have a million
>domains, and then the next day I have 100,000 then the escrow provider
>should look deeper into the data and have a discussion with the registry
>to work out what went wrong, or if the massive drop is legit) - Escrow
>providers should be more than just file repositories, otherwise they add
>no value whatsoever, and ICANN might as well save us all some money and
>just setup a central FTP server we all send the dump files to.
>
>The validation we could(should?) do/expect goes on and on, only then will
>we actually be more confident that we have better data - the thing is to
>do these types of validation, you would need to reintegrate an incremental
>file with a full file (as the incremental file may reference objects from
>the full dump (or a previous incremental) so the escrow provider will need
>to do more than schema validate a file anyway.
>
>Obviously if the method I worked out between me and my Escrow provider was
>database replication, they would need to ensure that if I replicated a
>'drop database' statement to them, they would still have the data from the
>previous replication, but again, that is a technical problem for the
>escrow provider to work out with the registry, and I can think of many
>technical ways to solve that problem.
>
>The escrow provider will have a contractual requirement to provide data to
>ICANN (in a format provided by ICANN - who will design the format in
>conjunction with the technical community) that is no more than 24 hours
>old, they will need to technically work out a way they can fulfil on that
>obligation with the registry operator - that¹s all ICANN needs to day. The
>escrow provider then needs a process where if they are not receiving the
>data from the registry operator in a timely manner (thus not able to
>fulfil on their obligation) or the registry operator is not doing
>something they need to do, then they escalate this issue to ICANN who will
>go the registry for contract breach - that¹s it. This situation exist if I
>stop sending daily full or incremental files anyway.
>
>Thanks
>
>Chris
>
>On 1/11/12 2:39 PM, "Francisco Obispo" <[email-masked]> wrote:
>
>>Hi Chris,
>>
>>Perhaps I'm reiterating what I said in my previous email, but I don't
>>think that's how the escrow provider is supposed to work.
>>
>>They are going to receive, verify and store an escrow dump from the
>>registry. Verify means going through the items in Specification 2 of the
>>Applicant's guidebook:
>>http://archive.icann.org/en/topics/new-gtlds/rfp-clean-30may11-en.pdf
>>
>><specification2_extract>
>>Verification Procedure.
>>
>>	€ (1)  The signature file of each processed file is validated.
>>
>>	€ (2)  If processed files are pieces of a bigger file, the latter is put
>>together.
>>
>>	€ (3)  Each file obtained in the previous step is then decrypted and
>>uncompressed.
>>
>>	€ (4)  Each data file contained in the previous step is then validated
>>against the format defined in [1].
>>
>>	€ (5)  If [1] includes a verification process, that will be applied at
>>this step.
>>
>>If any discrepancy is found in any of the steps, the Deposit will be
>>considered incomplete.
>>
>>[1] Domain Name Data Escrow Specification (work in progress),
>>http://tools.ietf.org/html/draft-arias-noguchi-registry-data-escrow
>>[2] OpenPGP Message Format, http://www.rfc-editor.org/rfc/rfc4880.txt
>>[3] OpenPGP parameters,
>>http://www.iana.org/assignments/pgp-parameters/pgp-parameters.xhtml
>>
>></specification2_extract>
>>
>>The validation is an XML Schema Validation based on the IETF Draft
>>"arias-noguchi". By passing XML Validation (which is a very simple
>>check), the Escrow provider can be sure that the data not only is
>>complete, but that the fields conform the data types specified by the
>>spec.
>>
>>Validating a dump wouldn't be more than just:
>>
>>$ xmllint --noout --sax --schema /path/to/arias-noguchi.xsd
>>escrow_dump.xml
>>
>>and if the file is compressed:
>>
>>$ bzcat escrow_dump.xml.bz2 | xmllint --noout --sax --schema
>>/path/to/arias-noguchi.xsd -
>>
>>(that's an example using libxml2)
>>
>>Having an CSV file would require the escrow provider write their own
>>parser to validate comma separated values.. This is very dangerous, and
>>time consuming, depending on how they do it, specially if they have to
>>cope with custom formats from different registries.
>>
>>If I was an escrow provider I wouldn't want to be liable in generating
>>the dump file, because if something is missing, and the registry goes
>>away, or a DELETE, TRUNCATE TABLE, DROP DATABASE, or anything like it
>>happens and the replica throws the data away, there will be nowhere to
>>fetch the data from (backups perhaps?)
>>
>>On the variants issue, this would be part of the extension framework, and
>>since this varies from registry to registry, it would be something that
>>it would be have to be mapped and documented accordingly.
>>
>>Best regards,
>>
>>Francisco
>>
>>On Oct 31, 2012, at 7:17 PM, Chris Wright <[email-masked]>
>>wrote:
>>
>>> Understood, and if that¹s the way Verisign wants to do it, then go for
>>>it,
>>> you can deposit it in the format required to be provided to ICANN, no
>>> issue what so ever, but why should everyone be forced to do that? It
>>>would
>>> actually be really trivial for both me and my provider to do database
>>> replication, and then they simply run a script to validate the data
>>>(that
>>> I would supply them) - Im yet to actually see what the 'validation'
>>>they
>>> are supposed to do actually is - if its just object counts - then I
>>>would
>>> assert that dumping the data doesn't even really need to be done, but
>>>even
>>> it it does, they run the dump scripts and get the data in the format
>>> needed - not hard at all and completely automated.
>>
>>Francisco Obispo 
>>Director of Applications and Services - ISC
>>email: [email-masked]
>>Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
>>PGP KeyID = B38DB1BE
>>
>
>_______________________________________________
>ire mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/ire

---

From: Chris Wright <[email-masked]>
Date: Fri, 2 Nov 2012 15:01:21 +1100

Thanks Gustavo,

I will raise my issues with ICANN as you suggest.

I have given some feedback already on the draft, that being that I concur
with the others here who have said they prefer CSV format, I have
suggested a different approach to the CSVs.

On your specific draft, I am very confused by how it is attempting to
handle IDNs, this is especially difficult as none of the examples in the
document show IDN examples, so that may help.

Can you elaborate more on the proposed handling of IDNs, I am especially
concerned about how registries who don't generate and store all variants
are supported? (Registries that simply use an algorithm to block names
from others, and match names for the registrant to be able to add as
'provisioned' variants to the domain object).

I feel that the fastest way to get this sorted would be to get the
registries in the room and crank it out - for such a specific use Im not
sure that it really needs an IETF working group (even though this isn't
really a working group).

ICANN seems to have a very flawed understanding of what Informational RFCs
are, as apposed to standards track RFCs. This is very evident in the
reference to informational RFCs in the pre delegation testing process
(especially around language tables). I fear that by constructing this
'specification' under the 'illusion' of an IETF consensus process, will
only further entrench this misunderstanding.

Thanks

Chris

---

From: Gustavo Lozano <[email-masked]>
Date: Fri, 2 Nov 2012 12:37:22 -0700

Hello Chris,

Comments in line.

>suggested a different approach to the CSVs.

GL. The feedback, that you provided us with, is incorporated in version 01
of dnrd-objects-mapping, thank you for the feedback.

>'provisioned' variants to the domain object).

GL. The approach proposed in dnrd-objects-mapping-01 is:

A NNDN object is proposed which basically represent a label (not a full
domain object).

NNDNs could be used to store data of what is commonly called "reserved
domain names". Explicit IDN variants could be represented using NNDNs.

A registrant could register "cafe", and the registry could create explicit
IDN variants (café, cáfe or cáfé) as NNDN objects.

Implicit variants (used by registries who don't generate and store all
variants) are specified based on a canonical name.  In the "cafe" domain
object, you could specify the <variantGenerator> element and
<idnTableId>es</idnTableId> to indicate that IDN variants should be
created following the LGR defined in the "es" IDN table. If the LGR format
of the "es" IDN table doesn't support to define the state of the generated
IDN variants then you could specify it using a defaultState attribute of
the <variantGenerator> element.

The NNDN objects and implicit IDN variants could exist in three states:
  * blocked, undesirable for registration
  * withheld, could be registered.
  * allocated, mirror the information from the canonical domain name.

This approach covers the most complex scenario that we have knowledge of:
A TLD that supports multiple scripts, including LGR that could potential
generate thousands of variants, and some of those algorithmically
generated variants should not be available for registration.

>sure that it really needs an IETF working group (even though this isn't

GL. In this mailing list, there are participants from ccTLDs, new gTLDs,
current gTLDs, registrars, independent developers and many other parties
that can contribute with their expertise.

This specification might be used not only by new gTLDs, therefore it
appears that this mailing list is the correct venue to discuss this
specification.

If not this list, what is the process that you propose to discuss the data
escrow specification?

Thank you,

Regards,
Gustavo Lozano

---

From: Chris Wright <[email-masked]>
Date: Mon, 5 Nov 2012 12:29:10 +1100

>> If not this list, what is the process that you propose to discuss the data escrow specification?

I understand that your desire is that this 'might' be used for ccTLDs as well as gTLDs, but given the timeframes we are working to and the fact that this isn’t a proper working group I would suggest just focusing on the problem at hand. New gTLDs needing to escrow data. Given that I would humbly suggest that the fastest way to solve this is to run a technical summit inviting the new gTLD registry operators to 'nut this out' in a one or two day working session - put the thing out for comment for a few weeks, revise based on comments - then go with it - much like what we should have done for the TMCH solution.

This will be faster and get us an answer in a much quicker timer frame. Im willing to bet you are about to have all the right people in LA for the TMCH implementation meeting coming up on the 15/16 perhaps we can just add on the 17th (or the following Monday/Tuesday) to discuss this as well and get it over and done with?

Thanks

c.

-----Original Message-----
From: Gustavo Lozano [mailto:[email-masked]]
Sent: Saturday, 3 November 2012 6:37 AM
To: Chris Wright; Francisco Obispo
Cc: [email-masked]; Thippeswamy, Chethan
Subject: Re: [ire] DNRD CSV Draft

Hello Chris,

Comments in line.

On 11/1/12 9:01 PM, "Chris Wright" <[email-masked]> wrote:

>I have given some feedback already on the draft, that being that I
>concur with the others here who have said they prefer CSV format, I
>have suggested a different approach to the CSVs.

GL. The feedback, that you provided us with, is incorporated in version 01 of dnrd-objects-mapping, thank you for the feedback.

>handle IDNs, this is especially difficult as none of the examples in
>the document show IDN examples, so that may help.
>
>Can you elaborate more on the proposed handling of IDNs, I am
>especially concerned about how registries who don't generate and store
>all variants are supported? (Registries that simply use an algorithm to
>block names from others, and match names for the registrant to be able
>to add as 'provisioned' variants to the domain object).

A NNDN object is proposed which basically represent a label (not a full domain object).

NNDNs could be used to store data of what is commonly called "reserved domain names". Explicit IDN variants could be represented using NNDNs.

A registrant could register "cafe", and the registry could create explicit IDN variants (café, cáfe or cáfé) as NNDN objects.

Implicit variants (used by registries who don't generate and store all
variants) are specified based on a canonical name.  In the "cafe" domain object, you could specify the <variantGenerator> element and <idnTableId>es</idnTableId> to indicate that IDN variants should be created following the LGR defined in the "es" IDN table. If the LGR format of the "es" IDN table doesn't support to define the state of the generated IDN variants then you could specify it using a defaultState attribute of the <variantGenerator> element.

This approach covers the most complex scenario that we have knowledge of:
A TLD that supports multiple scripts, including LGR that could potential generate thousands of variants, and some of those algorithmically generated variants should not be available for registration.

>registries in the room and crank it out - for such a specific use Im
>not sure that it really needs an IETF working group (even though this
>isn't really a working group).

GL. In this mailing list, there are participants from ccTLDs, new gTLDs, current gTLDs, registrars, independent developers and many other parties that can contribute with their expertise.

This specification might be used not only by new gTLDs, therefore it appears that this mailing list is the correct venue to discuss this specification.

If not this list, what is the process that you propose to discuss the data escrow specification?

Thank you,

Regards,
Gustavo Lozano

>
>ICANN seems to have a very flawed understanding of what Informational
>RFCs are, as apposed to standards track RFCs. This is very evident in
>the reference to informational RFCs in the pre delegation testing
>process (especially around language tables). I fear that by
>constructing this 'specification' under the 'illusion' of an IETF
>consensus process, will only further entrench this misunderstanding.
>
>Thanks
>
>Chris
>
>>We value your suggestions regarding the new gTLD program; please send
>>your suggestions, questions or comments regarding the escrow
>>requirements present in the AGB or the registry transition process to
>>[email-masked].
>>
>>
>>The objective of the dndrd-objects-mapping draft is to create a
>>specification where the common elements and objects of domain
>>object is missing please provide feedback on the list.
>>
>>
>>In order to advance in the development of this specification, please
>>provide feedback regarding the objects, elements and attributes
>>described in the dndrd-objects-mapping-draft.
>>
>>
>>The main update of dndrd-objects-mapping version 01 is regarding IDN
>>variant handling. ICANN have been working on the IDN variants project
>>for some time and knowledge gathered from this project has been
>>incorporated into the draft. The proposed approach in the draft
>>supports variants of type explicit (listed as objects) and implicit
>>(computed based on a LGR).
>>
>>
>>Remember that this specification is being developed not only for new
>>gTLDs and it might be used by ccTLDs or current gTLDs to escrow its
>>SRS data.
>>
>>
>>
>>Regards,
>>Gustavo Lozano
>>
>>>I 100% agree with what your saying, from the perspective of that is
>>>what the agreement currently says.
>>>
>>>However I am arguing that the requirements from that agreement are
>>>silly and flawed, and that if we the technical community go back to
>>>ICANN and tell them as such we can get that addressed.
>>>
>>>agreement, including down to the level of how to name files. All
>>>ICANN needs to care about is how they get the data from the escrow provider.
>>>
>>>The validation currently asked for by the legal agreement, which as
>>>you rightly point our is trivial to do, is also almost pointless,
>>>schema validation proves nothing except that I submitted schema valid
>>>XML, I could fill schema valid XML with anything - it will not meet
>>>the objectives of ensuring data is true and accurate based on what is
>>>actually in my registry.
>>>
>>>Im fine with the escrow providing having to write custom stuff to
>>>validate data, they will need to to do 'real' validation, however
>>>again I wouldn't go to that level from ICANN perspective, I would
>>>specify a set of things the escrow provider needs to validate, and
>>>leave the 'how' to them.
>>>
>>>present as a registrar object Ensure every contact country code
>>>matches a country code on the ISO list
>>>
>>>On and on...
>>>Etc.. - we probably want to apply some checks to object properties as
>>>well (e.g. Check email address are valid (syntactically))
>>>
>>>Perhaps statistical analysis of object counts could be done, compared
>>>to previous counts and anomalies investigated (if one day I have a
>>>million domains, and then the next day I have 100,000 then the escrow
>>>provider should look deeper into the data and have a discussion with
>>>the registry to work out what went wrong, or if the massive drop is
>>>legit) - Escrow providers should be more than just file repositories,
>>>otherwise they add no value whatsoever, and ICANN might as well save
>>>us all some money and just setup a central FTP server we all send the dump files to.
>>>
>>>The validation we could(should?) do/expect goes on and on, only then
>>>will we actually be more confident that we have better data - the
>>>thing is to do these types of validation, you would need to
>>>reintegrate an incremental file with a full file (as the incremental
>>>file may reference objects from the full dump (or a previous
>>>incremental) so the escrow provider will need to do more than schema
>>>validate a file anyway.
>>>
>>>Obviously if the method I worked out between me and my Escrow
>>>provider was database replication, they would need to ensure that if
>>>I replicated a 'drop database' statement to them, they would still
>>>have the data from the previous replication, but again, that is a
>>>technical problem for the escrow provider to work out with the
>>>registry, and I can think of many technical ways to solve that
>>>problem.
>>>
>>>The escrow provider will have a contractual requirement to provide
>>>data to ICANN (in a format provided by ICANN - who will design the
>>>format in conjunction with the technical community) that is no more
>>>than 24 hours old, they will need to technically work out a way they
>>>can fulfil on that obligation with the registry operator - that¹s all
>>>ICANN needs to day.
>>>The
>>>escrow provider then needs a process where if they are not receiving
>>>the data from the registry operator in a timely manner (thus not able
>>>to fulfil on their obligation) or the registry operator is not doing
>>>something they need to do, then they escalate this issue to ICANN who
>>>will go the registry for contract breach - that¹s it. This situation
>>>exist if I stop sending daily full or incremental files anyway.
>>>
>>>Thanks
>>>
>>>Chris
>>>
>>>>Perhaps I'm reiterating what I said in my previous email, but I
>>>>don't think that's how the escrow provider is supposed to work.
>>>>
>>>>registry. Verify means going through the items in Specification 2 of
>>>>the Applicant's guidebook:
>>>>    € (2)  If processed files are pieces of a bigger file, the latter
>>>>is put together.
>>>>
>>>>    € (3)  Each file obtained in the previous step is then decrypted
>>>>and uncompressed.
>>>>
>>>>    € (4)  Each data file contained in the previous step is then
>>>>validated against the format defined in [1].
>>>>
>>>>    € (5)  If [1] includes a verification process, that will be applied
>>>>at this step.
>>>>
>>>>[2] OpenPGP Message Format,
>>>>http://www.rfc-editor.org/rfc/rfc4880.txt
>>>>[3] OpenPGP parameters,
>>>>complete, but that the fields conform the data types specified by
>>>>the spec.
>>>>
>>>>parser to validate comma separated values.. This is very dangerous,
>>>>and time consuming, depending on how they do it, specially if they
>>>>have to cope with custom formats from different registries.
>>>>
>>>>If I was an escrow provider I wouldn't want to be liable in
>>>>generating the dump file, because if something is missing, and the
>>>>registry goes away, or a DELETE, TRUNCATE TABLE, DROP DATABASE, or
>>>>anything like it happens and the replica throws the data away, there
>>>>will be nowhere to fetch the data from (backups perhaps?)
>>>>
>>>>On the variants issue, this would be part of the extension
>>>>framework, and since this varies from registry to registry, it would
>>>>be something that it would be have to be mapped and documented
>>>>accordingly.
>>>>
>>>>Best regards,
>>>>
>>>>Francisco
>>>>
>>>>> Understood, and if that¹s the way Verisign wants to do it, then go
>>>>>for it,  you can deposit it in the format required to be provided
>>>>>to ICANN, no  issue what so ever, but why should everyone be forced
>>>>>to do that? It would  actually be really trivial for both me and my
>>>>>provider to do database  replication, and then they simply run a
>>>>>script to validate the data (that  I would supply them) - Im yet to
>>>>>actually see what the 'validation'
>>>>>they
>>>>> are supposed to do actually is - if its just object counts - then
>>>>>I would  assert that dumping the data doesn't even really need to
>>>>>be done, but even  it it does, they run the dump scripts and get
>>>>>the data in the format  needed - not hard at all and completely
>>>>>automated.
>>>>
>>>>Francisco Obispo
>>>>Phone: +1 650 423 1374 || INOC-DBA *3557* NOC PGP KeyID = B38DB1BE
>>>>
>>>

---

From: "Gould, James" <[email-masked]>
Date: Mon, 5 Nov 2012 14:03:50 +0000

We could also consider meeting the day before on the 14th. If a set of
people are in LA, than I agree that it makes sense to have a technical
discussion on this.  Francisco, is this a possibility?

--
  
JG

James Gould
Principal Software Engineer
[email-masked]
 
703-948-3271 (Office)
12061 Bluemont Way
Reston, VA 20190
VerisignInc.com

On 11/4/12 8:29 PM, "Chris Wright" <[email-masked]> wrote:

>I understand that your desire is that this 'might' be used for ccTLDs as
>well as gTLDs, but given the timeframes we are working to and the fact
>that this isn’t a proper working group I would suggest just focusing on
>the problem at hand. New gTLDs needing to escrow data. Given that I would
>humbly suggest that the fastest way to solve this is to run a technical
>summit inviting the new gTLD registry operators to 'nut this out' in a
>one or two day working session - put the thing out for comment for a few
>weeks, revise based on comments - then go with it - much like what we
>should have done for the TMCH solution.
>
>This will be faster and get us an answer in a much quicker timer frame.
>Im willing to bet you are about to have all the right people in LA for
>the TMCH implementation meeting coming up on the 15/16 perhaps we can
>just add on the 17th (or the following Monday/Tuesday) to discuss this as
>well and get it over and done with?
>
>Thanks
>
>c.
>
>
>GL. The feedback, that you provided us with, is incorporated in version
>01 of dnrd-objects-mapping, thank you for the feedback.
>
>>
>A registrant could register "cafe", and the registry could create
>explicit IDN variants (café, cáfe or cáfé) as NNDN objects.
>
>
>created following the LGR defined in the "es" IDN table. If the LGR
>format of the "es" IDN table doesn't support to define the state of the
>generated IDN variants then you could specify it using a defaultState
>attribute of the <variantGenerator> element.
>
>