Subject: [ietf-provreg] new epp commands?

From: Ulrich Wisser <[email-masked]>
Date: Tue, 19 Jan 2010 12:41:48 +0100

Hi,

here at .SE many registrars asked for lists of their 
domains/hosts/contacts. Obviously there are no commands in the standard 
to extract lists of objects from the epp server.

Before proposing new commands I'd like to ask why there aren't any such 
commands? Has it been a deliberate decision or didn't just nobody ask 
for it?

/Ulrich
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
List run by majordomo software.  For (Un-)subscription and similar details
send "help" to [email-masked]

---

From: Jaap Akkerhuis <[email-masked]>
Date: Tue, 19 Jan 2010 13:08:39 +0100

My two cents in-line

These registrars don't have an administration on there own?

It has been deliberate not to turn EPP into yet another whois
protocol. Also, not all registries do have this data because not
all are "thick registries".

	jaap

---

From: InterNetX - Marco Schrieck <[email-masked]>
Date: Tue, 19 Jan 2010 13:21:37 +0100

Hi Ulrich,

for us as registrar it is very important to have such a list. We need
this to ensure that our system is working correct.

Most registries provide such lists via ftp or http. NASK (.PL) do this
via EPP commands. But if you have million of domains in an account
then the XML is very large and you may have problems with parsing it.
Or you limit the request and inquire the list step by step. This solution
has the problem that the list can changes while inquiring.

So for me the best way is an external list (with timestamp).

Regards
Marco

--
Marco Schrieck
Bereichsleiter Entwicklung

InterNetX GmbH
Maximilianstr. 6
93047 Regensburg
Germany

Tel: +49 941 59559-0
Fax: +49 941 59579-050

Geschäftsführer/CEO: Thomas Mörz
Amtsgericht Regensburg, HRB 7142

ICQ:     232016987
GPG-Key: 0xB44D5EAE
GPG-Fingerprint: 8285 3373 9776 E21A 94B1 38B1 2E52 D28C B44D 5EAE

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 19 Jan 2010 08:31:58 -0500

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Ulrich Wisser
> Sent: Tuesday, January 19, 2010 6:42 AM
> To: EPP Provreg
> Subject: [ietf-provreg] new epp commands?
> 
> Hi,
> 
> domains/hosts/contacts. Obviously there are no commands in 
> the standard to extract lists of objects from the epp server.
> 
> Before proposing new commands I'd like to ask why there 
> aren't any such commands? Has it been a deliberate decision 
> or didn't just nobody ask for it?

It was most definitely a deliberate decision.  The primary reason was
because many people felt that this type of reporting feature would be
out of place in a provisioning protocol due to the large amount of data
that could be returned.

Scott

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 19 Jan 2010 14:33:03 +0100

.SE is a thick registry, which means you have the data in reality both in the registry and the registar. Anything that can help ensuring the two databases are in sync is a good thing.

So yes, I would support such a command be standardized.

   Patrik

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 19 Jan 2010 08:50:30 -0500

> out of place in a provisioning protocol due to the large amount of data

Also, there was another IETF working group aiming to solve that
problem, IIRC.  CRISP produced IRIS.  I'm not aware of any
implementations, however.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Kurt Jaeger <[email-masked]>
Date: Tue, 19 Jan 2010 15:12:23 +0100

Hi!

> implementations, however.

DENIC has some code and docs on IRIS.

http://www.denic.de/domaincheck/domaincheck-hintergrund.html

-- 
MfG/Best regards, Kurt Jaeger                                  10 years to go !
Dr.-Ing. Nepustil & Co. GmbH  fon +49 7123 93006-0  [email-masked]  
Rathausstr. 3                 fax +49 7123 93006-99
72658 Bempflingen             mob +49 171 3101372
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
List run by majordomo software.  For (Un-)subscription and similar details
send "help" to [email-masked]

---

From: "Marc Groeneweg" <[email-masked]>
Date: Tue, 19 Jan 2010 15:18:25 +0100

Andrew,

> implementations, however.
As far as I know, only one implementation exists, and that's Denic with
a dchk implementation. Verisign themselves have pulled the plug of their
IRIS testbed (as far as I can see).

Regards,
Marc

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 19 Jan 2010 16:21:07 +0100

> implementations, however.

IRIS solves a different problem. If you want to sync two databases with epp interface, you definitely want to use epp, as such things like roid, relationship between records etc is absolutely crucial.

But, if the IETF do not want to have a unified way of doing that, then we will done in .SE (I hope) and some other TLDs will invent their way of doing the same thing.

;-)

We have already gone through one issue in Sweden, and that have to do with the fact that access control in EPP was such that a registrar could access more data about records via whois than epp. That forced registrars to use whois in their automated processing, parse the data, risk being blacklisted etc... Now we have made sure (thanks to hard work at .SE) a registrar can always get data about domains in .SE via epp. Not one access mechanism for their domains, and one (whois) for domains the registrar is not a sponsor of.

When you start having tons of data in the registry (about holders, hosts, dnssec keys etc) having one stable way of communicating and not flip flopping between protocols for me absolutely makes sense.

   Patrik

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 19 Jan 2010 10:29:19 -0500

> IRIS solves a different problem. If you want to sync two databases with epp interface, you definitely want to use epp, as such things like roid, relationship between records etc is absolutely cruci…

I thought there was a way to get every single thing you could get in
EPP via IRIS.  No?  (I'm prepared to be wrong.  Afilias was never
really interested in the IRIS stuff, so I read the documents &
commented a little but didn't put a lot of effort into it.)

I wonder, however, whether it's even a good idea to sync two
databases, especially via EPP.  My time as a database geek taught me
nothing so much as that having two authoritative sources for data is a
way to be sure that one of them is wrong.  

A

---

From: [email-masked]
Date: Tue, 19 Jan 2010 16:30:49 +0000

> We have already gone through one issue in Sweden, and that have to 
> do with the fact that access control in EPP was such that a 
> registrar could access more data about records via whois than epp. 
> That forced registrars to use whois in their automated processing, 
> parse the data, risk being blacklisted etc... Now we have made sure 
> (thanks to hard work at .SE) a registrar can always get data about 
> domains in .SE via epp. Not one access mechanism for their domains, 
> and one (whois) for domains the registrar is not a sponsor of.

We've also implemented an <info> extension for sending back a list of 
domains:

  http://www.nominet.org.uk/registrars/systems/nominetepp/list/

Ray

-- 
Ray Bellis, MA(Oxon) MIET
Senior Researcher in Advanced Projects, Nominet
e: [email-masked], t: +44 1865 332211

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 19 Jan 2010 17:34:11 +0100

> commented a little but didn't put a lot of effort into it.)

Yes, of course you _can_, but it is much easier if the database structure etc is exactly the same for every record you want to work with.

> way to be sure that one of them is wrong.  

Well, that is how a thick registry works. By having epp as a synchronization protocol. If the registry is to have the information about who is the holder of each domain, while the registrar is to invoice the customer, the registrar definitely also will have information about the same holder. Transfer of domains and contacts, and updates of contact etc information ends up being an "interesting" game where both parties are moving around the pieces on the chess board at the same time.

In this case, we more or less only talk about the ability for the registrar to issue a query like:

- What domains am I sponsor for?

- What contacts am I sponsor for?

Etc.

   Patrik

---

From: James Gould <[email-masked]>
Date: Tue, 19 Jan 2010 12:27:11 -0500

We created an info extension for getting the additional whois information
(url, whois, and registrar name) to support transfers.  We use reports for
provided a complete list of sponsorship.  Unbounded queries via EPP is not
scalable.  

-- 

JG 

-------------------------------------------------------
James F. Gould
Principal Software Engineer
VeriSign Naming Services
[email-masked]
Direct: 703.948.3271
Mobile: 703.628.7063

21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

Notice to Recipient:  This e-mail contains confidential, proprietary and/or
Registry  Sensitive information intended solely for the recipient and, thus
may not be  retransmitted, reproduced or disclosed without the prior written
consent of  VeriSign Naming and Directory Services.  If you have received
this e-mail message in error, please notify the sender immediately by
telephone or reply e-mail and destroy the original message without making a
copy.  Thank you.

From: <[email-masked]>
Date: Tue, 19 Jan 2010 16:30:49 +0000
To: Patrik Fältström <[email-masked]>
Cc: Andrew Sullivan <[email-masked]>, EPP Provreg <[email-masked]>,
"[email-masked]" <[email-masked]>
Subject: Re: [ietf-provreg] new epp commands?

  http://www.nominet.org.uk/registrars/systems/nominetepp/list/
<http://www.nominet.org.uk/registrars/systems/nominetepp/list/>

Ray

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 19 Jan 2010 13:57:37 -0500

> provided a complete list of sponsorship.  Unbounded queries via EPP is not

Why not?  It's just an XML document.  Why couldn't one have a
mechanism for this?  (Whether it's sane is another matter.  I'm just
being querulous.)

A

---

From: Andrew Newton <[email-masked]>
Date: Tue, 19 Jan 2010 14:08:00 -0500

>>> IRIS solves a different problem. If you want to sync two databases with epp interface, you definitely want to use epp, as such things like roid, relationship between records etc is absolutely cru…

Yes, IRIS was focused on end-user to registry/registrar and not registrar to registry.  That was the big difference.

>> commented a little but didn't put a lot of effort into it.)

There was an IRIS serialization format for copying datasets, not necessarily synching.  Think LDIF for IRIS.

> Yes, of course you _can_, but it is much easier if the database structure etc is exactly the same for every record you want to work with.

I would agree with this, except this entire discussion seems to center around the fact the everyone does something in a different way.  Which kinda leads one to question a standard here.

As an aside, the ECRIT working group also has a synchronization format for their problem space and even their simpler cases seem difficult.

> - What contacts am I sponsor for?

Andrew is right that two authoritative sources makes the job much more difficult.  If the problem were constrained to "the registry is always right", then the task gets much easier.

As for doing this in IRIS or EPP or something else?  Resources are resources and I don't think there is much of a difference when it comes to CPU cycles, so it may as well be done inside EPP to lessen the pain of adoption and interoperability.  It seems to me that if some registries don't want to implement it, then they shouldn't.

-andy

---

From: "Bruce Tonkin" <[email-masked]>
Date: Wed, 20 Jan 2010 08:06:47 +1000

Hello All,

> out of place in a provisioning protocol due to the large 

I am certainly getting requests to have some standard commands around
some of the reporting aspects of domain name management.

In particular some way of polling for account balances etc.   The
process of adding funds is generally out-of-band - e.g via
accounting/finance functions, so it is useful for the provisioning
systems to be able to check account balances against database
information such as number of names coming up for renewal etc.

Regards,
Bruce Tonkin

---

From: [email-masked]
Date: Wed, 20 Jan 2010 09:34:38 +0000

> Andrew Sullivan <[email-masked]>

> mechanism for this?  (Whether it's sane is another matter.  I'm just

As has been mentioned, we have an EPP implementation of a list command. The
email-based system which predates EPP has a similar command, and is
routinely used by registrars to retrieve lists of ~1M domain names along
with account, host details etc. Registrars migrating to EPP have a
reasonable expectation to do the same thing, hence we have implemented it.

There were several scalability issues, but the most compelling was that
length of the message has to be known before it is sent, so that the 4-byte
header can be correctly written. This means that the entire message has to
be held in memory (or other storage), which in this case has the potential
to be very large. Multiply this size by the number of possible concurrent
requests, and you have an obvious hardware problem. Although the 4-byte
header clearly has its uses, it prevented us from streaming information
through the EPP server to the user - the storage issue disappears if you
can retrieve a small number of rows from the database, format them in XML,
send them down the socket, and repeat until you have fulfilled the request.
This way, the amount of storage needed per connection is bounded above and
is no different to a single-domain <info>, which is much easier to cope
with.

For this and other reasons, we settled for a hamstrung version with a
configurable upper bound. This is rather inferior to the equivalent 'less
sophisticated' email system but is useful enough, and is rather more
efficient, both server- and client-side, than the registrar sending one
<info> request for each of their domain names. By comparison, the
email-based system does not process concurrent requests, so the maximum
storage required is bounded above by the size of our registry, so holding
the entire message in memory was never a problem. Also, it does not return
XML, so the message itself is a lot more terse.

Alistair Braden