Subject: [ldapext] new version of LDIF

From: Yves Dorfsman <[email-masked]>
Date: Thu, 18 Dec 2008 07:49:53 -0700

I would like to propose a new version of the LDIF format.

I have been directed here by the people on the ldap mailing list at 
umich.edu. But, this Working Group is documented as closed. Are there people 
  still following this list ? Do I just start a discussion here, or is there 
a procedure to re-open this working group ? Or do we need to create a new 
working gorup ?

Thanks.

-- 
Yves Dorfsman.
http://www.sollers.ca

_______________________________________________
Ldapext mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ldapext

---

From: Ludovic Poitou <[email-masked]>
Date: Fri, 19 Dec 2008 15:47:45 +0100

Hi Yves,

Yes there are still people on the list and we still discuss about  
LDAP, extensions and updates to the RFCs.
Please start the discussion here.

What would you like to change in the LDIF format ?

I personally would like to see full support for UTF-8 characters in  
LDIF, and possibly other encodings.

Ludovic.

On Dec 18, 2008, at 3:49 PM, Yves Dorfsman wrote:

> umich.edu. But, this Working Group is documented as closed. Are  
> there people  still following this list ? Do I just start a  
> discussion here, or is there a procedure to re-open this working  
> group ? Or do we need to create a new working gorup ?
>
> Thanks.
>
> -- 
> Yves Dorfsman.
> http://www.sollers.ca
>
>

Ludovic Poitou                                    Sun Microsystems Inc.
OpenDS Community Lead               Directory Services
http://blogs.sun.com/Ludo/         Grenoble Engineering Center - France

Sun Microsystems requires the following notice:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  This email message is for the sole use of the intended
recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution is prohibited.
If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---

From: Hallvard B Furuseth <[email-masked]>
Date: Fri, 19 Dec 2008 16:14:48 +0100

Ludovic Poitou writes:
> LDIF,

yes...

>  and possibly other encodings.

That should be optional, so LDAP implementations are not burdened with a
requrement to support charset encoding if they do not actually need it.

If a file or entry has 'charset: ...' this should not only affect
'attr: val', not 'attr:: val' and 'attr:< file'.  The two latter may
contain non-text data, e.g. a jpeg picture.

So an app which writes latin-1 may still need to convert to utf-8, in
case it needs to write a value which must be base64-encoded.  E.g. a
value which contains a newline.

A fix would be to have two base64 attribute syntaxes: one which is
charset-converted after base64-decoding, and one which is not.

A looser grammar would be useful too.  Allow an LDIF file
with no entries.  Extra CRLFs at the end of the file.

I have various notes about wishes for LDIF spread around in my
mailbox...  Probably I should have undertaken to update LDIF format
myself, but I never seem to get around it.  I'll see what I can find.

-- 
Hallvard

---

From: Kurt Zeilenga <[email-masked]>
Date: Fri, 19 Dec 2008 07:56:03 -0800

As side from not requiring escaping of UTF-8, I'd like to see LDIF be  
revised to not only provide a representation of LDAP Add and Modify  
requests, but all LDAP requests and all LDAP responses.  For instance,  
I'd like for LDIF to be able to represent an LDAP Transaction (draft- 
zeilenga-ldap-txn), to fully cover LDAP search responses, etc..

Also, would be good to include better support for XML encode attribute  
values.  Steven Legg has (had) a good proposal here.

Given that there are many who have opinions as to what should be done,  
and how things should be done, this item screams for a working  
group.   I think it would be good for someone (maybe Yves) to put  
together a mini-requirements statement: a list of enhancements with  
MUST, SHOULD, MAY suggestions for the work.

I note that I have some reservations concerning some of the  
suggestions (especially in regards to extending LDIF beyond  
"interchange" and, in particular "charset:").  But at this point, I  
think it's best to focus on collection of suggestions, not debate over  
the merits of each suggestion.

-- Kurt

---

From: Yves Dorfsman <[email-masked]>
Date: Fri, 19 Dec 2008 09:02:06 -0700

Ludovic Poitou wrote:
> 
> Yes there are still people on the list and we still discuss about LDAP, 
> extensions and updates to the RFCs.

Thanks.

> LDIF, and possibly other encodings.

Yes absolutely ! That's exactly what I'd like to modify.

To me, the advantage of LDIF over other formats (say DSML), is that it is 
easy for a human to read, which makes it a really good format for backups 
for example. If something's broken, you can diff the versions, see what's 
wrong and fix it. But, encoding anything other than ASCII into base64 breaks 
that model for any language that uses characters outside the 7 bit SAFE-CHAR.

 From my experience with open-ldap and what I have read on the net about the 
other servers, all the LDAP servers will happily import LDIF files with 
"plain UTF-8", but they all follow the RFC when it comes to write LDIF, and 
base 64 encode anything that contains a characters other than SAFE-CHAR.
Same thing for the python-ldap library.

So really, there isn't a lot that needs to be changed, and it shouldn't 
disturb the existing products at all, it will just give them another option 
for writing.

-- 
Yves.
http://www.sollers.ca/blog

---

From: Ludovic Poitou <[email-masked]>
Date: Fri, 19 Dec 2008 17:08:16 +0100

> requrement to support charset encoding if they do not actually need  

Yes, optional.
I think this is most useful for input to an LDAP directory server,  
especially when LDIF data is generated from other sources.
For interoperability, servers producing LDIF for interchange should  
keep producing ASCII or UTF-8 (with a preference for the later).

> contain non-text data, e.g. a jpeg picture.

That may be subject to specification... but I tend to agree. My  
current use of a specific charset is for regular attr: val attributes,  
not for things that are base64 encoded.

Ludovic.

---

From: Yves Dorfsman <[email-masked]>
Date: Fri, 19 Dec 2008 09:11:56 -0700

> contain non-text data, e.g. a jpeg picture.

Hmmm... I think the IBM server already has such an extension. With utf-8 
finally getting some traction, is there really any point to spend any energy 
for other charset ?

What would the other charsets offer that utf-8 cannot, I mean beyond not 
having to convert ?

> with no entries.  Extra CRLFs at the end of the file.

Sure.

Thanks.

---

From: Hallvard B Furuseth <[email-masked]>
Date: Sat, 20 Dec 2008 06:55:36 +0100

> disturb the existing products at all, it will just give them another option 

In that case I think you're talking of an update to LDIFv1, to align
the RFC with current practice.  An update most implementations would
need to do nothing to support.  Is that what you are volunteering to
do?  If so, a free-for-all with change wishes would be a bit
pointless, unless someone else gets inspired to do LDIFv2.

Bumping the version number would require work on many implementations
and installations:  They'd begin to receive LDIFv2 files, and would
thus feel pressure to accept them.

Of course, which changes to include will still be debatable.  _Some_
implementatinos likely follow the RFC and reject 8-bit data.  Others
may follow the grammar except the UTF-8 part, and would choke on my
suggestion to loosen the grammar.  An optional feature ("charset:")
could likely go in if it doesn't conflict with an existing extension.

---

From: Hallvard B Furuseth <[email-masked]>
Date: Sat, 20 Dec 2008 07:08:12 +0100

>> If a file or entry has 'charset: ...' this should not only affect

Typo, it _should_ only affect...

> finally getting some traction, is there really any point to spend any energy 

I've no opinion, really.  UTF-8 has been "just about to take over the
world" for a decade now.  Maybe it's finally happening, but I'm not
holding my breath.  So as far as I'm concerned it'd be nice to have, but
low priority.

> What would the other charsets offer that utf-8 cannot, I mean beyond not 

Well, support for other applications than LDAPv3, since LDAPv3 will
require the LDIF parser to charset-convert to UTF-8.  But if such a
thing would need non-UTF-8 support it would likely have it now anyway.

---

From: simo <[email-masked]>
Date: Sat, 20 Dec 2008 11:38:26 -0500

> could likely go in if it doesn't conflict with an existing extension.

I think another point is also how far you want to go.
So far an LDIF file is basically just a dump, a copycat image of a
directory, something you can use basically only as loading from zero.
But it's usefulness decreases greatly if you want to use4 it on "dirty"
directories, as LDIF currently has no understanding of existing objects,
conditional modifications and so on.

I am sure many people have developed their own formats and tools to do
conditional updates, and so we have in our own project.

Now would it be interesting for others to go as far as adding clues to
the format about how the server should behave?
What to do if the entry or the attribute mentioned in the LDIF file
already exist, whether it should be replaced, changed, added on top,
etc ... ?

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <[email-masked]>
Senior Software Engineer at Red Hat Inc. <[email-masked]>

---

From: Michael Ströder <[email-masked]>
Date: Sat, 20 Dec 2008 20:00:30 +0100

>> I personally would like to see full support for UTF-8 characters in  

Yes.

> If a file or entry has 'charset: ...'

Uurgs! No! What's wrong with just sticking to UTF-8?

> with no entries.

Yes.

>  Extra CRLFs at the end of the file.

Why that?

Ciao, Michael.

---

From: Steven Legg <[email-masked]>
Date: Mon, 22 Dec 2008 10:02:43 +1100

Kurt Zeilenga wrote:
> I'd like for LDIF to be able to represent an LDAP Transaction 
> (draft-zeilenga-ldap-txn), to fully cover LDAP search responses, etc..
> 

The Internet-Draft has expired, but a copy of the latest revision is
available here:

     http://www.xmled.info/drafts/draft-sciberras-xed-eldif-05.txt

The main features of the proposal are that it uses UTF-8 and has a
human-readable way of presenting attribute values with line endings,
which XML has in abundance.

Regards,
Steven

> and how things should be done, this item screams for a working group.   
> I think it would be good for someone (maybe Yves) to put together a 
> mini-requirements statement: a list of enhancements with MUST, SHOULD, 
> MAY suggestions for the work.
> 
> I note that I have some reservations concerning some of the suggestions 
> (especially in regards to extending LDIF beyond "interchange" and, in 
> particular "charset:").  But at this point, I think it's best to focus 
> on collection of suggestions, not debate over the merits of each 
> suggestion.
> 
> -- Kurt