Subject: [NGO] Schema for InetAddress data types

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Dec 2006 09:55:44 -0800

Hi,

One big question -- choices or unions for the aggregate address types.
Another issue is table indexing and xpath filtering by InetAddress.

Andy

<xs:schema xmlns="http://netconfcentral.org/ncx/ncx"
 targetNamespace="http://netconfcentral.org/ncx/ncx"
 xmlns:xs="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.netconfcentral.org/xsd/ncx/inetAddress.xsd"
 elementFormDefault="qualified" attributeFormDefault="unqualified" xml:lang="en"
 version="0.1"
 xmlns:ncx="http://netconfcentral.org/ncx/1.0"
 xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
 <xs:annotation>
  <xs:documentation>
    Converted from NCX module by ncxdump v0.1

    Module: inetAddress
    Owner: ncx
    Application: ncx
    Version: 0.1
    Copyright (C) Andy Bierman (2005-2006).  All Rights Reserved.
    Contact Info: Translated by Andy Bierman.
       Send comments to &lt;[email-masked]&gt;.

    Description:
      InetAddress data types converted from the following sources: 

        * RFC 4001:  Textual Conventions for Transport Addresses

        * draft-romascanu-netconf-datatypes-01.txt
  
        * libsmi output from 'smidump -f xsd' command
          http://www.ibr.cs.tu-bs.de/projects/libsmi/   
      
     The InetAddress data types are literal translations for easier
     SNMP compatibility.  ASCII versions are also available.

    Revision History:

      Revision: 0.1
      Initial version.

  </xs:documentation>
 </xs:annotation>

 <xs:import namespace="http://www.w3.org/XML/1998/namespace"
  schemaLocation="http://www.w3.org/2001/xml.xsd"/>

 <xs:simpleType name="InetAddressType">
  <xs:annotation>
   <xs:documentation>
    InetAddressType TC from RFC 4001, pages 6 - 7.
         Extended to support more IPv6 address types
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:enumeration value="unknown"/>
   <xs:enumeration value="ipv4"/>
   <xs:enumeration value="ipv6"/>
   <xs:enumeration value="ipv4z"/>
   <xs:enumeration value="ipv6z"/>
   <xs:enumeration value="ipv6m"/>
   <xs:enumeration value="ipv6s"/>
   <xs:enumeration value="dns"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetZoneIndex">
  <xs:annotation>
   <xs:documentation>
    InetAddress Zone Index from RFC 4001.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:unsignedInt">
   <xs:minInclusive value="1"/>
   <xs:maxInclusive value="65535"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetAddressUnknown">
  <xs:annotation>
   <xs:documentation>
    Entry for a generic type, not included here
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string"/>
 </xs:simpleType>

 <xs:simpleType name="InetAddressIPv4">
  <xs:annotation>
   <xs:documentation>
    InetAddressIPv4 TC from RFC 4001, pages 7 - 8.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:pattern value="(([0-1]?[0-9]?[0-9]|2?[0-5]?[0-5])\.){3}
                     ([0-1]?[0-9]?[0-9]|2?[0-5]?[0-5])"/>
   <xs:minLength value="7"/>
   <xs:maxLength value="15"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:complexType name="InetAddressIPv4z">
  <xs:annotation>
   <xs:documentation>
    InetAddressIPv4z TC from RFC 4001, pages 8 - 9.,
         draft-romascanu-netconf-datatypes-01.txt, page 5
   </xs:documentation>
  </xs:annotation>
  <xs:sequence>
   <xs:element name="ipv4Address" type="InetAddressIPv4"/>
   <xs:element name="zoneIndex" type="InetZoneIndex"/>
  </xs:sequence>
 </xs:complexType>

 <xs:simpleType name="InetAddressIPv6">
  <xs:annotation>
   <xs:documentation>
    InetAddressIPv6 TC from RFC 4001, page 8.,
         draft-romascanu-netconf-datatypes-01.txt, page 6
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:pattern value="(([0-9a-fA-F]{1,4}:){6})
            (([0-9a-fA-F]{1,4}:[0-9a-fA-F]{1,4})|
            ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetAddressIPv6m">
  <xs:annotation>
   <xs:documentation>
    draft-romascanu-netconf-datatypes-01.txt, page 6
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:pattern value="(([0-9a-fA-F]{1,4}:){6})
              (([0-9a-fA-F]{1,4}:[0-9a-fA-F]{1,4})|
              ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetAddressIPv6s">
  <xs:annotation>
   <xs:documentation>
    draft-romascanu-netconf-datatypes-01.txt, page 6
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:pattern value="(([0-9a-fA-F]{1,4}:)*|([0-9a-fA-F]{1,4}))*(::)
             (([0-9a-fA-F]{1,4}:)*|([0-9a-fA-F]{1,4}))*"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:complexType name="InetAddressIPv6z">
  <xs:annotation>
   <xs:documentation>
    InetAddressIPv6z TC from RFC 4001, page 9.
         draft-romascanu-netconf-datatypes-01.txt, page 6.
   </xs:documentation>
  </xs:annotation>
  <xs:sequence>
   <xs:element name="ipv6Address" type="InetAddressIPv6"/>
   <xs:element name="zoneIndex" type="InetZoneIndex"/>
  </xs:sequence>
 </xs:complexType>

 <xs:simpleType name="InetAddressDNS">
  <xs:annotation>
   <xs:documentation>
    InetAddressDNS TC from RFC 4001, pages 9 - 10.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:minLength value="1"/>
   <xs:maxLength value="255"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetAddressPrefixLength">
  <xs:annotation>
   <xs:documentation>
    InetAddressDNS TC from RFC 4001, pages 9 - 10.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:unsignedInt">
   <xs:maxInclusive value="2040"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetPortNumber">
  <xs:annotation>
   <xs:documentation>
    InetPortNumber TC from RFC 4001, pages 10 - 11.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:unsignedInt">
   <xs:maxInclusive value="65535"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetAutonomousSystemNumber">
  <xs:annotation>
   <xs:documentation>
    InetAutonomousSystemNumber TC from RFC 4001, page 11.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:unsignedInt"/>
 </xs:simpleType>

 <xs:simpleType name="InetScopeType">
  <xs:annotation>
   <xs:documentation>
    InetScopeType TC from RFC 4001, pages 11 - 12.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:enumeration value="interfaceLocal"/>
   <xs:enumeration value="linkLocal"/>
   <xs:enumeration value="subnetLocal"/>
   <xs:enumeration value="adminLocal"/>
   <xs:enumeration value="siteLocal"/>
   <xs:enumeration value="organizationLocal"/>
   <xs:enumeration value="global"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:simpleType name="InetVersion">
  <xs:annotation>
   <xs:documentation>
    InetVersion TC from RFC 4001, pages 12 - 13.
   </xs:documentation>
  </xs:annotation>
  <xs:restriction base="xs:string">
   <xs:enumeration value="unknown"/>
   <xs:enumeration value="ipv4"/>
   <xs:enumeration value="ipv6"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:complexType name="InetAddress">
  <xs:annotation>
   <xs:documentation>
    InetAddress TC from RFC 4001, page 7. 

         The actual format of the string content is determined by
         the mandatory 'type' attribute as follows:
         
          unknown: InetAddressUnknown
          ipv4:    InetAddressIPv4
          ipv6:    InetAddressIPv6
          ipv4z:   InetAddressIPv4z 
          ipv6z:   InetAddressIPv6z
          ipv6m:   InetAddressIPv6m
          ipv6s:   InetAddressIPv6s
          dns:     InetAddressDNS

        The use of separate elements instead of a union or 
        substitution group approach allows for schema validation
        of the different address types, instead of just run-time
        validation.
   </xs:documentation>
  </xs:annotation>
  <xs:choice>
   <xs:element name="addr" type="InetAddressUnknown"/>
   <xs:element name="ipv4-addr" type="InetAddressIPv4"/>
   <xs:element name="ipv6-addr" type="InetAddressIPv6"/>
   <xs:element name="ipv4z-addr" type="InetAddressIPv4z"/>
   <xs:element name="ipv6m-addr" type="InetAddressIPv6m"/>
   <xs:element name="ipv6s-addr" type="InetAddressIPv6s"/>
   <xs:element name="dns-addr" type="InetAddressDNS"/>
  </xs:choice>
  <xs:attribute name="type" type="InetAddressType" use="required"/>
 </xs:complexType>
</xs:schema>
_______________________________________________
NGO mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ngo

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 06 Dec 2006 22:57:14 +0100 (CET)

> One big question -- choices or unions for the aggregate address types.

I prefer a union.  This is what we are using today.  We don't have the
different ipv6 formats as different types, we just have a single v6
type:

  <xs:simpleType name="InetAddressIPv6">
    <xs:restriction base="xs:string">
      <xs:minLength value="0"/>
      <xs:maxLength value="39"/>
      <!-- Full -->
      <xs:pattern value="(([0-9a-fA-F]{1,4}:){7})([0-9a-fA-F]{1,4})"/>
      <!-- Mixed -->
      <xs:pattern value="(([0-9a-fA-F]{1,4}:){6})(([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))"/>
      <!-- Shortened -->
      <xs:pattern value="(([0-9a-fA-F]{1,4}:)*([0-9a-fA-F]{1,4}))*(::)(([0-9a-fA-F]{1,4}:)*([0-9a-fA-F]{1,4}))*"/>
      <!-- Shortened mixed -->
      <xs:pattern value="(([0-9a-fA-F]{1,4}:)*([0-9a-fA-F]{1,4}))*(::)(([0-9a-fA-F]{1,4}:)*([0-9a-fA-F]{1,4}))*(([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))"/>
    </xs:restriction>
  </xs:simpleType>

This way the client can use whichever format it think is convenient in
the instance document; you don't have to decide this in the datamodel.

Also, we have the following unions which we find convenient:

  <xs:simpleType name="InetAddress">
    <xs:union memberTypes="InetAddressIP InetAddressDNS"/>
  </xs:simpleType>

  <xs:simpleType name="InetAddressIP">
    <xs:union memberTypes="InetAddressIPv4 InetAddressIPv6"/>
  </xs:simpleType>

I think this version of InetAddress is much simpler, but it
does not handle the zone addresses.  This could be solved by using
separate simpleTypes (instead of the complexTypes you suggested), like
in RFC4001.

As an example, an element of type InetAddress could be:
   
   <address>FF01::101</address>

In your example it would be (why do you have an attribute and an
element to find the type?)

   <address type="ipv6s"> 
      <ipv6s-addr xmlns="http://netconfcentral.org/ncx/ncx">FF01::101</ipv6s-addr>
   </address>   

(BTW, in your schema, InetAddressIPv6m and InetAddressIPv6s seem to
have the same regexp).

> Another issue is table indexing and xpath filtering by InetAddress.

With the union we're using this is not an issue.  Unless I have
misunderstood what you mean.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 07 Dec 2006 07:14:32 -0800

> different ipv6 formats as different types, we just have a single v6

I prefer a discriminated union, which it what SNMP
tries to do with InetAddressType/InetAddress object pairs.

How about if the unions have a required 'type' attribute?

  for InetAddressIPv6:

   <xs:attribute name="type" type="InetAddressIPv6Type" use="required"/>

  for InetAddress:

How about sequences (structs) just for the { zoneIndex, address } tuples?

> (BTW, in your schema, InetAddressIPv6m and InetAddressIPv6s seem to

oops - darn cut-and-paste bugs!

> misunderstood what you mean.

I think the InetAddressType must be present, and in the same 'place'
every time.  Since the type value cannot change unless its associated
address changes value as well, making it an attribute is not a problem.
(I like to use elements for data so I can used attributes for
metadata like 'last-change-time' or 'changed-by', etc.)

Many times a data model will be defined which only uses a subset of
the InetAddressTypes that can be represented.  It is useful to be
able to enforce such a subset in generic validation code, vs. agent callback
validation code.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 07 Dec 2006 20:47:11 +0100 (CET)

>    <xs:attribute name="type" type="InetAddressType" use="required"/>

But the type info would be redundant:

   <address type="ipv4">10.0.0.1</address>

In SNMP it's necessary to use two objects, but why use it if it's not
needed in XML?

> How about sequences (structs) just for the { zoneIndex, address } tuples?

Yes, but then they can't be part of the big InetAddress union, since
unions can only have simpleTypes as members.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 07 Dec 2006 13:39:35 -0800

>    <address type="ipv4">10.0.0.1</address>

This is not true if the address string has a bug in it so it's wrong.
It's not obvious to code that checks for certain IP types,
to reverse-match the pattern of the string to the address type.
It's cleaner and more robust to use a discriminated union
than an plain union.

> unions can only have simpleTypes as members.

No I guess not.  But they can be defined separately
as { zoneIndex, inetAddress } tuples.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 08 Dec 2006 10:28:21 +0100

Hi,

the regular expression for IPv4 address seems to be incorrect.

>    <xs:pattern value="(([0-1]?[0-9]?[0-9]|2?[0-5]?[0-5])\.){3}

It won't accept components such as 239. This one should work:

<xs:pattern value="((25[0-5]|2[0-4][0-9]|[01]?[0-9]?[0-9]).){3}(25[0-5]|
2[0-4][0-9]|[01]?[0-9]?[0-9])"/>

Lada

-- 
Ladislav Lhotka, CESNET
PGP key id: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 Dec 2006 10:45:50 +0100 (CET)

> <xs:pattern value="((25[0-5]|2[0-4][0-9]|[01]?[0-9]?[0-9]).){3}(25[0-5]|

We're using this pattern for v4 addresses:

      <xs:pattern value="(([0-1]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])\.){1,3}([0-1]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])"/>

which also accepts "shortened" addresses (like telnet does) such as
127.1

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 Dec 2006 11:00:51 +0100 (CET)

> to reverse-match the pattern of the string to the address type.

There are several programs that handle this, one example is the normal
telnet on my linux box.  Another example is our code :)

> It's cleaner and more robust to use a discriminated union

I guess you can argue that it's more robust to always add the type to
all elements in the instance documents, such as
   
   <port type="rfc4001:InetPortNumber">25</port>   

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 08 Dec 2006 11:15:32 +0100

> which also accepts "shortened" addresses (like telnet does) such as

I think this is ambiguous and should be avoided.

Lada

-- 
Ladislav Lhotka, CESNET
PGP key id: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2006 11:16:13 +0100

> It's cleaner and more robust to use a discriminated union

I doubt that the ASN.1 tags in SNMP BER encoded messages ever helped
to make things more robust. In many cases, even the tags happen to be
wrong (e.g. counter values coming with integer tags, timetick values
coming with a counter tag, ...).

/js

-- 
Juergen Schoenwaelder		 {International|Jacobs} University Bremen
<http://www.eecs.iu-bremen.de/>	 P.O. Box 750 561, 28725 Bremen, Germany

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2006 11:19:19 +0100

> which also accepts "shortened" addresses (like telnet does) such as

I consider this an unneeded feature which is likely to cause
confusion. Do you suggest that all application dealing with netconf
data have to be prepared to deal with this notation?

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 Dec 2006 12:20:07 +0100 (CET)

> data have to be prepared to deal with this notation?

No, I agree that it is obscure.  I'd be happy to change this in our
implementation.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 Dec 2006 06:51:55 -0800

> coming with a counter tag, ...).

It's tagging in the data model, not the ASN.1 encoding.
Why do we have all these rules in MIBs that say InetAddressType
must be present when an InetAddress is used?  Is it pointless
in MIBs to distinguish and identify non-deterministic content?
I don't think so.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 08 Dec 2006 10:03:29 -0500

Juergen Schoenwaelder writes:
>> ...which also accepts "shortened" addresses (like telnet does) such as
>> 127.1

To add the the confusion, prefixes are often abbreviated by trimming
trailing ".0"s.  For example, "10/8" is 10.0.0.0/8, so telnet's
interpretation of "127.1" as "127.0.0.1" is very different than
most routing-related software's interpretation of "127.1/16" as
"127.1.0.0/16".  Accepting this leading-prefix style is probably
more important than accepting the BSD inet_aton() style.

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2006 16:41:10 +0100

> It's tagging in the data model, not the ASN.1 encoding.

Yes. But I continue to ask the question whether things really get more
robust by adding more tags and noise around some value since you
increase the number of potential inconsistencies and the number of
potentially different reactions to those increased number of
situations you can encounter.

> in MIBs to distinguish and identify non-deterministic content?

Our OCTET STRINGs we use to carry address information in SNMP are
binary and hence there is no way to distinguish between a 16 byte IPv6
address and a host name which happens to be 16 octets long. The type
value is therefore essential to have.

In XML, we talk about rendered address values and fortunately the
rendering rules were created to allow humans and machines to determine
what type of address we look at.

I do not know what non-deterministic contents means in this context so
I prefer to not comment on that.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 Dec 2006 08:04:04 -0800

> I prefer to not comment on that.

1) There is an 'unknown' data type, which means it can match any
    pattern (or none) and still be valid.  How can you support that
    without tagging the inetAddressType?

2) It is fragile design to assume that no address type can ever be
    mistaken for another one, based on the pattern match.  Are you so sure
    that there will never be 2 InetAddress variants for which 1 pattern
    matches some valid addresses of another type?  (This is already false
    because of the 'unknown' address type.)

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2006 17:48:23 +0100

>    without tagging the inetAddressType?

In case you talk about InetAddressType, the following holds true:

:       unknown(0)  An unknown address type.  This value MUST
:                   be used if the value of the corresponding
:                   InetAddress object is a zero-length string.
:                   It may also be used to indicate an IP address
:                   that is not in one of the formats defined
:                   below.

This translates to an empty XML element or something like that, no?

>    because of the 'unknown' address type.)

This is how all my command line applications work. Perhaps they are
all fragile.

I do not really care that much about the presence of the type
information.  I just like to get the reasoning straight and I am
not convinced by the "added robustness" argument nor does the
"non-deterministic contents" argument convince me so far.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 Dec 2006 09:13:00 -0800

> This translates to an empty XML element or something like that, no?

no -- last sentence:

It may also be used to indicate an IP address
that is not in one of the formats defined below.

> "non-deterministic contents" argument convince me so far.

IMO it is fragile to assume that no 2 address variant patterns can ever
overlap.  CLI applications that use unnamed and position-dependent
parameter names are fragile.  They break when version N+1 adds something
that a "version N" user doesn't expect, and what used to parse
correctly is no longer working.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2006 21:45:54 +0100

> correctly is no longer working.

Generally true.

If this would happen for IP addresses though, a significant number of
existing applications would break and I can't imagine that the IETF
would go down that road. And we are not talking about position
dependent parameters as far as I understand.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 09 Dec 2006 13:16:03 +0100

Hi,

I think it also depends on whether one is aiming at an essentially rigid
data model as the SNMP MIBs are or at a flexible portfolio of data types
that can be reused in different and not yet foreseen contexts. In the
latter case, giving extra discriminating information makes sense.

For example, we are using the following RELAX NG pattern where it would
be even less safe to leave the interpretation of the data unspecified:

ip-or-name-content =
  (attribute content-type { "ipv4" },
   ipv4-address-content)
  | (attribute content-type { "ipv6" },
     ipv6-address-content)
  | (attribute content-type { "name" },
     domain-name-content)

Lada

-- 
Ladislav Lhotka, CESNET
PGP key id: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 09 Dec 2006 15:04:38 -0800

> dependent parameters as far as I understand.

How about if the attribute was optional?

<xs:attribute name="type" type="InetAddressIPv6Type"/>

<xs:attribute name="type" type="InetAddressIPv6Type"/>

That way a standard mechanism is in place for strong typing
as well as the default of "figure it out from the pattern".

IMO, having the attribute mandatory is better for supporting
Xpath filters against the address type (simpler expressions).
Unless it is mandatory, the filter will simply ignore untagged addresses.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Dec 2006 09:38:17 +0100

> How about if the attribute was optional?

This would make it more complicated for the processing software.
 
> Unless it is mandatory, the filter will simply ignore untagged addresses.

Exactly. I have a recent experience with processing Open Document
Format. Trying to find certain logical constructs like figure captions
is just a nightmare because there is no robust markup to distinguish
them from plain paragraphs.

Lada

-- 
Ladislav Lhotka, CESNET
PGP key id: E74E8C0C