Subject: [arch-d] It's about Information

From: Bruce Curtis <[email-masked]>
Date: Mon, 4 Dec 2006 17:11:04 -0600

This may be obvious to everyone since I haven't seen much  
discussion about this model but it helped me understand the issues.

   In the IPv4 multihoming model the information about multiple paths  
from host A to host Z is in the routing table of the core routers in  
the Internet.

         ISP 1   ----------------       I    
---------------------------------   ISP 7
            |                                      
n                                                     |
           A   --  ISP 2  -------        t    
------------------------  ISP 8  --  Z
            |                                      
e                                                     |
          ISP 3   ---------------        r   
---------------------------------   ISP 9
                                                   n
                                                   e
                                                   t

   If we change the model and take the information about multiple  
paths out of the core routers and give each host a different IP  
number for each of it's ISPs, then to preserve some of the properties  
of the old model we still need  to have the multiple path information  
available to the network somehow.

          ISP 1   ----------------       I    
---------------------------------   ISP 7
            |                                      
n                                                     |
    A,B,C   --  ISP 2  -------        t   ------------------------   
ISP 8  --  X,Y,Z
            |                                      
e                                                     |
          ISP 3   ---------------        r   
---------------------------------   ISP 9
                                                   n
                                                   e
                                                   t

   If we imagine a time in the future where every host has 100 Mbps  
connection to the Internet like some homes in Japan today and we have  
networks that support Jumbo frames it's easier to consider the  
possibility that we put the required information in every packet from  
host A,B,C to host X,Y,Z.  That is, we put all of the destination  
addresses X,Y,Z in the packet as well as the source addresses A,B,C.
   So basically we take the information about multiple paths out of  
the routing tables where it doesn't scale and put the multiple path  
info in each packet, which should scale right along with the speed of  
the router.

   If we put this information in each packet then the new model has  
several properties that the old model has.  For example:

          Traffic engineering can be done at any point in the path.

           Anti-Spoof checking can be done at the interface between  
the customer and ISP.

           If a link is down between any ISP and host X,Y,Z the  
internet can reroute the packet
           to one of the other addresses for X,Y,Z.

   The more I thought about it the more sense this model made to me  
and it helped me understand many of the other proposed alternatives  
as attempts to optimize this model and remove some of the information  
from the packet.

   Also it made it clearer to me that the discussion about locator/ID  
split has more to do with trying to preserve a single address  
representation to current and future software and that it was the  
multiple locators and not the locator/ID split that helps solve the  
routing portion of the multihoming problem.

   To further understand the possibilities I realized that IPv6 did  
not provide the means to include all of these addresses but IPv6 does  
have a method of being extended to include such addresses, extension  
headers.  Actually putting the multiple destination addresses in  
extension headers might not be too much different from the current  
hop-by-hop extension headers.   (And there would have to be something  
in the standard to avoid loops in the case of a link failure,  
something that marks an address as already having failed, but again  
that might not be too different from the hop-by-hop extension headers.)

   Considering the extension headers helped me move one step closer  
to the real world and also to envision at least  one optimization.   
To me it seems that the natural point to discover the multiple paths  
from host A,B,C to host X,Y,X is at the host A,B,C, through DNS  
lookup etc.  It seems that shim6 is already capable of handling that  
information so I think it makes sense for the end host to put the  
X,Y,Z destination addresses in the packet.  However I can see two  
possibilities for where in the network the multiple source addresses  
are placed in the packet.  Members of the "I hate middleboxes club"  
could have their hosts fill in the multiple source addresses.   
However members of the "I only want my hosts to have one IP address"  
club could design their network to have a middle box at the interface  
to their ISPs and have the middelbox add the multiple source  
addresses.  The middle box would also have to do the right magic to  
route an inbound packet with the multiple destination addresses to  
the end host with the single IP address but it could do so without  
having to maintain state for every connection through the middlebox.   
(Middleboxes with flow-based architecture that are required to  
maintain state are one of my main objections to middleboxes.)  Anyway  
the model of putting enough information in the packet  would seem to  
support at least two  options for the end network, allowing some  
choice for the end network.  And if implemented with extension  
headers it might be partially backwards compatible with what is  
implemented today, it wouldn't break anything, but hosts/networks  
that did not support the new model wouldn't benefit from all of the  
redundancy of multihoming.

   There has been talk on the list about not wanting to overload  
identifiers but it has been helpful to me to think of unique  
unchanging IDs for hosts in terms of the above model.  If we add  
another IP number to each host so we have host A,B,C,D and host  
W,X,Y,Z but D and W have no associated ISPs things still work.  It is  
about the same case as  if D and W did have an associated ISP but the  
links to those ISPs were down.  IPs D and W could even be assigned  
based on GPS location or something similar if they where still  
guaranteed to be unique.  The identifiers are overloaded but it might  
be workable as an interim solution, something similar to shim6.

   Another possible optimization could be to strip off the multiple  
source addresses at the interface between the customer and the ISP  
and just leave the one that is associated with that ISP.  But then  
the receiving host wouldn't automatically know all of the return  
addresses of the sending host.

   Sorry if this was obvious to everyone but I thought the idea of  
putting all of the required info in the packets should at least be  
floated and shot down and hopefully the shooting down and discussion  
would be useful in the long run.  To me considering the extreme case  
was helpful in understanding the problem and in classifying some of  
the proposed alternatives.

---
Bruce Curtis                         [email-masked]
Certified NetAnalyst II                701-231-8527
North Dakota State University

_______________________________________________
Architecture-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/architecture-discuss

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 05 Dec 2006 16:25:13 +0100

Bruce Curtis wrote:
...
>   So basically we take the information about multiple paths out of  the 
> routing tables where it doesn't scale and put the multiple path  info in 
> each packet, which should scale right along with the speed of  the router.

That is essentially what shim6 does, except that it only sends the
multiple locator-addresses when it needs to. The price of not putting
the full set of locators in every packet is the need to create state
at both ends to manage the locator set.

    Brian

---

From: Bruce Curtis <[email-masked]>
Date: Wed, 6 Dec 2006 18:34:51 -0600

On Dec 5, 2006, at 9:25 AM, Brian E Carpenter wrote:

> Bruce Curtis wrote:
> ...
>>   So basically we take the information about multiple paths out  
>> of  the routing tables where it doesn't scale and put the multiple  
>> path  info in each packet, which should scale right along with the  
>> speed of  the router.
>

   Right, and one of the complaints about shim6 is that decisions  
about which of the multiple paths are used is moved from somewhere in  
the network (most commonly at the customer ISP interface) to the end  
host.  If the info were in each packet the decision could be made in  
the network again.

   Also I don't think that shim6 includes the multiple source  
addresses in the packets which led to the complaint that implementing  
anti-spoofing was then difficult or impossible.  Or that shim6 had to  
do extra work to put the correct source address in the packet so that  
anti-spoofing still worked the old way.  But again this complaint was  
probably about wanting to do traffic engineering for load balancing  
at the customer ISP interface also.

   I understood some of the complaints about shim6 and other  
proposals better when I compared them to the model where all the  
address info is in each packet.

---

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 07 Dec 2006 09:19:28 +0100

Bruce Curtis wrote:
>>> info in each packet, which should scale right along with the  speed 
>>> of  the router.
>>
>>
>   Right, and one of the complaints about shim6 is that decisions  about 
> which of the multiple paths are used is moved from somewhere in  the 
> network (most commonly at the customer ISP interface) to the end  host.  
> If the info were in each packet the decision could be made in  the 
> network again.

Yes. But then, some stakeholders might prefer the decision to be taken
by the host; that would allow a host to perform arbitrage between ISPs,
in the economic interest of the host.

> 
>   Also I don't think that shim6 includes the multiple source  addresses 
> in the packets which led to the complaint that implementing  
> probably about wanting to do traffic engineering for load balancing  at 
> the customer ISP interface also.

Well, the tricky issue for shim6 is exit router selection - if ingress
filtering or RPF checks are in place, sending a packet via the wrong
exit router is catastrophic.

>   I understood some of the complaints about shim6 and other  proposals 
> better when I compared them to the model where all the  address info is 
> in each packet.

Yes, it's an interesting way to think about things, I agree.

    Brian
> ------------------------------------------------------------------------
>

---

From: "Bound, Jim" <[email-masked]>
Date: Thu, 7 Dec 2006 07:15:55 -0500

I like this thinking model.  Why is this not a form or source address
routing with a lot of other addded value using locators.  I also believe
that GPS information can add great value to routing and doing some
research on that concept for ad hoc nets but have assumed no way could
we ever get a standard done to use it?
 
/jim

________________________________

	From: Bruce Curtis [mailto:[email-masked]] 
	Sent: Monday, December 04, 2006 6:11 PM
	To: [email-masked]
	Subject: [arch-d] It's about Information

	  In the IPv4 multihoming model the information about multiple
paths from host A to host Z is in the routing table of the core routers
in the Internet.

	        ISP 1   ----------------       I
	           |                                     n
	           |                                     e

	  If we change the model and take the information about multiple
paths out of the core routers and give each host a different IP number
for each of it's ISPs, then to preserve some of the properties of the
old model we still need  to have the multiple path information available
to the network somehow.

	         ISP 1   ----------------       I
---------------------------------   ISP 7 
	           |                                     n
|
	   A,B,C   --  ISP 2  -------        t
------------------------  ISP 8  --  X,Y,Z
	           |                                     e
|
	         ISP 3   ---------------        r
---------------------------------   ISP 9
	                                                  n
	                                                  e
	                                                  t

	  If we imagine a time in the future where every host has 100
Mbps connection to the Internet like some homes in Japan today and we
have networks that support Jumbo frames it's easier to consider the
possibility that we put the required information in every packet from
host A,B,C to host X,Y,Z.  That is, we put all of the destination
addresses X,Y,Z in the packet as well as the source addresses A,B,C.  
	  So basically we take the information about multiple paths out
of the routing tables where it doesn't scale and put the multiple path
info in each packet, which should scale right along with the speed of
the router.

	  If we put this information in each packet then the new model
has several properties that the old model has.  For example:

	         Traffic engineering can be done at any point in the
path.
	         
	          Anti-Spoof checking can be done at the interface
between the customer and ISP.

	  The more I thought about it the more sense this model made to
me and it helped me understand many of the other proposed alternatives
as attempts to optimize this model and remove some of the information
from the packet.

	  Also it made it clearer to me that the discussion about
locator/ID split has more to do with trying to preserve a single address
representation to current and future software and that it was the
multiple locators and not the locator/ID split that helps solve the
routing portion of the multihoming problem.

	  To further understand the possibilities I realized that IPv6
did not provide the means to include all of these addresses but IPv6
does have a method of being extended to include such addresses,
extension headers.  Actually putting the multiple destination addresses
in extension headers might not be too much different from the current
hop-by-hop extension headers.   (And there would have to be something in
the standard to avoid loops in the case of a link failure, something
that marks an address as already having failed, but again that might not
be too different from the hop-by-hop extension headers.)

	  Considering the extension headers helped me move one step
closer to the real world and also to envision at least  one
optimization.  To me it seems that the natural point to discover the
multiple paths from host A,B,C to host X,Y,X is at the host A,B,C,
through DNS lookup etc.  It seems that shim6 is already capable of
handling that information so I think it makes sense for the end host to
put the X,Y,Z destination addresses in the packet.  However I can see
two possibilities for where in the network the multiple source addresses
are placed in the packet.  Members of the "I hate middleboxes club"
could have their hosts fill in the multiple source addresses.  However
members of the "I only want my hosts to have one IP address" club could
design their network to have a middle box at the interface to their ISPs
and have the middelbox add the multiple source addresses.  The middle
box would also have to do the right magic to route an inbound packet
with the multiple destination addresses to the end host with the single
IP address but it could do so without having to maintain state for every
connection through the middlebox.  (Middleboxes with flow-based
architecture that are required to maintain state are one of my main
objections to middleboxes.)  Anyway the model of putting enough
information in the packet  would seem to support at least two  options
for the end network, allowing some choice for the end network.  And if
implemented with extension headers it might be partially backwards
compatible with what is implemented today, it wouldn't break anything,
but hosts/networks that did not support the new model wouldn't benefit
from all of the redundancy of multihoming.

	  There has been talk on the list about not wanting to overload
identifiers but it has been helpful to me to think of unique unchanging
IDs for hosts in terms of the above model.  If we add another IP number
to each host so we have host A,B,C,D and host W,X,Y,Z but D and W have
no associated ISPs things still work.  It is about the same case as  if
D and W did have an associated ISP but the links to those ISPs were
down.  IPs D and W could even be assigned based on GPS location or
something similar if they where still guaranteed to be unique.  The
identifiers are overloaded but it might be workable as an interim
solution, something similar to shim6.

	  Another possible optimization could be to strip off the
multiple source addresses at the interface between the customer and the
ISP and just leave the one that is associated with that ISP.  But then
the receiving host wouldn't automatically know all of the return
addresses of the sending host.
	  
	  Sorry if this was obvious to everyone but I thought the idea
of putting all of the required info in the packets should at least be
floated and shot down and hopefully the shooting down and discussion
would be useful in the long run.  To me considering the extreme case was
helpful in understanding the problem and in classifying some of the
proposed alternatives.

	

	---

---

From: Bruce Curtis <[email-masked]>
Date: Thu, 7 Dec 2006 11:53:07 -0600

On Dec 7, 2006, at 2:19 AM, Brian E Carpenter wrote:

>>>> of  the routing tables where it doesn't scale and put the  
>>>> multiple  path  info in each packet, which should scale right  
>>>> along with the  speed of  the router.
>>>
>>>
>> about which of the multiple paths are used is moved from somewhere  
>> in  the network (most commonly at the customer ISP interface) to  
>> the end  host.  If the info were in each packet the decision could  
>> be made in  the network again.
>

   Right but the model with all the info in the packet allows either  
option, in one person's network they can let the host chose the path  
and the packets flow as they may and in another person's network they  
can have intelligent boxes at the customer ISP interface to do load  
balancing or traffic engineering.

>> addresses in the packets which led to the complaint that  
>> implementing  anti-spoofing was then difficult or impossible.  Or  
>> that shim6 had to  do extra work to put the correct source address  
>> in the packet so that  anti-spoofing still worked the old way.   
>> But again this complaint was  probably about wanting to do traffic  
>> engineering for load balancing  at the customer ISP interface also.
>

   Right, but if all of the address info was in the packet we could  
envision an RPF check that succeeds as long as only one of the  
multiple source addresses was correct.  Also it would be easier for  
the ISP to implement the RPF check if they chose to, they wouldn't  
need to know the customers other possible locators, they would only  
have to check the list of source addresses to see if one of them  
matched that ISPs address range.

> Yes, it's an interesting way to think about things, I agree.

   I'm glad others find it useful for the discussion.

---

---

From: Bruce Curtis <[email-masked]>
Date: Thu, 7 Dec 2006 12:10:40 -0600

On Dec 7, 2006, at 6:15 AM, Bound, Jim wrote:

> I like this thinking model.  Why is this not a form or source  
> address routing with a lot of other addded value using locators.

   The way I thought about the model all of the routing was still  
based on the multiple destination addresses and the source addresses  
would just be used for anti-spoof RPF checking.  But if the source  
info is in every packet lots of other options I hadn't considered are  
possible with this model as well.

> I also believe that GPS information can add great value to routing  
> and doing some research on that concept for ad hoc nets but have  
> assumed no way could we ever get a standard done to use it?
>
> /jim
>

   I know there have been proposals about having the addresses or  
locators based on location or GPS and it solved some routing problems  
but I think the ISPs had some objections.

    When I was thinking through how things would work with this model  
I was thinking of the GPS info as more of an ID than a locator.    
Sometimes when there is a problem host it would be nice to have it's  
location info, but there are privacy issues and perhaps other  
downsides as well.

    But one way to use the same model but turn my thinking on it's  
head is to consider the GPS info as the primary locator and the other  
multiple locators as ISP specific locators.  The question that comes  
to mind is if that would lessen the ISP objections to the GPS based  
address model.  I'm not an ISP so the answer doesn't come naturally  
to me, I'll have to think about if for a while.

> paths from host A to host Z is in the routing table of the core  
> routers in the Internet.
>
>
>         ISP 1   ----------------       I    
> number for each of it's ISPs, then to preserve some of the  
> properties of the old model we still need  to have the multiple  
> path information available to the network somehow.
>
>
>          ISP 1   ----------------       I    
> connection to the Internet like some homes in Japan today and we  
> possibility that we put the required information in every packet  
> from host A,B,C to host X,Y,Z.  That is, we put all of the  
> destination addresses X,Y,Z in the packet as well as the source  
> addresses A,B,C.
> as attempts to optimize this model and remove some of the  
> information from the packet.
>
>   Also it made it clearer to me that the discussion about locator/ 
> ID split has more to do with trying to preserve a single address  
> not provide the means to include all of these addresses but IPv6  
> extension headers.    Actually putting the multiple destination  
> addresses in extension headers might not be too much different from  
> the current hop-by-hop extension headers.   (And there would have  
> to be something in the standard to avoid loops in the case of a  
> link failure, something that marks an address as already having  
> failed, but again that might not be too different from the hop-by- 
> hop extension headers.)
>
> To me it seems that the natural point to discover the multiple  
> paths from host A,B,C to host X,Y,X is at the host A,B,C, through  
> DNS lookup etc.  It seems that shim6 is already capable of handling  
> that information so I think it makes sense for the end host to put  
> the X,Y,Z destination addresses in the packet.  However I can see  
> two possibilities for where in the network the multiple source  
> addresses are placed in the packet.  Members of the "I hate  
> middleboxes club" could have their hosts fill in the multiple  
> source addresses.  However members of the "I only want my hosts to  
> have one IP address" club could design their network to have a  
> middle box at the interface to their ISPs and have the middelbox  
> add the multiple source addresses.  The middle box would also have  
> to do the right magic to route an inbound packet with the multiple  
> destination addresses to the end host with the single IP address  
> but it could do so without having to maintain state for every  
> information in the packet  would seem to support at least two   
> options for the end network, allowing some choice for the end  
> network.  And if implemented with extension headers it might be  
> partially backwards compatible with what is implemented today, it  
> wouldn't break anything, but hosts/networks that did not support  
> the new model wouldn't benefit from all of the redundancy of  
> multihoming.
>
> W,X,Y,Z but D and W have no associated ISPs things still work.  It  
> is about the same case as  if D and W did have an associated ISP  
> but the links to those ISPs were down.  IPs D and W could even be  
> assigned based on GPS location or something similar if they where  
> still guaranteed to be unique.  The identifiers are overloaded but  
> it might be workable as an interim solution, something similar to  
> shim6.
>
> floated and shot down and hopefully the shooting down and  
> discussion would be useful in the long run.  To me considering the  
> extreme case was helpful in understanding the problem and in  
> classifying some of the proposed alternatives.
>
>
> ---

---

---

From: Tony Li <[email-masked]>
Date: Thu, 7 Dec 2006 10:20:35 -0800

On Dec 7, 2006, at 10:10 AM, Bruce Curtis wrote:
> based on the multiple destination addresses and the source  
> addresses would just be used for anti-spoof RPF checking.  But if  
> the source info is in every packet lots of other options I hadn't  
> considered are possible with this model as well.

I think that there's an important distinction here: it's vital to the  
upper layers that packets contain a source identifier.  However, the  
inclusion of a source locator is not clearly necessary.

Tony

---

From: Bruce Curtis <[email-masked]>
Date: Thu, 7 Dec 2006 13:01:49 -0600

On Dec 7, 2006, at 12:20 PM, Tony Li wrote:

> I think that there's an important distinction here: it's vital to  
> the upper layers that packets contain a source identifier.   
> However, the inclusion of a source locator is not clearly necessary.
>
> Tony

   It's vital to the stacks on the end hosts that the packets contain  
a source identifier.  If we imagine an example that is sort of the  
opposite extreme of the model I was thinking of I can see your point  
that inclusion of the source locator in the packet is not necessary  
at all for the network to deliver the packet.  However having the  
source locator in the packet has been handy for anti-spoof RPF checks  
and there are lots of standards that depend on having the source  
locator in the packet.  PMTU Discovery and all of the ICMP error  
reports depend on having the source locator in the packet so that  
they know where to send the error info to.  If it were possible for  
the router to look up the locator info based on the identifier then  
the ICMP error reports wouldn't be an issue, but if the lookup were  
expensive it could lead to possible DoS attacks.  Multicast also  
depends heavily on the source locator for RPF checking, but multicast  
is already maintaining state so I guess if the source locator info is  
in some early packets and the router caches the source identifier to  
be associated with the state created then it might not be necessary  
for the later packets to contain a source locator.  But that could  
potentially lead to some scaling issues for the multicast RPF check  
if the check was done against a cache of source identifiers rather  
than comparing the source locator to the routing table.

---

---

From: "Tony Li" <[email-masked]>
Date: Thu, 7 Dec 2006 11:23:57 -0800

If we imagine an example that is sort of the opposite extreme of the
model I was thinking of I can see your point that inclusion of the
source locator in the packet is not necessary at all for the network to
deliver the packet. However having the source locator in the packet has
been handy for anti-spoof RPF checks and there are lots of standards
that depend on having the source locator in the packet. 

I think that's there's a huge separation between "handy" and "required".
As we're positing new architecture, I think that we should be very clear
in separating the two.  I agree that having a source locator has been
handy for RPF checks, but is that the only way?  If the identifier space
is structured, then filtering on the identifier might be tractable.  And
again, as we're positing new architecture anyway, compliance to existing
standards doesn't seem like it is strictly "required".  In fact, if it
is, it would seem challenging to make any forward progress.

  PMTU Discovery and all of the ICMP error reports depend on having the
source locator in the packet so that they know where to send the error
info to. If it were possible for the router to look up the locator info
based on the identifier then the ICMP error reports wouldn't be an
issue, but if the lookup were expensive it could lead to possible DoS
attacks.  

Agreed.  However, ICMP error reports are already rate limited.  Whatever
mechanism there is for identifier to locator mapping (if any) needs to
be robust against attacks regardless of this possible abuse vector.

 Multicast also depends heavily on the source locator for RPF checking,
but multicast is already maintaining state so  guess if the source
locator info is in some early packets and the router caches the source
identifier to be associated with the state created then it might not be
necessary for the later packets to contain a source locator.  

With multicast, as always, things are reversed.  Then the question
becomes whether the destination locator provides any value.
 
Tony

---

From: Wesley Eddy <[email-masked]>
Date: Thu, 7 Dec 2006 14:51:44 -0500

> than comparing the source locator to the routing table.

A separate topic for this list might be whether we can simply deprecate
non-local ICMP error reporting.  The local functions like ND, MLD, etc.
are useful, but for non-local signaling ICMP is fairly poor.  PLPMTUD
is a significant step in this direction.  We should ask what part ICMP
plays in the architecture in reality, and if that differs from the part
it was originally intended to play or the part we think it should play.

The recent discussions in TCPM in San Diego and on the TCPM mailing list
may be interesting to people.

-- 
Wesley M. Eddy
Verizon Federal Network Systems

---

From: Bruce Curtis <[email-masked]>
Date: Thu, 7 Dec 2006 14:14:07 -0600

On Dec 7, 2006, at 1:23 PM, Tony Li wrote:

> If we imagine an example that is sort of the opposite extreme of  
> the model I was thinking of I can see your point that inclusion of  
> the source locator in the packet is not necessary at all for the  
> network to deliver the packet. However having the source locator in  
> the packet has been handy for anti-spoof RPF checks and there are  
> lots of standards that depend on having the source locator in the  
> packet.
>
> I think that's there's a huge separation between "handy" and  
> "required".  As we're positing new architecture, I think that we  
> should be very clear in separating the two.

   Yes.  I'm guilty at times of thinking too much in terms of the old  
architecture, the discussion on this list has been very interesting  
and has helped my thoughts to be less bound by the old architecture.

> I agree that having a source locator has been handy for RPF checks,  
> but is that the only way?  If the identifier space is structured,  
> then filtering on the identifier might be tractable.

   Someone on the list mentioned RPF checks by AS as a possibility.   
I can see that if the identifier were required to contain the AS or  
something similar that might work.  But on my campus it has been very  
handy to just enable the RPF checking on every subnet. That contains  
any spoofing as close to the source as possible.  I can see where it  
is likely possible to include enough structure in the identifier to  
identify the ISP customer but it might be more challenging to have it  
include info about each of my subnets, and if we do then isn't the  
identifier becoming closer to a locator again?

> And again, as we're positing new architecture anyway, compliance to  
> existing standards doesn't seem like it is strictly "required".  In  
> fact, if it is, it would seem challenging to make any forward  
> progress.

   Right, protocols, or even "existing code" would probably have been  
a better choice of words but in either case your point that changes  
are required is true.

>  Multicast also depends heavily on the source locator for RPF  
> checking, but multicast is already maintaining state so  guess if  
> the source locator info is in some early packets and the router  
> caches the source identifier to be associated with the state  
> created then it might not be necessary for the later packets to  
> contain a source locator.
>

   Hmm, good point.  As long as there is a destination identifier and  
the packet is somehow marked as multicast I can't think of a  
requirement for the destination locator.  Perhaps a destination  
locator or something similar might be handy for scope issues?    But  
then again that might be better handled by structure in the  
destination identifier?

---

---

From: John Day <[email-masked]>
Date: Thu, 7 Dec 2006 15:48:48 -0500

In a complete architecture, addresses are internal to what we call 
the network layer.  No one else needs to be aware of them.

John

At 11:23 -0800 2006/12/07, Tony Li wrote:
>source locator in the packet is not necessary at all for the network 
>to deliver the packet. However having the source locator in the 
>packet has been handy for anti-spoof RPF checks and there are lots 
>of standards that depend on having the source locator in the packet. 
>
>
>should be very clear in separating the two.  I agree that having a 
>source locator has been handy for RPF checks, but is that the only 
>way?  If the identifier space is structured, then filtering on the 
>identifier might be tractable.  And again, as we're positing new 
>architecture anyway, compliance to existing standards doesn't seem 
>like it is strictly "required".  In fact, if it is, it would seem 
>challenging to make any forward progress.
>
>
>   PMTU Discovery and all of the ICMP error reports depend on having 
>the source locator in the packet so that they know where to send the 
>error info to. If it were possible for the router to look up the 
>locator info based on the identifier then the ICMP error reports 
>wouldn't be an issue, but if the lookup were expensive it could lead 
>to possible DoS attacks.  
>
>
>Agreed.  However, ICMP error reports are already rate limited. 
>Whatever mechanism there is for identifier to locator mapping (if 
>any) needs to be robust against attacks regardless of this possible 
>abuse vector.
>
>
>caches the source identifier to be associated with the state created 
>then it might not be necessary for the later packets to contain a 
>source locator.  
>
>

---

From: "Tony Li" <[email-masked]>
Date: Thu, 7 Dec 2006 13:43:40 -0800

I agree that having a source locator has been handy for RPF checks, but
is that the only way? If the identifier space is structured, then
filtering on the identifier might be tractable. 

Someone on the list mentioned RPF checks by AS as a possibility. I can
see that if the identifier were required to contain the AS or something
similar that might work. But on my campus it has been very handy to just
enable the RPF checking on every subnet. That contains any spoofing as
close to the source as possible. I can see where it is likely possible
to include enough structure in the identifier to identify the ISP
customer but it might be more challenging to have it include info about
each of my subnets, and if we do then isn't the identifier becoming
closer to a locator again? 

I can see at least two possible directions here:
 
The 'identifier' can be opaque.  If your particular site requires
sufficient scale, then you may choose to add more structure to your
particular identifier space.
 
We could also generalize the locator to be hierarchical.  Thus, the
locator would have a global portion and then be hierarchically
allocated, with as much depth as is required and appropriate.  The
challenges with this are that at any level, the bit boundaries between
locator levels are not fixed, so equipment needs to be more flexible.
Further, we would be attempting to fit arbitrary levels of hierarchy
into a fixed field.
 
The latter seems cleaner to my mind.

Hmm, good point. As long as there is a destination identifier and the
packet is somehow marked as multicast I can't think of a requirement for
the destination locator. Perhaps a destination locator or something
similar might be handy for scope issues? But then again that might be
better handled by structure in the destination identifier? 

Exactly.
 
Tony

---

From: "Bound, Jim" <[email-masked]>
Date: Thu, 7 Dec 2006 22:47:51 -0500

Good brainstorming on GPS.
 
thanks
/jim

	From: Bruce Curtis [mailto:[email-masked]] 
	Sent: Thursday, December 07, 2006 1:11 PM
	To: [email-masked]
	Subject: Re: [arch-d] It's about Information

		I like this thinking model.  Why is this not a form or
source address routing with a lot of other addded value using locators. 

	  The way I thought about the model all of the routing was still
based on the multiple destination addresses and the source addresses
would just be used for anti-spoof RPF checking.  But if the source info
is in every packet lots of other options I hadn't considered are
possible with this model as well.

		I also believe that GPS information can add great value
to routing and doing some research on that concept for ad hoc nets but
have assumed no way could we ever get a standard done to use it?
		 
		/jim

	   When I was thinking through how things would work with this
model I was thinking of the GPS info as more of an ID than a locator.
Sometimes when there is a problem host it would be nice to have it's
location info, but there are privacy issues and perhaps other downsides
as well.

	   But one way to use the same model but turn my thinking on
it's head is to consider the GPS info as the primary locator and the
other multiple locators as ISP specific locators.  The question that
comes to mind is if that would lessen the ISP objections to the GPS
based address model.  I'm not an ISP so the answer doesn't come
naturally to me, I'll have to think about if for a while.

			From: Bruce Curtis
[mailto:[email-masked]] 
			Sent: Monday, December 04, 2006 6:11 PM
			To: [email-masked]
			Subject: [arch-d] It's about Information

			  This may be obvious to everyone since I
haven't seen much discussion about this model but it helped me
understand the issues. 

			  In the IPv4 multihoming model the information
about multiple paths from host A to host Z is in the routing table of
the core routers in the Internet.

			        ISP 1   ----------------       I
	
n
	
e
	
t

			  If we change the model and take the
information about multiple paths out of the core routers and give each
host a different IP number for each of it's ISPs, then to preserve some
of the properties of the old model we still need  to have the multiple
path information available to the network somehow.

			         ISP 1   ----------------       I
	
n
	
e
	
t

			  If we imagine a time in the future where every
host has 100 Mbps connection to the Internet like some homes in Japan
today and we have networks that support Jumbo frames it's easier to
consider the possibility that we put the required information in every
packet from host A,B,C to host X,Y,Z.  That is, we put all of the
destination addresses X,Y,Z in the packet as well as the source
addresses A,B,C.  
			  So basically we take the information about
multiple paths out of the routing tables where it doesn't scale and put
the multiple path info in each packet, which should scale right along
with the speed of the router.

			  If we put this information in each packet then
the new model has several properties that the old model has.  For
example:

			         Traffic engineering can be done at any
point in the path.
			         
			          Anti-Spoof checking can be done at the
interface between the customer and ISP.

			          If a link is down between any ISP and
host X,Y,Z the internet can reroute the packet
			          to one of the other addresses for
X,Y,Z.

			  The more I thought about it the more sense
this model made to me and it helped me understand many of the other
proposed alternatives as attempts to optimize this model and remove some
of the information from the packet.

			  Also it made it clearer to me that the
discussion about locator/ID split has more to do with trying to preserve
a single address representation to current and future software and that
it was the multiple locators and not the locator/ID split that helps
solve the routing portion of the multihoming problem.

			  To further understand the possibilities I
realized that IPv6 did not provide the means to include all of these
addresses but IPv6 does have a method of being extended to include such
addresses, extension headers.  Actually putting the multiple destination
addresses in extension headers might not be too much different from the
current hop-by-hop extension headers.   (And there would have to be
something in the standard to avoid loops in the case of a link failure,
something that marks an address as already having failed, but again that
might not be too different from the hop-by-hop extension headers.)

			  Considering the extension headers helped me
move one step closer to the real world and also to envision at least
one optimization.  To me it seems that the natural point to discover the

			  There has been talk on the list about not
wanting to overload identifiers but it has been helpful to me to think
of unique unchanging IDs for hosts in terms of the above model.  If we
add another IP number to each host so we have host A,B,C,D and host
W,X,Y,Z but D and W have no associated ISPs things still work.  It is
about the same case as  if D and W did have an associated ISP but the
links to those ISPs were down.  IPs D and W could even be assigned based
on GPS location or something similar if they where still guaranteed to
be unique.  The identifiers are overloaded but it might be workable as
an interim solution, something similar to shim6.

			  Another possible optimization could be to
strip off the multiple source addresses at the interface between the
customer and the ISP and just leave the one that is associated with that
ISP.  But then the receiving host wouldn't automatically know all of the
return addresses of the sending host.
			  
			  Sorry if this was obvious to everyone but I
thought the idea of putting all of the required info in the packets
should at least be floated and shot down and hopefully the shooting down
and discussion would be useful in the long run.  To me considering the
extreme case was helpful in understanding the problem and in classifying
some of the proposed alternatives.

			---
			Bruce Curtis
[email-masked]
			Certified NetAnalyst II
701-231-8527

	---

---

From: "Bound, Jim" <[email-masked]>
Date: Thu, 7 Dec 2006 22:55:56 -0500

I agree in principle with Tony below when I said source routing I was
being lazy. I mean't a means as Bruce suggested using an extenion or
hop-by-hop to hold locators and the variants Bruce has suggested.  I am
getting convinced from others we really need to permit the source to be
tool for routing path optimizations at points on the network, and remove
from needs for security and transport layer. But if we were able to do
that some questions we need to answer:

1. make sure none use src ever for Ipsec make it a MUST of course if we
find an identifier for transport connection-id associations we might not
even need the IP address for IPsec including dst address.

2. the src address is the way the node responds to its peer in most
cases today whatever we do best if we do not add performance hit for
lookups for dst address except at initial handshake, unless some update
to an identifier above changes the connection-id state then thats ok and
assume there would be valid reason for the user to take the performance
hit.

3. others I am sure.

Just brainstorming.

/jim 

> -----Original Message-----
> From: Tony Li [mailto:[email-masked]] 
> Sent: Thursday, December 07, 2006 1:21 PM
> To: Bruce Curtis
> Cc: [email-masked]
> 		I like this thinking model.  Why is this not a 
> form or source address routing with a lot of other addded 
> value using locators. 
> 		
> 
> 
> 	  The way I thought about the model all of the routing 
> was still based on the multiple destination addresses and the 
> source addresses would just be used for anti-spoof RPF 
> checking.  But if the source info is in every packet lots of 
> other options I hadn't considered are possible with this 
> model as well.
> 
> 
> 
> I think that there's an important distinction here: it's 
> vital to the upper layers that packets contain a source 
> identifier.  However, the inclusion of a source locator is 
> not clearly necessary.
> 
> Tony
> 
> 
>

---

From: "Bound, Jim" <[email-masked]>
Date: Thu, 7 Dec 2006 22:56:32 -0500

I again agree with Tony.   If we did it right locator would not be
required at all.
 
/jim

	From: Bruce Curtis [mailto:[email-masked]] 
	Sent: Thursday, December 07, 2006 2:02 PM
	To: [email-masked]
	Subject: Re: [arch-d] It's about Information

				I like this thinking model.  Why is this
not a form or source address routing with a lot of other addded value
using locators. 

			  The way I thought about the model all of the
routing was still based on the multiple destination addresses and the
source addresses would just be used for anti-spoof RPF checking.  But if
the source info is in every packet lots of other options I hadn't
considered are possible with this model as well.

		I think that there's an important distinction here: it's
vital to the upper layers that packets contain a source identifier.
However, the inclusion of a source locator is not clearly necessary.

		Tony

	  It's vital to the stacks on the end hosts that the packets
contain a source identifier.  If we imagine an example that is sort of
the opposite extreme of the model I was thinking of I can see your point
that inclusion of the source locator in the packet is not necessary at
all for the network to deliver the packet.  However having the source
locator in the packet has been handy for anti-spoof RPF checks and there
are lots of standards that depend on having the source locator in the
packet.  PMTU Discovery and all of the ICMP error reports depend on
having the source locator in the packet so that they know where to send
the error info to.  If it were possible for the router to look up the
locator info based on the identifier then the ICMP error reports
wouldn't be an issue, but if the lookup were expensive it could lead to
possible DoS attacks.  Multicast also depends heavily on the source
locator for RPF checking, but multicast is already maintaining state so
I guess if the source locator info is in some early packets and the
router caches the source identifier to be associated with the state
created then it might not be necessary for the later packets to contain
a source locator.  But that could potentially lead to some scaling
issues for the multicast RPF check if the check was done against a cache
of source identifiers rather than comparing the source locator to the
routing table.

	---

---

From: "Bound, Jim" <[email-masked]>
Date: Thu, 7 Dec 2006 22:59:23 -0500

question: is your vision for example moving ND messages (sorry the last
one I actually wrote code for) functions to a layer above IP?

thx
/jim 

> From: Wesley Eddy [mailto:[email-masked]] 
> Sent: Thursday, December 07, 2006 2:52 PM
> To: Bruce Curtis
> >   It's vital to the stacks on the end hosts that the 
> packets contain a 
> > source identifier.  If we imagine an example that is sort of the 
> > locator in the packet has been handy for anti-spoof RPF checks and 
> > there are lots of standards that depend on having the 
> source locator 
> > in the packet.  PMTU Discovery and all of the ICMP error reports 
> > depend on having the source locator in the packet so that they know 
> > where to send the error info to.  If it were possible for 
> the router 
> > to look up the locator info based on the identifier then the ICMP 
> > error reports wouldn't be an issue, but if the lookup were 
> expensive 
> > it could lead to possible DoS attacks.  Multicast also 
> depends heavily 
> > on the source locator for RPF checking, but multicast is already 
> > maintaining state so I guess if the source locator info is in some 
> > early packets and the router caches the source identifier to be 
> > associated with the state created then it might not be 
> necessary for 
> > the later packets to contain a source locator.  But that could 
> > potentially lead to some scaling issues for the multicast 
> RPF check if 
> > the check was done against a cache of source identifiers 
> rather than 
> > comparing the source locator to the routing table.
> > 
> 
> A separate topic for this list might be whether we can simply 
> deprecate non-local ICMP error reporting.  The local 
> functions like ND, MLD, etc.
> are useful, but for non-local signaling ICMP is fairly poor.  
> PLPMTUD is a significant step in this direction.  We should 
> ask what part ICMP plays in the architecture in reality, and 
> if that differs from the part it was originally intended to 
> play or the part we think it should play.
> 
> The recent discussions in TCPM in San Diego and on the TCPM 
> mailing list may be interesting to people.
> 
> --
> Wesley M. Eddy

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 8 Dec 2006 13:52:08 +0100

Hi Bruce,

El 05/12/2006, a las 0:11, Bruce Curtis escribió:

> routing tables where it doesn't scale and put the multiple path info 
> in each packet, which should scale right along with the speed of the 
> router.
>
>           Anti-Spoof checking can be done at the interface between the 
> customer and ISP.
>
>   The more I thought about it the more sense this model made to me and 
> it helped me understand many of the other proposed alternatives as 
> attempts to optimize this model and remove some of the information 
> from the packet.
>
>   To further understand the possibilities I realized that IPv6 did not 
> provide the means to include all of these addresses but IPv6 does have 
> a method of being extended to include such addresses, extension 
> headers.

i think that a difficulty with this is that you not only need to 
include the alternative locator information, but also provide the means 
to secure the binding between the identifier and the alternative 
locators in order to prevent redirection attacks. This could be done 
adding security information in the packet itself, but this is quite 
much more information than just the locators. The other option would be 
to verify the binding from a third trusted party, but in this case, 
what is the point of including the alternative locator information in 
te first place? (you could obtain it from the third trusted party)

Regards, marcelo

PS: you can check 
http://www.watersprings.org/pub/id/draft-bagnulo-multi6-mhexthdr-00.txt 
if you are interested in a broken version of a similar idea that 
completelly missed all the security problems :-)

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 8 Dec 2006 08:05:56 -0600

Hi, Bruce,

("cough") but wouldn't ingress filtering based on one address in the set of addresses being topologically correct just encourage source address spoofers to be more careful about the addresses they spoof with? 

Thanks,

Spencer
  From: Bruce Curtis 

    Right but the model with all the info in the packet allows either option, in one person's network they can let the host chose the path and the packets flow as they may and in another person's network they can have intelligent boxes at the customer ISP interface to do load balancing or traffic engineering.

        Also I don't think that shim6 includes the multiple source  addresses in the packets which led to the complaint that implementing  anti-spoofing was then difficult or impossible.  Or that shim6 had to  do extra work to put the correct source address in the packet so that  anti-spoofing still worked the old way.  But again this complaint was  probably about wanting to do traffic engineering for load balancing  at the customer ISP interface also.

    Right, but if all of the address info was in the packet we could envision an RPF check that succeeds as long as only one of the multiple source addresses was correct.  Also it would be easier for the ISP to implement the RPF check if they chose to, they wouldn't need to know the customers other possible locators, they would only have to check the list of source addresses to see if one of them matched that ISPs address range.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 8 Dec 2006 08:10:14 -0600

I wasn't Present At The Creation, but my understanding of ICMP says this 
would be a good thing to think about early in the discussion. Between ICMP 
uses that just don't work (source quench) and ICMP uses that don't work 
because of too-careful firewalling, etc. we can't rely on ICMP for much, 
once we leave the local network.

Thanks,

Spencer

---

From: Bruce Curtis <[email-masked]>
Date: Fri, 8 Dec 2006 12:19:33 -0600

On Dec 8, 2006, at 8:05 AM, Spencer Dawkins wrote:

> Hi, Bruce,
>
> ("cough") but wouldn't ingress filtering based on one address in  
> the set of addresses being topologically correct just encourage  
> source address spoofers to be more careful about the addresses they  
> spoof with?
>
> Thanks,
>
> Spencer

   Well basically address spoofers could only totally spoof the  
source locators that don't match the initial path the  packet took  
from the customer to the ISP.  One of the source locators would have  
to be correct.

    The additional source locators would make tracking the attack  
back slightly more difficult than today.  The attacker could use lots  
of randomly chosen source locators in the packets  but then after  
looking at a few packets it should become obvious which one never  
changes and is the real source locator.  If the attacker chose to put  
for example 5 source locators in each packet and never changed them  
it would not be obvious which one indicated the real path the packet  
took from customer to ISP, but it would be limited to 5  
possibilities.  One method around the problem would be for ISPs to  
mark the packet in some way to indicate which source locator passed  
an anti-spoof test.

    Or along the lines of some other discussion in the group, if the  
source locators don't provide value past the customer ISP interface  
they could be deleted from the packet at the point they pass an anti- 
spoof test.

> option, in one person's network they can let the host chose the  
> path and the packets flow as they may and in another person's  
> network they can have intelligent boxes at the customer ISP  
> interface to do load balancing or traffic engineering.
>
>>
>>> that shim6 had to  do extra work to put the correct source  
>>> address in the packet so that  anti-spoofing still worked the old  
>>> way.  But again this complaint was  probably about wanting to do  
>>> traffic engineering for load balancing  at the customer ISP  
>>> interface also.
>>

---

---

From: Bruce Curtis <[email-masked]>
Date: Fri, 8 Dec 2006 13:33:41 -0600

On Dec 8, 2006, at 6:52 AM, marcelo bagnulo braun wrote:

>> extension headers.
> include the alternative locator information, but also provide the  
> means to secure the binding between the identifier and the  
> alternative locators in order to prevent redirection attacks. This  
> could be done adding security information in the packet itself, but  
> this is quite much more information than just the locators. The  
> other option would be to verify the binding from a third trusted  
> party, but in this case, what is the point of including the  
> alternative locator information in te first place? (you could  
> obtain it from the third trusted party)
>
> Regards, marcelo
>
> PS: you can check http://www.watersprings.org/pub/id/draft-bagnulo- 
> multi6-mhexthdr-00.txt if you are interested in a broken version of  
> a similar idea that completelly missed all the security problems :-)

   I guess I'm also still missing the security problems with multiple  
destination locators.  Can you give an example?

   I do see that with multiple source locators an attacker could use  
it's real source locator plus an additional spoofed source locator to  
try to highjack a session.

---

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 8 Dec 2006 20:52:55 +0100

> destination locators.  Can you give an example?

see rfc 4218 for an analysis to threats in multihoming with multiple 
addresses and see rfc4225 for an analysis of the threats in the 
mobility environment

regards, marcelo

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 8 Dec 2006 21:50:53 +0100

> addresses and see rfc4225 for an analysis of the threats in the 

to be more explicit

suppose that node A has multiple locators and node b is sending packets 
to node a

i think that the critical issue here is how does the node b learns (and 
verifies) all the addresses of node a. If no proper protection is 
provided, this can result in redirection attacks.

Besides, in this case, not only node b is using the multiple locator 
information, but also intermediate routers are using it. i am not sure 
how much verification need to make these other parties (routers) that 
are acutally using the multiple address informtion in order not to 
introduce new vulnerabilities. I don't think that the threats are so 
big as in the case of the actual peer of the communication (node b in 
this case) though

regards, marcelo