Subject: [MEXT] About flow binding directionality

From: marcelo bagnulo braun <[email-masked]>
Date: Sun, 19 Jul 2009 18:10:40 +0200

Hi,

I have read the long discussion about this.
It seems that in order for the HA to be able to send
meaningful/useful information to the MN about a flow binding,
either:

1- the HA knows something relevant that is not available to the
MN about the path conditions between the HA and the different CoAs,
or,

2- The HA has enough information to make a clue-full decision and
we want to support a model where the network side influences the MN
choices.

Let me try to take a stab on those, and i am confident you will let
me know what i am missing.

Let's start by the first case i.e.
1- the HA knows something relevant that is not available to the
MN about the path conditions between the HA and the different CoAs,

Suppose the following scenario.

     +-----+
     | CN  |
     +-----+
        |
 +------------------+
 |      ISP3        |
 |                  |
 +------------------+
        |
+----------------------------------------+
| Home Network                           |
|                +----+                  |
|                | HA |                  |
|                +----+                  |
|                 |  |                   |
|        +--------+  +---------+         |
|        |                     |         |
+--------|---------------------|---------+
         |                     |
         |                     |
   +------------+    +---------------+
   | ISP1       |    | ISP2          |
   |            |    |               |
   +------------+    +---------------+
       |                  |
       |        +----+    |
       +--------| MN |----+
           CoA1 +----+ CoA2

So, in case it is not obvious, i describe it a bit
more in detail.
Assume that the home network has 3 providers.
ISP1 that is the same that it is providing one access to the MN 
ISP1 that is the same that it is providing the other access to the MN
And ISP3, that is connected to the CN

(i know this is a somehow forced example, but it is easily generalizable
to more complex scenarios, where the HA could play a useful role in
traffic engineering for the home network)

So, for obvious reasons, the traffic flowing from the MN to the HA through
CoA1 will flow through ISP1 and the traffic flowing from the MN to the
HA through CoA2 will flow through ISP2.

Now, suppose that the link between the ISP1 and the home network is 
becoming
congested, or is about to reach the expected quota or whatever traffic
management issue that is perfectly known to the home network.

Since the HA, as it is located in the home network, it would make sense
that the HA can have easy access to that circumstance and it would
make sense that the HA asks the MN to move the traffic from the CoA1 to the
CoA2.

Of course, this is not all the information that is needed to actually 
decide
what is the best CoA, but i think that the point is that this is relevant
information for the MN to have. Of course, the problem is that what if 
the MN
actually believes that the other CoA is the right one to use?
I believe that this is up to the MN  policy and internal heuristics, about
how to make a reasonable decision considering all the input it has.
At this point, the point that i am trying to make is that the HA may have
access to information that the MN has not and if anything, if the MN
has no preference, it can follow the HA suggestions.

Does this makes sense to you?

Let me try the second case i.e.

The HA has enough information to make a clue-full decision and
we want to support a model where the network side influences the MN
choices.
 
Now, in order for this work, we need to assume a bunch of things,
which will certainly not be true in the general case, but may be
true in some specific cases.

First of all, we need to agree whether there is some value in allowing
the HA (i.e. the network side) to micro manage the MN.
If not, then, this case has no value.
If yes, then i guess that under a set of assumptions, this can be done.
I understand that the assumptions basically comes down to the things
that are needed for the HA to have the information.

So these assumptions include:
- The HA knows what access is associated to each CoA
- The HA knows the situation of the visited network, including
the radio access link with the MN. This essentially
implies that the HA and the visited network are the same admin domain.
- The HA also knows the situation of the rest of the network, in order
to have a full picture of the different paths.
- The HA knows something about the flows. This can be achieved cause
the apps running in the MN are managed by the network, or cause they
usually use the well known ports in a reasonable way or because they
the late binding approach (i.e. the HA moves the existent flows once
it is aware of their characteristics e.g. the rate fo the flow)

So, maybe the assumptions can be relaxed, but i think that the point is
that under a set of assumptions that may match to a viable scenario,
it is possible for the HA to manage the MN.

So, does people agree that this is a viable scenario?
And does people see any value in having the network side managing the MN?

Fire up.

We should try to wrap this up for ietf75....

Regards, marcelo

---

From: Hesham Soliman <[email-masked]>
Date: Mon, 20 Jul 2009 11:33:21 +1000

>      | CN  |

=> Never mind the scenario's simplicity, I have two simple questions that I
need an answer for in order to be able to consider this scenario:
1. How does the HA learn the Path corresponding to a particular CoA?
Assuming that the internet is a bit complex with potentially one or more
transit networks that are meshed in some cases.

2. If the HA is not a border router on its site, how does it know which
ISP's link is up/congested in real time? Is there an RFC or best practices
link? This has to be in real time for the scenario to make sense.

I think the second scenario is making way too many assumptions that it's
almost a scenario for corner cases that don't exist, so I'll stick with
discussing this scenario for now.

Hesham

> _______________________________________________
> MEXT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mext

---

From: marcelo bagnulo braun <[email-masked]>
Date: Mon, 20 Jul 2009 08:51:46 +0200

> transit networks that are meshed in some cases.

correct (this is why i stated that the described scenario was simplistic)
I think that this can be obtained from BGP. I mean, If the home site is 
multihomed it is not uncommon it runs BGP.
BGP will know what CoA prefix is reachable through which ISP. Now, we 
need to assume symmetric traffic i.e. the the forward path is the same 
than the reverse path. I think this is fairly common in the fringes (it 
is certainly not common across big providers the peer in multiple 
locations, but this doesn't seem to be the case here)

> link? This has to be in real time for the scenario to make sense.
Well, i think management protocols like SNMP can tell the HA link load 
information, correct?

That information can be used by the HA to tell which link is more loaded.

In general, today, when multihomed sites want to move traffic from one 
link to another, they simply move a bunch of destiantions that are 
coming from one link to anohter one. You actually could do the same here.

Suppose that you have half of your MN using CoA1 and half of them using 
CoA2 (just to have a simple example).
Now, the link with ISP1 is heavily loaded and the one with ISP2 has 
little traffic.
It would be enough to start moving MNs that are using CoA1 to use CoA2.

I know this all sounds a bit complex, but i think in general traffic 
engineering across domains is something pretty complex, so i am not sure 
this is beyond what current bgp techniques require.

Makes sense?

Regards, marcelo

---

From: marcelo bagnulo braun <[email-masked]>
Date: Mon, 20 Jul 2009 11:07:10 +0200

Hesham Soliman escribió:
>> I think that this can be obtained from BGP.
>>     
>
> => Ok how do you actually get that from BGP, what protocol does the HA run
> to the BGP router to get this?

yes.
The HA would run BGP and would obtain what is the next hop for each of 
the CoAs.
Based on that the HA would know which ISP is being used to send traffic 
to each of the CoAs.

Now, the next point is what direction is congested.
If the direction from the HA to the MN is the one that is congested, 
then the HA
has a very tight control and knows that if it starts sending the traffic 
to the other
CoA it would move traffic from one ISP to the other one. that would 
solve half of the
problem (i.e. one direction). In order to solve the other part, we need 
to assume that
the traffic will flow simetrically. While this is not guranteed, it is 
as i stated before, not
uncommon espcially in the fringes.

>  SNMP?
>
>  I mean, If the home site is
>   
>> BGP will know what CoA prefix is reachable through which ISP.
>>     
>
> => But how common is that for a multihomed site? I mean, when a site is
> multihomed, how common is it that 50% of the Internet reaches me through
> ISP1 and the other 50% reaches me through ISP2?
If you have two links of the same capacity, this is exaclty what you 
want, right?
So, there are a bunch of Traffic enginnering technique to try to achieve 
exactly that.

The point that i am trying to make is that the HA could also help in 
this task.

>  And how can that policy be
> enforced for inbound traffic assuming that these are independent ISPs?
>   

There is no enforcement here. It is more about playing the game to
try to get what you want.
People tweak BGP to try to move traffic from one link to another. this is
exactly the same game the HA would start playing in order to get
load balancing

> Are we making too many assumptions due to the simplification of the
> scenario?
>
>   
for me, the strongest assumption being made is about the traffic being 
symmetric.
However, this only affects half of the traffic (i.e. on one of the 
directions) and
when you are delaing with load balancing, it is pretty much a game that 
is played by
approximation, so, you start moving knobs until you get what you want.

> => I don't know if they give that information in real time.
AFAIK, they can provide information about the number of bytes course 
over the last n seconds.
If the HA gets that information periodically, it can estimate the 
bandwidth consumption
on each link

>  I also don't
> know what kind of heuristics we're expecting the HA to follow to come to an
> intelligent decision to move some MNs given that it has no idea about how
> useful that move is for the MN's other destinations or applications that it
> may be using.  
>
>   
well, if the HA knows the BGP information and knows the link load of 
each exit link, it can
know what CoA is reachable through each link and can decide to move some 
of the
ones that are using the CoA that corresponds to the congested link to 
the non congested link

this will provide very precise load balancing for the HA to the MN 
direction and it is likely
to have a significant effect on the reverse direction as well.

> => Yes but this is different from what you're describing, you want to signal
> to a remote host to change it's source because you think that that src will
> come through ISPX. You have to be sure that:
>
> - This is *always* the case
>   
not really
I am trying to achieve load balancing
If this is true for a significant number of the cases, it is good enough.

> - The new address is not going to suffer a worse performance (don't know how
> you can know that)
>   

As i said earlier, the point being made is that the HA can provide some 
input information to make
the decision.
If both paths are equally acceptable to the MN, why wouldn't the MN take 
into account the HA
preferences?
Of course, if the MN knows that one CoA is much worse than the other 
one, it is likely that
the MN will keep on sending the traffic through the good one, no matter 
what the HA asks. But that
is ok. This is usually the case in traffic management. Eahc party does 
what the other is asking
unless it goes against their own interests.
So this applies both the the direction from the HA to the MN.
If the HA knows that the path to CoA1 is worse for it than the one 
through the CoA2. it will
use the one through CoA2 no matter what the MN says.

So, the point being made is that this is useful input for the MN to 
consider in its decision process.

> - The information you have is based on a trend and is timely
>
>   
right, but this is the best you can do when it comes to load balancing, 
right?

> => I think everything is simple on this level, but it all relies on your
> assumption that the HA can know that path to always be true.
as i stated earlier, it dones' have to be always true.
It has to be true a significant number of times (at least more than half 
of the times :-)

>  How realistic
> is it that an unknown part of the IP address space can reach a site through
> 1 ISP?
>
>   
As i said above, it is certianly always true in the HA to the MN direction.
In the reverse direction, it is more uncertain, and it will depend on 
the peering agreements
and on the topology of the home network.
I think it will be true in some cases and not so true in other cases.
However, this can actually be enforced by peering agreements.

> => It may be understood for BGP but we're dealing with a different scenario
> here that involves end hosts making choices based on completely different
> factors and relying on e2e measurements to know what's good for them and
> their apps. I think doing something like this from the HA can lead to many
> uninformed decisions as oppsosed to keeping things e2e.
>
>   

while i am sympathetic to the e2e argument, i think that if we actually 
go that path,
we actually need to do things very differently and use multipath TCP or SCTP
The reason why i say that is that the IP layer cannot properly 
characterize the e2e
path cause it cannot measure e2e congestion.
So, having the HA provide additional path information could help the MN 
to get
a more complete view.

>> Makes sense?
>>     
>
> => Not to me. I'm not convinced that the HA can detect paths for all
> scenarios and know enough to move traffic ...etc, especially when the
> alternative (e2e) seems to be working so far. What advantage does this
> solution (if it works) have over e2e approaches?
>   
As i said above, what do you mean by e2e?
Current MIP approach cannot characterize the congestion of the e2e path.
Moreover, current MIP approach cannot even detect e2e failures.
So, i would believe that multipath tcp is a superior alternative, but
the case it is not so clear to me for the flow binding case.

Regards, marcelo

---

From: Hesham Soliman <[email-masked]>
Date: Mon, 20 Jul 2009 21:18:17 +1000

> CoA it would move traffic from one ISP to the other one.

=> The HA can't decide to send traffic to a different CoA from the one in
the BC. I think you mean it will tell the MN to switch CoA, that's all it
can do. 

that would 
> uncommon espcially in the fringes.

=> But do you mean it's common regardless of where the MN is located? I mean
if I'm in NY and have a split VPN configuratation and my HA is in Madrid, we
can still be ok with this symmetry?

> So, there are a bunch of Traffic enginnering technique to try to achieve

=> I don't know them but I'll take your word, although I don't really know
how you can be sure about forcing packets from certain sources anywhere on
the Internet to come in a certain inbound link.

> The point that i am trying to make is that the HA could also help in

=> "Help" is fine, "sending rules" is not fine. So if you only want to send
a suggestion (there is a lot to discuss about the content of such
suggestion) then I can understand that a lot more than telling the MN to
switch. There is also a good case for sending policies of course but that's
another topic. 

> for me, the strongest assumption being made is about the traffic being

=> Yes that's a significant one.

> approximation, so, you start moving knobs until you get what you want.

=> I don't really care about how good the load balancing outcome will be,
I'm only concerned about sending the wrong information to the MN.

> If the HA gets that information periodically, it can estimate the

=> ok.

> ones that are using the CoA that corresponds to the congested link to

=> So, given your earlier statement about how traffic engineering can be
used to move inbound traffic from one ISP to another, why do you need to
change the src address in this case instead of using the same traffic
engineering techniques to do load balancing? Is it not something that can be
done dynamically? Or is it because you need finer control?

> If this is true for a significant number of the cases, it is good enough.

=> Yes but if half the time you detect the path incorrectly you will be
sending the wrong information to the MN half the time. You might still get
load balancing but you make the MNs switch addresses for no reason. I don't
know if that has drastic consequences but it only seems to work for large
numbers. So that part of the protocol will be labelled "For large numbers
only" :) 

> If both paths are equally acceptable to the MN, why wouldn't the MN take

=> Ok, a suggestion makes me feel a bit better about this, but you could
still end up disrupting service if the MN switches then discovered that the
e2e nature of the other address is terrible for its connection.

> what the HA asks.

=> Ok, if it knows apriori, I agree.

> It has to be true a significant number of times (at least more than half

=> Understood, from the home network admin's point of view this is fine, I
don't think it's fine from a protocol design point of view to say "this will
work for large numbers only".

> path cause it cannot measure e2e congestion.

=> Why not? e2e delays and connection quality are used by content adaptation
techniques regularly. You don't need multi-path TCP for that. You're only
trying to work out the e2e properties of a particular connection.

> Current MIP approach cannot characterize the congestion of the e2e path.

=> MIP is only a tool that acts on the request of another engine making
decisions. The input to such decisions can be e2e information coming from
the transport layer. So, yes of course MIPv6 can't do this e2e, but it
doesn't mean it can't be done by the host. MIP is only a tool in the toolset
and doesn't actually provide any intelligence in decision making process
involved in selecting an interface.

> Moreover, current MIP approach cannot even detect e2e failures.

=> Like I said, it's not a MIP function, a transport layer can detect
failure and provide hints. RFC 4861 mentions such hints that can be useful.
Only a very small part of the puzzle is solved by MIP.

Hesham

---

From: Hesham Soliman <[email-masked]>
Date: Mon, 20 Jul 2009 18:13:48 +1000

=> Ok how do you actually get that from BGP, what protocol does the HA run
to the BGP router to get this? SNMP?

ISP1 and the other 50% reaches me through ISP2? And how can that policy be

> Well, i think management protocols like SNMP can tell the HA link load

=> I don't know if they give that information in real time. I also don't
know what kind of heuristics we're expecting the HA to follow to come to an
intelligent decision to move some MNs given that it has no idea about how
useful that move is for the MN's other destinations or applications that it
may be using.  

=> I think everything is simple on this level, but it all relies on your
assumption that the HA can know that path to always be true. How realistic
is it that an unknown part of the IP address space can reach a site through
1 ISP?

Hesham