Subject: mboned: AMT Encapsulation Suggestion

From: Marshall Eubanks <[email-masked]>
Date: Wed, 5 Apr 2006 11:39:57 -0400

Hello;

(Speaking only as a WG member.)

There were two main issues brought up with AMT at the Dallas IETF :

- What encapsulations should be supported and
- should end node sourcing be supported.

I did not hear consensus on either issue; both need to be resolved to  
take this to WGLC.

(Another question that I had recently presented to me is how well the
current protocol will work with the TURN, STUN and
ICE IETF firewall / NAT traversal protocols. Some additional text  
might be useful on this subject.)

In this e-mail, I would like to focus on and make a suggestion about  
encapsulation. I haven't seen much email traffic on this, please  
accept my apologies if I missed some.

The issue is, as I understand it, that the authors would like to  
support only UDP encapsulation, as that
makes it easier to support firewall traversal, while some  
implementors would rather support instead or in addition GRE  
encapsulation, as this has better router hardware support.

While in principle any encapsulation could be supported, if more than  
one is supported, then the feeling at the meeting was that there  
would need to be encapsulation discovery in the protocols. (There are  
of course other encapsulation protocols, such as IP in IP, but I have  
heard no call to support them. I counted 8 encapsulation protocols  
listed in a talk in another IETF WG meeting in Dallas.)

Worse than requiring the insertion of encapsulation discovery,  
support of multiple encapsulation seems to break the protocol. Of  
course any major change may break existing implementations, but there  
is a worse potential problem. Because an  anycast address is used to  
reach the nearest AMT Relay Router, if some relay routers (say)
support UDP and some only support GRE (and the same for the AMT  
gateways), then there is a possibility of
having a UDP encapsulation only Relay Router resolving for a GRE only  
gateway, or vice versa, which of course would not work. With anycast,  
you do not get to pick your partner, so there would be no way to fix  
this.

If this short description of the problem is incorrect, please correct  
me.

If the WG desires to support multiple encapsulations, it seems to me  
that there there is a solution that would solve the encapsulation mis- 
match problem.

To quote from the draft

    AMT relays and gateways cooperate to transmit multicast traffic
    sourced within the native multicast infrastructure to AMT sites:
    relays receive the traffic natively and unicast-encapsulate it to
    gateways; gateways decapsulate the traffic and possibly forward it
    into the AMT site.

When a encapsulated stream is set up, through an IGMP membership report,
there is a 3-way handshake, described as follows :

    When a gateway or relay wants to send a membership report, it first
    sends an AMT Request with a request nonce in it.  The receiving side
    (the respondent) can calculate a message authentication code (MAC)
    based on (for example) the source IP address of the Request, the
    source UDP port, the request nonce, and a secret key known only to
    the respondent.  The algorithm and the input used to calculate the
    MAC does not have to be standardized since the respondent generates
    and verifies the MAC and the originator simply echoes it.

    An AMT Membership Query is sent back including the request nonce and
    the MAC to the originator of the Request.  The originator then sends
    the IGMP/MLD Membership/Listener Report or Leave/Done along with the
    request nonce and the received MAC back to the respondent finalizing
    the 3-way handshake.

The basic way I see to support multiple encapsulations is to add  
signaling to the 3 way handshake.

Specifically, bits 8 through 15 (at least) are
"reserved" in each AMT message, and could be used to signify  
encapsulation status. At present, they are
"Sent as 0, ignored on receipt."

So, my suggested solution is :

UDP MUST be supported.

Bit 8 in the AMT message is used to signify both a preference for and  
support of GRE encapsulation.
(UDP is the default, and does not get a bit. That way, for UDP only  
support, the message is unchanged and the action is unchanged from  
the existing protocol, so this will not break existing implementations.)

I think that is it reasonable that the relay decide which  
encapsulation should be used, as in general it will be doing more work.

Suppose that a gateway wants to receive a multicast stream.
If it is able to support GRE encapsulation, it sets bit 8 of the AMT  
Request  to "1" and transmits it using UDP.
If the relay desires and can support GRE, it keeps bit 8 set, if not,  
it sets it to zero
in the AMT membership Query, also sent using UDP. (Note that these  
control messages are not encapsulated, and
so MUST be sent UDP.)

When the gateway receives the AMT membership Query, it keeps the bit  
unchanged in the AMT Membership update.
The appropriate encapsulation MUST be used in the return AMT  
membership update.

So, at the end of the 3 way  handshake, both parties know what  
encapsulation is going to be used, and that it is supported on both  
ends, and that it is getting through.

If more than one encapsulation type is desired, then the gateway  
should set all the appropriate bits to
1, signifying support, and the relay can select which one it wants to  
use.

If a relay wants to receive a multicast stream from a gateway, it  
sets bit 8 of the AMT Request  to "1" and transmits it (in UDP) only  
if it can support and wants to receive GRE encapsulation. The gateway  
then should keep the bit set to one and use GRE if it supports GRE,  
else set it to zero and use UDP, in the AMT Membership Query (also  
sent in UDP). As before, this bit will be echoed, and the appropriate  
encapsulation MUST be used in the return AMT membership update.

[If more than one encapsulation is supported, then the
relay gets to chose one, and the stream will use either that one (if  
the gateway supports it) or UDP (otherwise).]

Note that relays are picked using anycast, but after that  
communication is done using regular IP addresses,
so it would be quite possible to have one stream using UDP, another  
(say, to a different gateway), GRE, in this scheme.

Advantages of this scheme : Existing implementations will still work  
without modification.
It should always work (no encapsulation mis-match failures) in the  
absence of firewall issues.

Disadvantages : UDP MUST be supported by both relays and gateways. If  
for some reason UDP on the AMT ports is blocked, the protocol will  
not work, even if GRE encapsulation would work.

NAT traversal. If the originator of the AMT Request message does not  
receive the appropriate Membership Queries in an appropriate time, it  
SHOULD try again with a different encapsulation, if possible.

Optional signaling of capabilities : The AMT Relay Discovery and  
Relay Advertisement messages MAY use the above schemes to signal the  
support of encapsulation protocols. Use of this information is  
implementation specific.

I think that this scheme will do what the implementors want, but I  
will leave it up to the authors and the
group whether or not it is worth while and worthy of implementation.

Regards
Marshall

_______________________________________________________________
user interface: http://darkwing.uoregon.edu/~llynch/mboned.html
web archive:  http://darkwing.uoregon.edu/~llynch/mboned/

---

From: "Greg Shepherd" <[email-masked]>
Date: Wed, 5 Apr 2006 10:32:22 -0700

On 4/5/06, Pekka Savola <[email-masked]> wrote:
> On Wed, 5 Apr 2006, Marshall Eubanks wrote:
> > If it is able to support GRE encapsulation, it sets bit 8 of the AMT Request
> > to "1" and transmits it using UDP.
> > If the relay desires and can support GRE, it keeps bit 8 set, if not, it sets
> > it to zero
> > in the AMT membership Query, also sent using UDP. (Note that these control
> > messages are not encapsulated, and
> > so MUST be sent UDP.)
> ...
> > So, at the end of the 3 way  handshake, both parties know what encapsulation
> > is going to be used, and that it is supported on both ends, and that it is
> > getting through.
>
> Actually, we don't know whether the encapsulation type signalled would
> "get through", as all the signalling is done using UDP.  A firewall or
> NAT in the middle could very well drop GRE, right?

Only the data needs to be encaped in the negotiated type. The control
messages can all use UDP.

Greg

> I'd be hesitant to do this because I view this as a brittle solution,
> and would actually require that implementations would have to support
> more encapsulations -- and it seems this approach is only useful if
> all the routers and hosts support both GRE and UDP as described below.
>
> Remember, the argument was that there is no H/W implementation of UDP
> encapsulation on routers -- this doesn't change the problem for
> routers which have clients that can't do GRE.  If S/W path using UDP
> is a MUST, either the CPU would be swamped w/ UDP or the
> implementation wouldn't support that MUST and we're back in square
> one.  The relays would need to be able to be GRE-only (or clients
> should be able to support both GRE and UDP, which isn't possible) in
> order to address that requirement.
>
> Based on this analysis, unless I'm missing something obvious, it seems
> clear we'd need:
>
> 1) a separate anycast prefix for GRE, and
> 2) do the signaling over GRE as well so that we'd know whether GRE
> actually works
>
> Thus I'd argue leaving GRE out for now.
>
> --
> Pekka Savola                 "You each name yourselves king, yet the
> Netcore Oy                    kingdom bleeds."
> Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Pekka Savola <[email-masked]>
Date: Wed, 5 Apr 2006 20:42:47 +0300 (EEST)

> messages can all use UDP.

Yes, and that's exactly (one of) the problem(s): because the control 
messages are UDP, you can be sure that UDP works, but even if you 
negotiate the use of GRE, there's no way to tell whether it will 
actually work or not.

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 5 Apr 2006 13:47:55 -0400

On Apr 5, 2006, at 1:42 PM, Pekka Savola wrote:

> Yes, and that's exactly (one of) the problem(s): because the  
> control messages are UDP, you can be sure that UDP works, but even  
> if you negotiate the use of GRE, there's no way to tell whether it  
> will actually work or not.
>

In advance. You would know soon after you set the tunnel up.

And, if I was doing the implementation, if I learned that GRE was  
desired during AMT router discovery, I would set up a probe
or heartbeat so that I knew if was OK to use before a tunnel was set up.

Regards
Marshall

---

From: Pekka Savola <[email-masked]>
Date: Wed, 5 Apr 2006 20:55:41 +0300 (EEST)

> In advance. You would know soon after you set the tunnel up.

Rather, you'd never receive any data and no data you send would ever 
be received by anyone.   While that's rather common multicast problem 
today, I don't think it's too fun to debug.  Multicast should be 
getting more reliable and predictable, not less :-)

> And, if I was doing the implementation, if I learned that GRE was desired 
> during AMT router discovery, I would set up a probe

Which kind of probe or heartbeat?  GRE implementations are not 
required to support these, if you refer to GRE keepalives.  Not sure 
what other kind of probe you might have in mind.

---

From: Tom Pusateri <[email-masked]>
Date: Wed, 05 Apr 2006 14:05:51 -0400

>Thus I'd argue leaving GRE out for now.

In the past, we've always made these types of decisions based on whether
we would affect hosts or not and we were much more willing to require
changes in routers than in hosts.

AMT has tried to abide by this and assume only the most basic support in
the host.

I suspect that no hosts are going to set up GRE tunnels to relays but more
likely leaf routers will set up GRE tunnels to core routers to provide an
AMT gateway for an entire LAN.

I wonder if routers have to support UDP for a large number of hosts,
then what is the point of adding GRE to the protocol because without
UDP, it's pretty much a non-starter.

Is there anyone besides Dino asking for GRE?

Thanks,
Tom

---

From: Pekka Savola <[email-masked]>
Date: Wed, 5 Apr 2006 20:20:31 +0300 (EEST)

...

---

From: John Zwiebel <[email-masked]>
Date: Wed, 5 Apr 2006 11:24:26 -0700

Although WRT the firewall problem UDP is, of course, the right choice...
Some implementations can much more easily support GRE over UDP.

So now you have to make a choice of what kind of systems do you need
to deploy to support UDP -easily-.

The discussion shouldn't be GRE vs UDP, without considering system  
support,
the answer is obviously UDP.

It would appear that GRE capable systems can be more easily and rapidly
deployed throughout the internet.  If so, then it would seem that
the answer becomes "obviously" GRE.

Throw in the idea of performance concerns (which is why "some" people
are pushing GRE in the first place) and perhaps the answer is
even more obviously GRE.

If "someone" can provide thousands of -high-performance- UDP capable  
systems,
and guarantee their rapid deployment, then I'd be willing to 'vote'  
for UDP.
Otherwise, the answer (for me) would have to be GRE.

So let us stop asking the "what" question, but rather concentrate on  
"how"
and "who".  Otherwise, we'll continue down a rather unfruitful path.

WRT the 'sourcing' question, as long as its a MAY then we should be
able to move forward.  I'm not convinced that a source that is connected
to a non-native multicast ISP is worth my time.  There -may- be some
"neat" applications that can be tested under those conditions, but one
needs to remember the goal of AMT is to push -native- multicast  
everywhere.
Not install another kludge that we have to debug for the next 10 years
(like MSDP).

On Apr 5, 2006, at 10:55 AM, Pekka Savola wrote:

> Rather, you'd never receive any data and no data you send would  
> ever be received by anyone.   While that's rather common multicast  
> problem today, I don't think it's too fun to debug.  Multicast  
> should be getting more reliable and predictable, not less :-)
>
> required to support these, if you refer to GRE keepalives.  Not  
> sure what other kind of probe you might have in mind.
>
> --

---

From: Tom Pusateri <[email-masked]>
Date: Wed, 05 Apr 2006 14:43:54 -0400

>the answer becomes "obviously" GRE.

But does the existing GRE infrastructure scale appropriately?
How many GRE tunnels can it support? Each becomes an interface which
has much more overhead than is needed.

There is also policy to consider. Many ISPs complained about allowing
GRE tunnels to their routers. They felt more open to abuse than a
UDP port.

>needs to remember the goal of AMT is to push -native- multicast  

If AMT is deployed on a large scale, it will never go away.

Tom

---

From: "Greg Shepherd" <[email-masked]>
Date: Wed, 5 Apr 2006 13:34:11 -0700

> But does the existing GRE infrastructure scale appropriately?

Yes - just like hardware that supports UDP encap.

> How many GRE tunnels can it support?

Enough to fill the egress interfaces.

> Each becomes an interface which

Again, no different than if this were UDP encap.

The question is not how do either of them scale, the question is how
fast can we deploy AMT with the existing routed infrastructure today.

> If AMT is deployed on a large scale, it will never go away.

Agreed.

Greg

---

From: John Zwiebel <[email-masked]>
Date: Wed, 5 Apr 2006 13:40:09 -0700

>> If AMT is deployed on a large scale, it will never go away.

<sigh>
One can at least hope.  :-)
OTOH, such AMT deployments should continue to lead to native
multicast connections when the receivers Bandwidth demand
gets large enough.
_______________________________________________________________
user interface: http://darkwing.uoregon.edu/~llynch/mboned.html
web archive:  http://darkwing.uoregon.edu/~llynch/mboned/

---

From: "Greg Shepherd" <[email-masked]>
Date: Wed, 5 Apr 2006 13:51:58 -0700

> multicast connections when the receivers Bandwidth demand

But there may still be last-mile layer2 issues which AMT will address.
I want to whiteboard/brainstorm with you and Dino about if/how AMT
could be used to address current SSM-nonawareness in L2 devices for
Eth, DSL, Cable, etc..

Greg

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 5 Apr 2006 17:01:19 -0400

Dear Greg;

> could be used to address current SSM-nonawareness in L2 devices for

It seems to me that any time you are faced with a NBMA network, AMT
might be useful.

<putting on co-chair hat>

That reminds me - there was strong desire last time for more time
at the meeting. Should we just ask for more time, or should we try  
and organize a BOF, and, on what ?

I would lean towards a BOF.

Regards
Marshall

---

From: "Greg Shepherd" <[email-masked]>
Date: Wed, 5 Apr 2006 14:09:08 -0700

> I would lean towards a BOF.

I believe Joel mentioned the potential usefullness of mcast-ucast
translation for non-ssm L2 networks at the MLM BOF in Vienna. - just a
data-point.

Greg

---

From: Leonard Giuliano <[email-masked]>
Date: Wed, 5 Apr 2006 14:47:21 -0700 (PDT)

-) > > > > 
-) > > > > If AMT is deployed on a large scale, it will never go away.
-) > > > 
-) > > > Agreed.
-) > > > 
-) > > 
-) > > <sigh>
-) > > One can at least hope.  :-)
-) > > OTOH, such AMT deployments should continue to lead to native
-) > > multicast connections when the receivers Bandwidth demand
-) > > gets large enough.
-) > 
-) > But there may still be last-mile layer2 issues which AMT will address.
-) > I want to whiteboard/brainstorm with you and Dino about if/how AMT
-) > could be used to address current SSM-nonawareness in L2 devices for
-) > Eth, DSL, Cable, etc..
-) > 
-) 
-) It seems to me that any time you are faced with a NBMA network, AMT
-) might be useful.
-) 

There will always be a need for AMT, whether it be in the last mile, the
first mile or one of the miles in the middle.  Due to the all or nothing  
nature of mcast, you basically need PIM (or IGMP/proxy) running on every  
L3 device on the Internet.  A single, global, contiguous instance of PIM  
that encompasses every device on the Internet is a bar that is set
incredibly high.  No other protocol, not OSPF, ISIS, LDP, RSVP even BGP is
expected to run contiguously on every L3 device.  So, not to sound all
Princess Leah here, but AMT is our only hope.  There will always be a need
to hop over certain segments of the network to circumvent the continuity  
requirement of mcast, and those segments could be anywhere.

---

From: John Zwiebel <[email-masked]>
Date: Wed, 5 Apr 2006 15:00:25 -0700

On Apr 5, 2006, at 2:47 PM, Leonard Giuliano wrote:

> So, not to sound all

Not to be all Han Solo here, but permitting deployment of
a non-contiguous mcast environment is detrimental to the
fight against the empire.

So "always" is too long for me.
Better stop now, the metaphors are coming in fast and furious :-)
_______________________________________________________________
user interface: http://darkwing.uoregon.edu/~llynch/mboned.html
web archive:  http://darkwing.uoregon.edu/~llynch/mboned/

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 5 Apr 2006 21:15:35 -0700

> fight against the empire.

To stay with the language: There is no single global instance of
PIM-SSM that Leonard is talking aboutr, even if you deploy it hop-by-hop
end-to-end.

The only coordination you need between each of these routers
is as much coordination as you need to forward hop-by-hop an IP unicast
packet. So PIM-SSM forwards PIM (S,G) joins hop by hop. Same thing. Same
level of coordination (or not)(. In that matter, each router is a PIM-SSM
resistance cell ;-)

IGPs like ISIS and OSPF they of course are the empire with a bureaucracy
(like PIM-SM is too), because they need tight coordination amongst themselves
(disjstra etc. on the IGP side, RP/DR coordination on PIM-SM.

> Better stop now, the metaphors are coming in fast and furious :-)

Ask me. Now i have to stand some beating from at least Greg for
using the undefined term PIM-SSM ;-)

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 5 Apr 2006 21:37:33 -0700

> has much more overhead than is needed.

We've got mGRE tunnels, which isn't really multicast, but multipoint,
eg: on a headend router you've got one tunnel interface and the
20,000 tailends are just adjacencies. But this is all just customer 
name recognition with once less scalable implemented technology.
"IP does not scale", "RSVP does not scale", "Foo does not scale" *doh*.

> GRE tunnels to their routers. They felt more open to abuse than a UDP port.

Customer feelings have to be answered with good vibrations,
customer technical concerns with serious technical discussions.

If you remove the layers of fud, then AMT will share all the same
problems that other already deployed hub&spoke technologies for multicast
have, and in the commercial space we do have pretty large deployments
of those (thousands of tunnels on hub routers). I think we understand
and have solved most of those on the vendor side, but there's a lot more
to it than just "set up a tunnel" - whatever it's called, GRE, AMT,
IPsec, MobileIP, L2TPv3,...

From my perspective, the things AMT changes are:

   a) the good vibrations associated with it
   b) potential of using UDP, but that's just a NAT benefit 
      but a router-support issue (GRE is just more often found
      to be supported in HW).
   c) A new lightweight session setup mechanism. Whether that one
      will actualy satisfy service providers security requirements
      is to be seen. I would not be surprised if potential headend
      operators come back and say "can we please have ppp or a comparable
      authentication mechanism before we give out a tunnel".
   d) The hope for end hosts to implement simply usable endpoints

So, b) and c) are kinda so and so - we will see. So the real kick for
AMT is a) with hope for d). For everything else, we will play a big Deja-Vu
to other hub&spoke tunneling technologies used with multicast. But that's
fine. AMT isn't the first technology trying to be successful by changing
small pieces of an otherwise known and unchanged equation and betting on a).

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 5 Apr 2006 21:44:19 -0700

> AMT has tried to abide by this and assume only the most basic support in

Well. But AMT is based on the premise of host-side work, and i think
it's a small but important gap between host and router preferences
that should just be supported on either side for the case of AMT,
because it's successfull adoption can depend on hub side availability as
much as host side availability.

> UDP, it's pretty much a non-starter.

Not today, no. UDP is starting to make inroads slowly via some other
layered encaps, but it's still pretty uncommon. One of the primary
reasons of course is the checksum.

> Is there anyone besides Dino asking for GRE?

Me.

---

From: "Greg Shepherd" <[email-masked]>
Date: Thu, 6 Apr 2006 04:41:19 -0700

> using the undefined term PIM-SSM ;-)

PIM-SSM? What's that? Do you mean PIM-SM?

Actually when I first started reading your message I was getting
agitated, but when I got to you last line here I burst out laughing.
Thanks for the morning laugh.

..but that reminds me.

Greg

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 6 Apr 2006 22:55:37 -0700

>> could be used to address current SSM-nonawareness in L2 devices for

    Good point Shep.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 6 Apr 2006 22:58:13 -0700

>> expected to run contiguously on every L3 device.  So, not to sound all

    Actually I think it is a feature that you can run one protocol on all 
    routers then to have to interwork different protocols at different points 
    in the network.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 6 Apr 2006 23:25:59 -0700

>> Is there anyone besides Dino asking for GRE?

    Okay, I've gone through all emails in this thread. Let me speak honestly
    and openly with my multicast-advocate hat on and not my vendor hat.

    o GRE encapsulation is used for data packets only. That is, from relay to
      gateway and gateway to relay.

    o GRE encapsulation allows routers today to become AMT relays with a
      pretty good degree of scalability. Scalability can be enhanced further
      if multiple tunnels are implemented as an NBMA network. But replication
      performance is fairly optimal up front via hardware acceleration.

    o GRE encapsulation by AMT relay routers doesn't necessarily require GRE   
      support in hosts. If an AMT gateway implementation using GRE 
      encapsulation is built in a Linksys box, we can preserve NAT 
      functionality by looking deeper into the packet to allow a GRE codepoint
      for AMT through the firewall. 

    o I would argue that doing AMT inside of UDP inside of GRE would work.
      And a host implementation could do this easily. (i.e. Windows could 
      implement the AMT pseudo-interface over a GRE adaptor interface). In
      the case where the host GRE encapsulates, the NAT-box can look deeper
      here as well.

    o We can do UDP encapsulation fast (in hardware) in the long-term and I 
      envision we (I will) can influence this in many platforms.

    We really want some scalability up front or else this project may not
    get off the ground as real versus a technology-demo. Plus, we would
    have more justification for vendor resource allocation for putting UDP 
    encapsulation in ASICs showing there is real deployment and a business 
    case developing.

    I envision the phases of this project would go something like this:

    First phase: Host resident gateway doing UDP data encapsulation with a 
                 software based replicating relay.

    Second phase: NAT-box resident gateway doing UDP encapsulation with a
                  software based replicating relay.

    Third phase: Wider deployment of second phase which requires the need for
                 GRE encapsulation. First phase scenarios can occur on a
                 small scale basis.

    Final phase: All combinations above with UDP encapsulation in hardware
                 will appear.

    I suggest we do the negotiation in the most trivial way. And Marshall, I
    think we may not have to worry about preserving existing implementations.
    I believe this project is in an early enough phase where it costs nearly
    nothing to change the implementations. Plus, they have to change anyways
    for the next draft Tom puts out (based on Dave et al experience).

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 6 Apr 2006 23:33:15 -0700

>> PIM-SSM that Leonard is talking aboutr, even if you deploy it hop-by-hop

    But Lenny still has a point in a PIM-SM/MSDP environment. His point is
    that every router between a source (and RPs) and all receivers has PIM
    running.

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 6 Apr 2006 23:53:42 -0700

>                  software based replicating relay.

Where the question is of course whether we use UDP because the
host stack has problems to do GRE or because the NAT'ing home gateway
won't let GRE through.

>                   software based replicating relay.

As soon as we are at this phase, GRE is better. Not only is the NAT
box very much a router and thus GRE is easier, but the AMT tunnel
would also logically be outside of NAT and/or because NAT and the AMT
tunnel are collocated in the same box/code it's always possible to
avoid problems between those two.

>                  GRE encapsulation. First phase scenarios can occur on a

Are you saying that the first phase CAN NOT get wider deployment because
of the performance issues with UDP on headends ? Or because you think
it will be less popular to run the AMT tunnel in applications/host-stack
on the host ?

In the past it has always been easier for end users to get such early
adopter solutions to work on host in applications. And given how many
home-gateway vendors there are, i would bet that you won't get AMT
adopted on manyt of them. Of course, home-gateways are cheap enough
nowadays that you might be able to simply buy the one where other
community members have expanded a public-domain Linux implementation.
But there are many customers who use SP provided/recommended home
gateways only.

>     Final phase: All combinations above with UDP encapsulation in hardware

Hey wait a second. Where did native IP multicast go here ? ;-))

But yes, agreed. Unless either AMT dies due to native multicast ;-)
or because it's successfull but widely adopted in home gateway.

(like Dynamic DNS support, which also started on hosts and managed
 to make the top lists of all home gateway vendors).

Overall, IMHO UDP HW support is still the least preferred option under
wide scale deployment:

    Native Multicast
    Home Gateway AMT (GRE)
    Host AMT (UDP)

>     for the next draft Tom puts out (based on Dave et al experience).

So... Why again do we not simply propose to use a very simple
profile of L2TP (v2/v3) ? Got a negotiation phase and is also passed through
probably all home gateways due to it's use in some VPN tunnels.

Not ?

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 7 Apr 2006 00:06:06 -0700

>     that every router between a source (and RPs) and all receivers has PIM

I thought Lenny was arguing PIM-SSM.

If Lenny was arguing PIM-SM/MSDP:
There's no single instance either. There's an MSDP infrastructure
of MSDP like BGP (single instance BGP for the whole Internet ?),
and there are individual PIM-SM Domains/Instances in every AS
(eg: between interdomain MSDP speakers).

But seriously, PIM-SM/MSDP is not a technology for the Internet at
large. Consider 50 million residential users have multicast and can source.
You really thing you can run a single ASM session without being DOS'ed ?
PIM-SM/MSDP is for subsets of the Internet with higher mutual trust,
like "the good old Internet" (research/edu and progressive enterprises).

In the rest of the Internet, multicast can only survive and grow with SSM.
And in SSM there's no coordination between routers beyond the coordination
you get when you configure static routes along a path - you wouldn't
call either a "single instance".

Cheers
    Toerless

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 7 Apr 2006 00:02:49 -0700

>> it will be less popular to run the AMT tunnel in applications/host-stack

    The later.

>> Hey wait a second. Where did native IP multicast go here ? ;-))

    It's there but just with 2 extra headers in front of it.

>> or because it's successfull but widely adopted in home gateway.

    But IP multicast will run natively on the AMT site.

>> Overall, IMHO UDP HW support is still the least preferred option under

    Maybe from a router vendor point of view. But Toerless, you have to 
    consider the host vendor's point of view as well.

    We have to agree to interoperate. So that is what we are trying to do.

>> probably all home gateways due to it's use in some VPN tunnels.

    Because the L2TP header is too large. Because L2TP is overkill IMO.
    Because Marshall's suggestion was simpler.

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 7 Apr 2006 00:29:00 -0700

> >> it will be less popular to run the AMT tunnel in applications/host-stack

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

>     consider the host vendor's point of view as well.

But above you said that it will be less popular to run ATM tunnel
in application/host-stacks.

The most important reason for UDP is to terminate the AMT tunnel
not in the host stack, but rather directly inside of an application,
eg: For the typical multicast case: Nobody gives you the multicast
support you need: Not the ISP, not that the host vendor. Let's do all
in the application! 

What view should a host vendor have ? A host vendor can implement
GRE as easily as UDP. Only applications can't!!! If a host vendor
is interested to score points with his users, then his view should
be to implement UDP to get across broken home gateways and GRE to
support the widest range of headends with performance.

>     We have to agree to interoperate. So that is what we are trying to do.

Sure. Supporting two encaps makes it more difficult, but we
seem to agree that it's important initially. We disagree where
the solution will turn to long-term, but not even in a contradictory
way: You say router vendors could long term implement UDP encap in HW.
No disagreement. I just say that if AMT becomes successfull it will
also be easy to have it converge on GRE. Only when AMT endpoints in
applications will be the long-term predominant solution will we
need to use UDP in HW, and then i don't think AMT is successfull
(no host-vendor-stack, no home-gateway picking it up !).

>     Because Marshall's suggestion was simpler.

Sure. But if solutions that are widely accepted in unicast are
considered to be too difficult to make a multicast solution successfull,
does that not rather indicate insufficient demand for the multicast
piece rather than too much complexity ?  

Cheers
    Toerless

---

From: Pekka Savola <[email-masked]>
Date: Fri, 7 Apr 2006 10:35:25 +0300 (EEST)

Even though responding to Dino's mail, this is directed mainly 
to Toerless,

On Fri, 7 Apr 2006, Dino Farinacci wrote:
...
>    We have to agree to interoperate. So that is what we are trying to do.

Obviously, but as AMT support at Home Gateways is not going to be 
ubiquitous in the next 10 years or so, the hosts will always need to 
be able to do it on their own: independence of the network ensures 
that the service will always work (as long as the interoperability 
with relays work).

That is the model that the host implementors will have to take.. 
Host tunneling w/ UDP is always supported, other features (such as a 
home gateway already supporting AMT) are bonus.

>    Because Marshall's suggestion was simpler.

One of the most important features of AMT is that it requires no state 
or config at the relay for idle clients.  (E.g., even a 500$ PC box 
can serve 10,000,000 AMT users as long as only 100 of them are 
transmitting or receiving at the same time).

AFAICS, using L2TP would eliminate that feature, which is a 
non-starter.

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 7 Apr 2006 01:19:55 -0700

> that the service will always work (as long as the interoperability 

Again, be more specific when you say host implementation. Host OS
stack implementation or Application side implementation ?

> AFAICS, using L2TP would eliminate that feature, which is a 

Not really. You can easily have BRAS configs where only consume
ressources for active subscribers. It's just not common anymore to
provision BRAS for such oversubscription (claim to support 10,000,000
users but allow only 100 in at any time). That was more common during
dialup times than during todays flat-rate-dsl times. 

Seriously: The only reason against all the other encaps is only the
application-side-implementation and really bad home-gateway NAT.
I am just irritated that the discussion forgets this very limited
scope of reasoning and claims more wider scope reasons for the encap
and signaling choices.

Cheers
    Toerless

---

From: Leonard Giuliano <[email-masked]>
Date: Fri, 7 Apr 2006 09:51:00 -0700 (PDT)

On Fri, 7 Apr 2006, Toerless Eckert wrote:

-) On Thu, Apr 06, 2006 at 11:33:15PM -0700, Dino Farinacci wrote:
-) > >> To stay with the language: There is no single global instance of
-) > >> PIM-SSM that Leonard is talking aboutr, even if you deploy it hop-by-hop
-) > >> end-to-end.
-) > 
-) >     But Lenny still has a point in a PIM-SM/MSDP environment. His point is
-) >     that every router between a source (and RPs) and all receivers has PIM
-) >     running.
-) 
-) I thought Lenny was arguing PIM-SSM.
-) 

What I meant was that even in the simplest case of SSM, you still need 
PIM-SSM (sic) turned on every interface of every router and FW from every 
source to every receiver on the Internet.  Now we all think this is no big 
deal, but there is fairly substantial empirical evidence that says there 
are a few operators out there who don't quite see things the same way we 
do.

I wasn't trying to stir up an academic debate on the 
similiarities/differences of various protocols or the meaning of a single 
instantiation.  I simply wanted to point out that there will always exist 
pragmatic realities (technical, political, religious, ignorance, laziness, 
legacy equip, etc) which will prevent PIM (or IGMP/snooping) from being 
enabled on every single router and fw interface on the Internet.  Hence 
the need for AMT can exist at any point on the network.

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 7 Apr 2006 10:02:27 -0700

> are a few operators out there who don't quite see things the same way we 

Sure. but that still does not make it a single instance and it still does
not make it difficult. We also have a long tradition of operators resisting
change and later on overcoming it. You can't always build technology to
please operators, and even you do, it may not result in good technology.
You build technology to best solve problems, and in most cases technology
is adopted because there is a business case anyhow, and not for customers
self gratification.

MSDP by the way is a good example of a protocol designed to please SPs.
Looks like BGP, quacks like BGP and can do multicast. SPs loved to deploy
it because of that. Does that make it a good or successful protocol ?

> instantiation.

Why not ? It's a fun discussion. And i wanted to jump to it, because
your choice of words is exactly how i would expect a customer who does
not understand multicast to argue. ANd i wanted to try to difute that
call ;-))

> I simply wanted to point out that there will always exist 
> legacy equip, etc) 

agreed to all of those except technical. 

> which will prevent PIM (or IGMP/snooping) from being 
> enabled on every single router and fw interface on the Internet.
> Hence the need for AMT can exist at any point on the network.

Sure. But AMT should still be the unselfish protocol that would
happily give itself up whenever it promotes native multicast ;-))

-- 
Thanks
   Toerless Eckert

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 7 Apr 2006 09:27:08 -0700

>> in application/host-stacks.

    *I* did not say that.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 7 Apr 2006 09:26:12 -0700

>> ubiquitous in the next 10 years or so, the hosts will always need to 

    I beg to differ.  ;-)

Dino

---

From: Leonard Giuliano <[email-masked]>
Date: Mon, 10 Apr 2006 09:42:20 -0700 (PDT)

-) > 
-) > What I meant was that even in the simplest case of SSM, you still need 
-) > PIM-SSM (sic) turned on every interface of every router and FW from every 
-) > source to every receiver on the Internet.  Now we all think this is no big 
-) > deal, but there is fairly substantial empirical evidence that says there 
-) > are a few operators out there who don't quite see things the same way we 
-) > do.
-) 
-) Sure. but that still does not make it a single instance and it still does
-) not make it difficult. We also have a long tradition of operators resisting
-) change and later on overcoming it. You can't always build technology to
-) please operators, and even you do, it may not result in good technology.
-) You build technology to best solve problems, and in most cases technology
-) is adopted because there is a business case anyhow, and not for customers
-) self gratification.
-) 
-) MSDP by the way is a good example of a protocol designed to please SPs.
-) Looks like BGP, quacks like BGP and can do multicast. SPs loved to deploy
-) it because of that. Does that make it a good or successful protocol ?
-) 
-) > I wasn't trying to stir up an academic debate on the 
-) > similiarities/differences of various protocols or the meaning of a single 
-) > instantiation.
-) 
-) Why not ? It's a fun discussion. And i wanted to jump to it, because
-) your choice of words is exactly how i would expect a customer who does
-) not understand multicast to argue. ANd i wanted to try to difute that
-) call ;-))
-) 

Hmm, I don't find debating the meaning of the word "instance" to be quite 
as fun, especially when it has nothing to do with my original point.  If 
you prefer another word, feel free to substitute it, as I don't really 
feel that strongly about it.

However, I do find it more fun to discuss AMT deployment.  I don't mean to 
just preach to the choir, but your choice of words is exactly how i would 
expect a customer who does not understand the need for AMT to argue.  So 
I'd like to take the opportunity to refute that argument:

Much has been done over the years to make mcast simpler, better 
documented, better understood and easier to deploy.  This WG in 
particular, has done a great job recently documenting mcast deployment 
issues (MSDP deployment draft, some of Pekka's drafts, etc).  There are 
enough drafts, whitepapers, cookbooks, and books out there that no one 
could reasonably claim there's not enough info on the subject.  
Unfortunately, the reasons for operators to deploy remain abstract and 
esoteric.  Now, it is quite obvious and clear to us that mcast can 
eventually enable new content that will make the Internet ultimately more 
useful and profitable.  However, that argument won't drive ALL operators.  
Maybe a FEW, perhaps even SOME, but certainly not ALL.  And for mcast to 
be really useful, we need ALL (or nearly all) to deploy it.

Operators, on the whole, do not deploy technologies because they are cool, 
or that it may someday be useful if everyone does it.  I don't believe 
that better documentation or awareness of mcast, or even if we locked them 
all in a room and showed them the shiniest .ppt slides we could come up 
with is going to change that.  Rather, operators will deploy something (1) 
as a reaction to a threat/opportunity or (2) bc they think it will lead 
directly to more more revenue.  At this point, Internet mcast fits neither 
category.  Complexity, I now believe, is really just a minimal issue, if 
not an excuse; operators will happily deploy the most complicated stuff 
anyone can think of if it fits one of those categories (see MVPNs), and 
will resist the simplest of changes if it doesn't (see SSM).  Thus, I 
believe that anything that requires even just a single line of router 
config on all routers on the Internet will be too high of a barrier if one 
of the 2 categories is not satisfied.

If you look at things that have seen explosive growth in recent years such 
as P2P, Vonage-like VOIP, Skype-like VOIP, you'll see that it grew in 
spite of operator involvement, not bc of it.  None of these required any 
involvement or coordination from operators other than simple unicast 
delivery of packets.  Operators were left reacting, alternately wondering 
how to stop these services or how to deploy something to make money off 
these services.  In any event, none of these services assumed that all 
operators in the world would do anthing to help them, and for that 
assumption they have thrived.  (And for those out there who say these 
services aren't profitable for operators, I would strenuously disagree; 
anything that brings new content and creates tremendous need and demand 
for Internet services is ultimately beneficial to service providers).

I believe we need to take the same approach.  To recognize that some 
forward-thinking operators out there exist and will deploy mcast natively, 
and may even be willing to stick boxes out there that'll relay mcast data 
to the unicast world (AMT).  But many/most won't do anything until it 
becomes a threat/opportunity on the radarscreen and forces them to react.  
For this reason, I believe AMT (or some type of unicast-mcast relaying 
mechanism) is something that should be at the top of our priorities.  
Simply waiting for everyone to deploy mcast natively will leave us in the 
same position a decade from now.  Nothing will change without some sort of 
disruptive, enabling mechanism like AMT.

-Lenny

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 10 Apr 2006 11:45:40 -0700

On Mon, Apr 10, 2006 at 09:42:20AM -0700, Leonard Giuliano wrote:
> expect a customer who does not understand the need for AMT to argue.
> So I'd like to take the opportunity to refute that argument:

Not sure which argument of mine you intend to refute, or what i 
supposedly don't understand. I would have happily supported everything
you said if you would have started with this last mail instead of delving
into the semantics of "single instance" and complexity. Something in the
order of what you write below should be in the AMT draft. After all,
this is still MBoned, and relating stuff to reality is - unlike many other
WGs - still part of the charter (i hope ;-), so having just what looks like
a protocol spec should in my opinion not be good enough for MBoned.

The thing i am still missing is also a writeup of why existing
mechanisms like L2TP/IPsec/mobileIP tunnels could not have been used,
given that there are lots of client side and headend implementations.
If there's at least proof that people investigated to use existing
implementations of those with multicast and failed, then this is not
only good motivation for AMT, but also something to kick folks that
implement those tunnelings (and then obviously don't well enough support
multicast).

Cheers
   Toerless

---

From: Pekka Savola <[email-masked]>
Date: Tue, 11 Apr 2006 08:12:45 +0300 (EEST)

Tailing down the Cc: list a bit..

> implement those tunnelings (and then obviously don't well enough support

Let me quote from what I wrote on Friday, and has been stated many 
times in the past (I recall you asking the similar "why not just do an 
L2TP or GRE tunnel server?" question at a MBONED WG meeting over a 
year ago):

"One of the most important features of AMT is that it requires no 
state or config at the relay for idle clients.  (E.g., even a 500$ PC 
box can serve 10,000,000 AMT users as long as only 100 of them are 
transmitting or receiving at the same time).

non-starter."

That said, I think these tunneling solutions solve two different 
problems.  If you want to do high B/W, more controlled tunneling 
between routers (e.g., from content provider to the service 
provider(s) networks), use manually configured GRE.  That makes 
perfect sense.  But when the distribution to the hosts needs to be 
automated, you'll need something that is a) automatic, and b) 
non-stateful (because it's a service that should be something 3rd 
parties must be able to offer and ISPs should be able to start slowly 
without large investment in tunnel headend boxes).

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 10 Apr 2006 22:45:22 -0700

Inline

> AFAICS, using L2TP would eliminate that feature, which is a 

Pekka, you are repeating yourself. Did you read
<[email-masked]> that i sent in response to your
<[email-masked]> ?

> without large investment in tunnel headend boxes).

<[email-masked]>. also check out how BRAS maintain
subscriber access interfaces.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 10 Apr 2006 23:07:06 -0700

>> transmitting or receiving at the same time).

    This claim is unfounded. So it doesn't matter if it is quoted. There is
    a CPU cost to replication in a PC, period. So this quote is non-sense.

Dino

---

From: Pekka Savola <[email-masked]>
Date: Tue, 11 Apr 2006 09:32:10 +0300 (EEST)

>    a CPU cost to replication in a PC, period. So this quote is non-sense.

I'm not sure which part you're objecting to.. You seem to be saying, 
"data forwarding on a PC isn't effective".  I don't dispute that, but 
I argue that data forwarding may not be an issue if you don't want to 
provision the system for more than (say) 1 Gbit/s of forwarded data at 
any point in time.  What I'm saying is, "you must be able to support 
hundreds of thousands or millions of users if they don't send or 
receive data", i.e., I'm worried what is the signalling overhead of 
more stateful alternatives.

FWIW, http://staff.csc.fi/psavola/6to4.pdf (published in CCR special 
issue in January 2005) shows that a PC can very well support millions 
of distinct [6to4] users in a month as long as most of them sit idle 
"waiting for IPv6 to be used", because the relay implementation is 
stateless.  It could support many, many more as well, as long as the 
total bandwidth required by the traffic stays at a reasonable level.

Would the same be possible with stateful tunneling mechanisms?

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 10 Apr 2006 23:45:24 -0700

>> total bandwidth required by the traffic stays at a reasonable level.

    Sure, on this time scale. I do worry about large-fanout for popular
    content. And the relay, in the multicast case, is not stateless, you
    still have to keep tunnel endpoint info per oif per (S,G) entry.

Dino