Subject: [rrg] IPv4 & IPv6 routing scaling problems

From: Robin Whittle <[email-masked]>
Date: Thu, 04 Feb 2010 13:37:44 +1100

Short version:    If the RRG recommends an IPv6-only solution,
                  this would be analogous to solving the
                  planetary warming problem only on Mars, on
                  the basis that Earth will cope for the
                  immediate future - and is so overcrowded
                  that most of humanity will need to move to
                  Mars anyway, real soon now.

                  CEE architectures are only viable on IPv6,
                  since a multihomed /16 network needs at least
                  two /16 PI prefixes - and so chews address
                  space in a manner which really can't work
                  on IPv4.

                  So if the RRG chose a CEE solution, this will
                  not help with the IPv4 scaling problem at all
                  until and unless IPv6 becomes a viable
                  alternative to IPv4 for mass adoption.

                  Current levels and growth rates in the DFZ
                  advertised prefixes for IPv4 and IPv6.

                  At current growth rates, the IPv6 number will
                  reach today's IPv4 figure of ~300k in 11 or 12
                  years.  By then, at current growth rates, the
                  IPv4 figure would be 1.46 million.

                  Discussion impact of mass-market IPv6 use for
                  cellphone and other mobile IP uses - and how
                  this would not directly drive the scaling
                  problem.  Only when large numbers of end-user
                  networks want their own IPv6 PI space would a
                  problem arise.

                  Replies to messages from Joel, Noel, Eric,
                  Brian, and Scott - all from the "SIP will work
                  fine with CES, CEE won't work with NAT" thread.

Hi Joel,

In "Re: [rrg] Multiple critiques, choice of single proposal, ..."
(msg05846) you wrote:

> The question of IPv4 support is an example of something the RG has
> not converged on.

In later messages in the "SIP will work fine with CES, CEE won't work
with NAT" thread you and others also discussed IPv4 and IPv6 scaling
problems.  I respond to those messages below.

One measure of the IPv4 scaling problem is is ~300k prefixes in the
DFZ with a doubling time of about 5 years.

  http://bgp.potaroo.net/

Another measure, which is hard to quantify, would be how many
end-user networks are not getting portability, multihoming and
inbound TE when they want / need it.

The IPv6 Internet has no scaling problem:

  http://bgp.potaroo.net/v6/as2.0/
  http://bgp.potaroo.net/v6/as6447/

These indicate about 2.6k prefixes with a doubling time of about 20
months.  At these rates it would take 11.4 years (2021) for this
measure of the IPv6 scaling problem to reach the level of today's
IPv4 scaling problem. By then, at the current growth rates, the IPv4
figure would be about 1.46 million.

I expect the rate of growth in the number of IPv4 routes in the DFZ
to to increase, since there will be increasing pressure to slice and
dice currently used address space into more, smaller, chunks once the
fresh unused expanses of space run out.  I think this will be
required to increase the number of global unicast addresses which can
be actively used.

I expect that a sparsely used prefix (whether of an ISP or a PI
prefix of an end-user network) will have its hosts, DSL services or
whatever moved to one half.  Then the other half will be advertised
as a separate prefix - perhaps by another organisation - and used
similarly intensively.

Also, I imagine that the rate of IPv6 adoption will increase.

However, the only scenario I can foresee for mass adoption of IPv6
involves large mobile phone companies using it.  This would give each
cell-phone a public IPv6 address, rather than the only other option -
putting each one behind an IPv4 NAT box.  They could still give NATed
IPv4 access via tunneling.  In a footnote "MNs with public
addresses?" below, I consider the potential problems of having a
public IPv4 address or IPv6 /64, including via a TTR Mobility.

While this could involve hundreds of millions of handsets, I guess
the number of such mobile networks would probably be small, and that
each one would only advertise one or a few prefixes.  So this scenario
wouldn't lead to a significant scaling problem.  Nor would a number
of large ISPs - such as those with millions of DSL, fibre, HFC cable
or WiMAX connected customers giving their millions of customers IPv6
space - lead to such a number of prefixes large enough to make a
significant contribution to any scaling problem.

I think we all agree that unscalable growth in the number of prefixes
advertised in the DFZ is caused by large numbers of end-user networks
advertising their PI prefixes.

I guess few end-user networks want or need IPv6 connectivity at
present.  However, if there were millions or billions of cell-phone
users with direct IPv6 connectivity, but limited (behind NAT) IPv4
connectivity, then there could be a substantial number of end-user
networks wanting their own portable, multihomable, PI space in the
IPv6 Internet so they can sell services to these millions of
end-users.  I guess this would start to drive the IPv6 scaling problem.

If an end-user network had a bunch of its staff and/or customers with
IPv6 cellphones, it would probably want its own portable,
multihomable, IPv6 space so it could communicate directly with these
people's mobile devices - so that would lead to general adoption of
IPv6 by more and more end-user networks which were not in the
business of selling things to mobile users.  Until we develop and
deploy a scalable way of providing these end-user spaces, any such
networks which can get PI space will be advertising it in the IPv6
DFZ and so further driving the future IPv6 routing scaling problem.

Joel, in (msg05925), you wrote:

> Robin Whittle wrote:
> ...
>>       IPv4 will be _central_ for a long time to come.
> ...

That was me quoting, in agreement, Patrick Frejborg, in (msg05895),
agreeing with Noel, who wrote:

    IPv4 will be around for a very long time - especially in the
    old economies

> IPv4 will clearly be around for a LONG time. IPv4 will, I expect,
> continue to be used for a LOT of content for a LONG time.

OK - we agree on this.

> However, I am not sure that matters for the RRG work.
>
> The IPv4 Internet works. The routers, and the routing system, cope
> with the current pressures. To my way of looking at things, the
> question is how will the Internet routing system cope with growth.
> But, definitionally, there really is not that much growth left in
> IPv4.

I disagree for two reasons.  I believe the number of prefixes in the
IPv4 DFZ will grow at an increasing rate - until we have a scaling
solution widely adopted.

  1 - It is possible to make a much larger proportion of IPv4
      space used for actual services, whether they be a host
      or a DSL router or whatever.   Some prefixes are pretty
      full but many others are sparsely populated.  By the
      mechanism I described above many more addresses can be
      used: consolidating a scattered bunch of used addresses
      into one end of a prefix and then advertising this end as
      and the other end as separate prefixes, where the other
      end prefix is now empty and ready to be intensively used,
      or divided into still more prefixes which themselves can
      be intensively used by a greater number of smaller
      ISP or end-user networks.

      See messages quoted below from Brian, Scott and Noel -
      who appear to agree with this argument.

  2 - The above process accelerates the growth in the number
      of prefixes advertised in the DFZ, which is the most
      tangible measure of the routing scaling problem.

> On the other hand, although IPv6 is tiny now, unless the entire
> Internet stops growing, v6 will become massive.

I suspect that with squeezing more use of the IPv4 space, and
increasing use of NAT, that another 10 or 20 years of natural growth
in usage can be accommodated in the IPv4 Internet - IF mass adoption
mobile IP gives each device an address behind NAT AND if most of
those mobile devices do not get their own IPv4 address of SPI space,
such as by TTR Mobility.

I guess there will be 5 billion or 8 billion or whatever cell-phones
(or whatever they might be called) with IP connectivity by 2020 or
2025.  They could all have IPv4 connectivity behind NAT, without port
forwarding.  They couldn't all have their own portable SPI space -
because there are only 3.7 billion global unicast IPv4 addresses.

They could all have their own global unicast PI IPv6 address or /64
prefix, which would be different for each access network they use,
and could vary within one large access network.  They could all have
their own portable /64 of SPI space via TTR Mobility.

Over a long period of time, I think something other than IPv4 will
have billions of hosts on it, and I think the most likely initial
settlers in this second mass-adoption Internet will be mobile
devices.  The most obvious second mass-adoption Internet is IPv6.

> If we do not work out an
> architecture that can cope with growth, when that pass the current
> IPv4 size, and keeps growing faster, we will be up the proverbial
> creek without any control whatsoever.

I agree, but the IPv4 scaling problem is likely to remain more
serious than the IPv6 one for a long time, such as a decade or two -
especially due to the increasing pressure to slice and dice IPv4
space into multiple smaller prefixes in order to cram as many
actively used addresses into the system as possible.

> Hence, I think it is
> actually quite reasonable to have an architecture and approach
> which only addresses IPv6 Internet Scaling. Without denying that
> IPv4 will be here, and important, for a very, very, long time.

I completely disagree.  As noted above, the IPv4 scaling problem is
bigger now - there is no IPv6 scaling problem today.  It is
reasonable to expect that the IPv4 scaling problem will remain bigger
than IPv6's for a decade or more.  IPv4 routing scaling difficulties
will grow at a greater rate *because* of the shortage of fresh
address space, since this is the only way to accommodate increased
growth in the number of utilized addresses.

I think this idea that the RRG should recommend an IPv6-only solution
is analogous to the UN setting up a 3 year taskforce on global
warming, and them coming back with a recommendation:

   We have a solution for the greenhouse effect warming problems
   on Mars.  We recommend that all resources be devoted to
   implementing it.  Earth will cope for the time-being, and is
   already way-overcrowded - so it won't be long before most of
   humanity moves to Mars.

Noel, in (msg05930) you wrote, quoting Joel:

>> definitionally, there really is not that much growth left in IPv4.
>
> Not necessarily; mind, I don't _know_ that there is going to be
> more growth, but I can see plausible circumstances where there is
> continued substantial growth in the number of routes in the DFZ.
>
> E.g. if some organizations which currently have large chunks of the
> globally-visible IPv4 namespace drop back to using smaller chunks
> of the that namespace, and the chunks they return are broken up
> into many smaller pieces for use by a number of currently
> non-connected entities.

OK - we agree entirely on this.  (However, I would have written
"address range" rather than "namespace".)

> The exact shape of the future can be awfully hard to predict. If
> you'd asked people back in 1995 if we'd be here today, people would
> probably have gawked at you... Never say 'Never'... :-)

Even without growth in the human population, it is easy to see that
the adoption of cell-phones won't stop until there's about one for
every person.  So I think we need to plan for up to 10 billion
IP-capable cellphones.  Ideally, each one would have its own global
unicast IPv4 or IPv6 address, to be used no matter what sort of
connection it has.  The TTR Mobility architecture can do this - but
obviously only a fraction of these 10 billion devices could have
their own IPv4 SPI address.

Joel, in (msg05932), you wrote:

> I'll readily grant that judgment calls such as I am making are
> subject to the whims of reality.
>
> In fact, if we do not get a reasonable, deployable, usable,
> multihoming and multiconnectivity solution for IPv6, then it won't
> matter whether folks bury v4 in prefixes, or switch to v6.

Not necessarily.  Maybe the only viable user base for mass IPv6
adoption will be mobile devices, starting in 2015 or whatever - and
maybe that would not lead to a 300k+ IPv6 scaling problem until 2025
or so.

In all those years, the scaling problem of IPv4 is going to get a lot
worse, and will remain worse than IPv6's for a long time after.

The IPv4 Internet matters.

It is the Internet which everyone uses and depends upon.

Only when everyone has their own IPv6 space, connectivity, host
stacks and applications will no-one rely on IPv4.  That is not going
to happen in the foreseeable future, such as by 2025.

> Conversely, if we do have a good solution, that is less painful
> than the various de-aggregation drivers for IPv4, we have a good
> shot at moving people because the pain is going to be quite high.

The IPv4 scaling problem will be a growing burden, but it is hard to
foresee a time when it will be possible to sell anyone an IPv6-only
service, except perhaps as noted above for cell-phones.

In the foreseeable future, I am sure it would always be profitable to
sell NATed IPv4 connectivity to homes (DSL, HFC, fibre, WiMAX etc.)
and to cell-phone users than to not give them this and only give them
IPv6 global unicast space.

For better or worse, this would just prolong our dependence on IPv4
as the only Internet which everyone uses.  There may well be another
Internet such as IPv6, but why would anyone bother paying for a
service on that if they couldn't get at least NATed IPv4 connectivity
as well?

> I would be happy to see an answer that also helped v4. But I think
> that is a distinctly secondary consideration.

I think your perspective is based on a long history of IETF optimism
that IPv6 will be widely adopted real soon.  This optimism has proven
to be unrealistic for ten or fifteen years.  I think the current wave
of IPv6 boosterism based on the purported imminent collapse of IPv4
is just a continuation of this pattern.

All that is happening with IPv4 is that the last fresh paddocks of
unused land are now being built upon.  There's plenty of scope for
squishing houses closer together - and then for building multi-storey
houses (NAT).   Without NAT, some kind of crunch would occur with
IPv4 if there was ever a need to connect more than about 3 or 3.5
billion hosts at any one time.  With NAT, there's no limit.

NAT is not the ideal of any-to-any Internet connectivity, just like
the crowded suburbs with multi-storey dwellings don't provide all the
benefits of living in an uncrowded piece of countryside.  But just as
crowded cities get more crowded, so will IPv4.  The attraction is the
same for cities as for the choice of which Internet use - most people
want and need to be where all the other people are.

Eric, you wrote, in (msg05933):

> I'm sure that Noel's insight below is not contentious to this
> list.

(Quoting Noel in msg05876):

  >> Because of the inertia of the IPv4 service interface, with
  >> the amazingly huge ball-and-chain of deployed base which
  >> speaks it, I think IPv4 addresses will be around for a very
  >> long time - even if, inside the network, we move to something
  >> else, e.g. between xTRs. In other words, TCP will continue to
  >> see IPv4 addresses - even if those are mapped into 'something
  >> else' shortly after leaving TCP.

> However, his observation was reinforced in my own thinking when I
> observed how many embedded systems have been using IPv4 for some
> time now. This taught me that my thinking has been far too small:
> IPv4 has become truly ubiquitous in the literal meaning of the word
> and now includes so many things that never previously were
> networked.

I agree.

> Another aspect of Noel's insight was reinforced to me when I noted
> that many environments (especially those operating in lower
> bandwidth situations) had chosen frame sizes to correspond with
> IPv4 packet sizes to the extent that deploying IPv6 in those
> environments has to overcome significant performance problems
> unless header compression is used.

Could header compression always compress headers sufficiently?

> However, the bottom line remains as it always has been: there is
> a strong business case for the end user to *not* support
> non-value-added protocol diversity.  I wish that RFC 1687 were no
> longer true for IPv6, but, unfortunately, it still is. The key
> question is for how much longer?

  A Large Corporate User's View of IPng
  E. Fleischman Boeing Computer Services  August 1994
  http://tools.ietf.org/html/rfc1687

Here are 3 of the 5 points in the executive summary:

    1)  Large corporate users generally view IPng with disfavor.

    2)  Industry and the IETF community have very different values
        and viewpoints which lead to orthogonal assessments
        concerning the desirability of deploying IPng.

    3)  This paper provides insight into the mindset of a large
        corporate user concerning the relevant issues surrounding an
        IPng deployment.  The bottom line is that a new deployment of
        IPng runs counter to several business drivers.  A key point
        to highlight is that end users actually buy applications --
        not networking technologies.

> The argument that we are rapidly running out of IPv4 addresses has
> always been a significant concern for me personally.  New
> deployments (e.g., civil aviation's ATN IPS) and certain industries
> that have vast numbers of networked devices (e.g., electrical power
> industry) are excellent candidates to adopt IPv6 for this very
> reason. However, for the majority of end users, I expect us to
> prefer to indefinitely use IPv4 by leveraging map-and-encaps
> techniques such as RANGER, despite the fact that RANGER is part of
> the effort to support a clean migration to IPv6.

I agree with all this, except I would use the term Core-Edge
Separation architectures, and at present I think LISP and Ivip are a
better solution than RANGER for either IPv4 or IPv6.

Brian, in (msg05934) you wrote, responding to Noel's (msg05930):

> That is the true unknown for IPv4 - now that we have reached the
> end game in prefix allocation, will the resulting food fight
> recreate a swamp of a few million tiny prefixes that can't be
> aggregated?

I think this is an evocative description!

However, if I was standing on railway tracks and saw a freight-train
coming, I wouldn't regard its imminent arrival as a "true unknown".

> This is the main reason that my CCR paper says: don't extrapolate
> the graphs. We really don't know if the number of prefixes per
> origin AS will remain stable or grow signficantly.

As I argued above, I think there are strong reasons to believe the
rate of growth will accelerate.  However, I haven't yet read your paper:

   Observed Relationships between Size Measures of the Internet
   Brian E. Carpenter       April 2009
   Department of Computer Science, University of Auckland
   ACM SIGCOMM Computer Communication Review Volume 39, Number 2,

Scott, in (msg05935) you wrote, responding to Joel's (msg05925):

> Joel, I don't quite get it.  First, if you only deal with v6
> then would you create two parallel Internets?  If so you need
> routing and addressing interworking.

I agree.  The protocols I can think of which directly inter-work are
email and DNS.

Also, as Patrick and I discussed in recent days in the "SIP will work
fine with CES, CEE won't work with NAT" thread, there is also a lot
of scope for IPv4/IPv6 agnostic client-server communications based on
FQDNs and HTTP.

> Second, there will be pressure for growth in the number of IPv4
> _prefixes_ and if we don't design a way to support it, people will
> find ways we don't like.

Yes - I agree entirely.

  - Robin

MNs with public addresses?

There are potential problems with having a cellphone with a global
unicast IPv4 address or an IPv4 /64  There would be no way of
controlling unwanted streams of packets, from targeted attacks or
from random or port scanning for vulnerable hosts.  These would cost
money and clog up the downstream link.

Behind NAT, the cellphone only has to receive packets which are
returned by hosts to which it has opened up a session.  A single,
reasonably random, IPv6 address might be less of a problem since it
can't be plastered with packets sent out to sequential or random
addresses.

TTR Mobility is intended to provide a portable IPv4 address (or
multiple of them) to each mobile device - or an IPv6 /64.  If
unwanted packets were a problem, then the TTR company would need to be
responsive to each customer's preferences and block unwanted packets
at the TTR, so they are not sent on the tunnel to the mobile device
itself.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 4 Feb 2010 09:20:04 -0800

>From: Robin Whittle [mailto:[email-masked]] 
>> From Eric Fleischman
>> always been a significant concern for me personally.  New deployments 
>> (e.g., civil aviation's ATN IPS) and certain industries that have vast 
>> numbers of networked devices (e.g., electrical power
>> industry) are excellent candidates to adopt IPv6 for this very reason. 
>> However, for the majority of end users, I expect us to prefer to 
>> indefinitely use IPv4 by leveraging map-and-encaps techniques such as 
>> RANGER, despite the fact that RANGER is part of the effort to support 
>> a clean migration to IPv6.

>I agree with all this, except I would use the term Core-Edge Separation 
>architectures, and at present I think LISP and Ivip are a better solution 
>than RANGER for either IPv4 or IPv6.

I value your articulate insights, Robin. 

You apparently read the above paragraph of mine as trying to comment on the technology choice confronting the RRG. Rather, I was explaining what the large end user has a very high probability of doing concerning the issues that the RRG is considering. This is a different topic, since I believe that the RRG is primarily oriented to ISPs and largely lacks the large end user viewpoint.

Large end users with an adequate supply of IPv4 addresses have a strong business motivation to maintain their network infrastructure "as is". The only thing that I can see changing this is if a "killer app" requires the use of IPv6, but thus far no such "killer app" has appeared except in niche contexts (e.g., smart grid, ATN IPS). Should one appear, then IPv6 will be deployed as needed and the network infrastructure will be modified to accommodate it. By contrast, should external business relationships or government decrees require IPv6, then the end users' externally facing network will support IPv6 but the internal infrastructure itself is unlikely to change.

Large end users lacking an adequate supply of IPv4 addresses for their continued growth are also likely to remain IPv4-only in a similar manner for parallel reasons. There are many map-and-encaps technologies that enable end users to retain IPv4 indefinitely in combination with private addresses. I only mentioned RANGER because I believe that it is a clean alternative.

Of course, should senior management become convinced to bear the expense to internally support IPv6 for patriotic or other non-business reasons, then this analysis will not pertain to that particular company.

Mergers and acquisitions are a special case that should worry the RRG because of the possibility of companies externally advertising multiple discontinuous PI address spaces. Of course, such issues already exist for corporations using pre-CIDR addresses.

Should governments push IPv6, then the issues confronting the RRG will become more problematic due to the need to externally route a combination of IPv4 and IPv6 for corporations that solely use IPv4 today and who will probably continue to only use IPv4 internally in the future.

--Eric

---

From: Robin Whittle <[email-masked]>
Date: Fri, 05 Feb 2010 12:20:12 +1100

Short version:     A taxonomy of different types of end-user network
                   including massive adoption of mobile devices in
                   the foreseeable future.

                   Large IPv4 networks doing whatever they can to
                   keep using IPv4.

                   Government-mandated adoption of IPv4 by many
                   larger networks would contribute to the IPv6
                   scaling problem - but may not actually advance
                   genuine IPv6 adoption.

Hi Eric,

Thanks for your appreciative message:

EF> The argument that we are rapidly running out of IPv4 addresses has
EF> always been a significant concern for me personally.  New
deployments
EF> (e.g., civil aviation's ATN IPS) and certain industries that have
vast
EF> numbers of networked devices (e.g., electrical power
EF> industry) are excellent candidates to adopt IPv6 for this very
reason.
EF> However, for the majority of end users, I expect us to prefer to
EF> indefinitely use IPv4 by leveraging map-and-encaps techniques
such as
EF> RANGER, despite the fact that RANGER is part of the effort to
support
EF> a clean migration to IPv6.

> You apparently read the above paragraph of mine as trying to
> comment on the technology choice confronting the RRG.

No - I was focused on the main body of what you wrote, not the
mention of map-and-encaps (CES) and RANGER at the end.

My interpretation was that your main points were:

 1 - End-user networks which need massive numbers of IP devices
     and which are installing all new hardware and software to
     achieve this are likely to adopt IPv6.  (I think this is
     particularly the case with wireless linked utility meters
     which are not required to be publicly accessible, don't
     handle large numbers of packets and don't need to communicate
     with all hosts on the IPv4 Internet.)

 2 - Most end-users - who are using the IPv4 Internet day-to-day
     and who need to be able to communicate with any other host
     which is involved in global communications (and all such hosts
     today use IPv4) - will do whatever they can to keep using
     IPv4, rather than attempt to adopt IPv6 to the exclusion of
     IPv4.  This is because the IPv6 Internet doesn't have the
     most important property of the IPv4 Internet: that everyone
     who wants to use Internet communications is using this
     Internet.

 3 - That a successful Core-Edge Elimination architecture (these
     were previously known as map-and-encaps) would enable a more
     intensive use of IPv4 address space - and therefore enable
     more continued use of IPv4 than would otherwise be the case.

A agree with all this, and wanted to signal this while at the same
time noting that I didn't think RANGER was the best example of a
successful Core-Edge Elimination architecture.

What you wrote was primarily about what und-user networks would do -
and I think with an implication that a CES architecture should be
chosen (which I agree with) and with RANGER as an example of such an
architecture - which it is.

> Rather, I was explaining what the large end user has a very high
> probability of doing concerning the issues that the RRG is
> considering. This is a different topic, since I believe that the
> RRG is primarily oriented to ISPs and largely lacks the large end
> user viewpoint.

I think the RRG would benefit from having a lot more hands on deck -
especially from ISPs and end-users of all types, from universities
and corporations, to small businesses, mobile operators and companies
which might be interested in providing global mobility via the TTR
approach.

It is hard for me to say whether ISPs are overly represented in the RRG.

My biggest concern, in addition to general lack of people who are
prepared to read and constructively discuss a variety of proposals,
is that there are a number of people who think it is both practical
and desirable to rewrite all existing Internet applications to
support the Locator / Identifier Separation naming model which they
prefer.  This is what all CEE architectures do - though they do so in
differing ways.

Loc/ID separation is more flexible than the current naming model of
the Internet - and to these people more architecturally correct.
However I think it will slow down the establishment of Internet
communications, and exacerbate the problems of battery-powered hosts
on slow, high-latency, less than 100% reliable wireless links.

CEE takes the load off the routing system.  If CEE was universally
adopted then there would be no more need for PI space.  All hosts
would be using portable Identifiers and there would be no need for PI
space - since portability, multihoming and inbound TE would work fine
with CEE with each end-user network using one or more PA prefixes.

Once CEE was universally adopted, we can keep the current routers,
BGP and the DFZ just as they are.  Until it is universally adopted,
PI prefixes will remain the only way of providing portability and/or
multihoming.

CEE involves burdening all hosts with extra work so the routing
system doesn't need to be altered.

It also involves all applications being modified - many of them
substantially or radically - for CEE.  It also involves a universal
abandonment of IPv4 and the adoption not just of IPv6, but of the
CEE-enhanced IPv6.

Most CEE protocols assume the use of IPv6, because there is room to
put both the Identifier and Locator in the one IPv6 address.  This
does not work with IPv4.  A more fundamental problem is that
multihoming with CEE involves using double the global unicast address
space, or triple if three upstream ISPs are used.  This is never
going to be practical in IPv4.

So not only must all applications be rewritten, they must be moved to
this modified version of IPv6, and all user's Internet access
services will need to support IPv6 as well.

All this to take the load off the routing system and use a naming
model which some people regard as more elegant or architecturally
correct.   I think the routing system should serve the needs of hosts
- and that the current naming model is better than that of Locator /
Identifier Separation.

> Large end users with an adequate supply of IPv4 addresses have a
> strong business motivation to maintain their network infrastructure
> "as is".

Yes - and this is in addition to the absolute requirement, for pretty
much all end-users, that their Internet service enables them to
communicate with all other Internet users.

I think there are two broad types of end-user now and a third in the
near future:

  1 - Those who can do what they want when their hosts are behind
      IPv4 NAT.  This includes most of what home and SOHO users do
      today, and a great deal of what all other end-users, do since
      email, WWW, instant messenger, file upload to public sites and
      SSH to public sites all work fine from behind NAT.

      To the extent that these people want or need a web site, they
      usually find it best to get a cheap hosting arrangement whereby
      another end-user company runs a virtual server for them.

      It seems that everyone and their dog (an Aussie term, but
      literally true in some cases) has a Facebook or MySpace page,
      and this serves their needs for an easily updated WWW page,
      without any fuss or expense with domain names or running their
      own virtual server.

      These people are generally perfectly well served by existing
      single-homed DSL etc. services, with a single IPv4 address
      and their own NAT router (though 99% don't know or understand
      what NAT means).

      However, I think a significant number of these end-users
      rely on port-forwarding from the single IP address their
      home or office is given.  This means they wouldn't be able to
      run certain applications which are important to them if the
      port number is fixed for the application and if the address
      they get is behind a NAT box in the ISP.  That NAT box will
      have one global unicast address and won't be able to satisfy
      the port forwarding requirements of the multiple customers
      which access the Net through the box.

      So these people do fine with a single IPv4 global unicast
      address per home or office - and DHCP allocation of the address
      is fine.  They generally don't need a fixed address, though it
      might be convenient for many if they did, since they could
      use port forwarding to, run their own web server, game
      server etc.  They don't need portability - since it is not too
      difficult to change any DNS they have on the rare occasions
      when they choose another ISP.

  2 - Those who need stable, reliable, public IP addresses to run
      mail-servers, web-servers which they host on their own
      premises, VPN access from other sites and their staff to
      the local network etc.

      a - Some of these can do all this with a single IPv4 address.
          I can do all this with a GNU-Linux box doing NAT, WWW, DNS,
          SMTP, IMAP, SSH etc - all running from a single fixed IPv4
          address.  My DSL service is highly reliable (I don't recall
          an outage in nearly 4 years) and I don't really need
          portability or multihoming.

          I don't need portability or multihoming, since the
          multihoming arrangements could scarcely be more reliable
          than the service I already get, and it would be easy to
          change my DNS to move to a new IPv4 address if I chose
          another ISP.

      b - Other end-user networks which can do what they want
          with a single IPv4 address per physical site, but which
          want/need portability and mulithoming.

          If I was running a more substantial business, especially
          one where I ran my own TLS site for credit card
          transactions and other important customer contact
          purposes, I would want multihoming.

          Even if these end-users have a substantial network,
          this will mainly be of hosts behind NAT.  With a single
          IPv4 address many of them wouldn't really want or
          need portability - but others would.

          This would be particularly the case if they had
          multiple sites and relied on the IPv4 address of each
          site for VPN connections - and didn't want to rework
          all their connections whenever one of their hundreds
          of shops, branch offices etc. chose a new ISP.

      c - End-user networks with substantial networks who need
          larger amounts of global unicast IPv4 address space -
          beyond a handful of IPv4 addresses.

          These end-users want/need portability, multihoming and
          probably inbound TE.

          These organisations range from schools, companies with
          more than a handful of employees, through universities,
          corporations etc. right up to the largest multinational
          companies.

          The larger ones get it now, with their own PI space.  The
          remainder are not getting it - and this is the part of the
          routing scaling problem which is hardest to measure.

      Most of these end-user networks which don't yet have PI
      space could probably work perfectly well from a much
      smaller amount of space than 256 IP addresses.  Quite a
      few of them could probably function OK with a single IPv4
      address.  For instance, a company with a chain of hundreds
      or thousands of shops or small offices all over the world
      could probably run fine with each such office or shop having
      a single IPv4 address

      Even amongst those with PI space, the huge popularity of
      the smallest chunk available (256 IPv4 addresses in a
      /24 prefix) indicates that many or most of them would
      probably get by with significantly less than 256 addresses.

  3 - In addition, in the foreseeable future, there are going to be
      billions of end-users with IP-capable "cell-phones" - or
      whatever such things will be known as - including things which
      are general purpose computing devices, hand-held or laptop.  I
      think these end-users are like the first group, except that
      they are generally not going to be running their own
      WWW, mail or game-servers on their cell-phone.

      There's no way of giving most of these end-users their
      own IPv4 global unicast address space, but I figure they
      could all get IPv4 access via NAT.

      They could all be given their own IPv6 global unicast /64, or
      their own global unicast IPv6 IP address, but these would
      change every time they got a new connection to whatever
      access networks they were using.  When using wired Ethernet
      or using WiFi, at home, or in the office, these cellphones
      laptops and probably iPads etc. would probably get an IPv4
      address behind NAT.

      With a TTR Mobility system in which the MN could tunnel
      to TTRs with either IPv4 or IPv6, and with the TTR system
      using both the IPv4 and IPv6 CES systems, then all these
      end-users could have their own globally mobile, IPv6
      space (such as a /64 or a single IPv6 address).  A subset
      of them could have their own globally mobile IPv4 address
      - since there aren't enough IPv4 addresses for all
      5 billion 10 billion or whatever such mobile devices.

I think you were referring to the larger end-user networks which
roughly fit my 2c classification.

> The only thing that I can see changing this is if a "killer
> app" requires the use of IPv6, but thus far no such "killer app"
> has appeared except in niche contexts (e.g., smart grid, ATN IPS).

Can you suggest any pages with good overviews of the Aeronautical
Telecommunications Network (ATN)?  I understand it is OSI-based, but
that the IPS (Internet Protocol Suite) is an important alternative.

There's nothing in the IPv6 protocols which seems to be superior to
IPv4 from the point of view of most "applications", when thinking of
an "application" as a piece of software running on a PC or server.  I
think there is more Mobile IP support for IPv6.   Better mobility
support and no effective limits on public and private address space
are also important in some "applications" - in a more expansive use
of the term to cover entire global systems, such as the ATN.

> Should one appear, then IPv6 will be deployed as needed and the
> network infrastructure will be modified to accommodate it.

Is this happening with ATN/IPS?

> By contrast, should external business relationships or government
> decrees require IPv6, then the end users' externally facing network
> will support IPv6 but the internal infrastructure itself is
> unlikely to change.

OK.

I am wary of government mandates pushing larger organisations to
adopt IPv6 or whatever.   Some think that they can afford the
whatever this costs - but ordinary people collectively pay for all
these organisations' expenses through taxation or by directly or
indirectly paying for their services.

> Large end users lacking an adequate supply of IPv4 addresses for
> their continued growth are also likely to remain IPv4-only in a
> similar manner for parallel reasons.  There are many map-and-encaps
> technologies that enable end users to retain IPv4 indefinitely in
> combination with private addresses. I only mentioned RANGER because
> I believe that it is a clean alternative.

I don't see how a CES system such as RANGER, LISP or Ivip can
indefinitely meet the needs of end-user networks which require more
global unicast addresses.  NAT can already give client-style access
to IPv4 Internet - and that has nothing to do with a CES architecture.

A CES architecture will enable each physical site of a large
organisation to get one or any number of IPv4 addresses, in a stable,
portable and multihomable fashion, via any one or more ISPs being
used for connectivity.  This will be more flexible and will be less
expensive than the current practice of each site advertising a /24 -
which chews a whole 256 IPv4 addresses, even when only one or a few
are needed at each site.

So I expect a good CES solution will enable a much greater
utilization of IPv4 address space.  But this is not an indefinite
solution, because the 3.7 billion addresses may be too few, at some
stage in the future, for the wants/needs of all Internet users.

It is easy to imagine 3 billion, 6 billion or whatever IP-based
cellphones which ideally would have their own IPv4 address, globally
portable, via TTR Mobility.   That can't happen.

I guess that some people will get their own globally mobile IPv4
address and the rest will make do with NATed IPv4 access.  If they
want globally mobile address space for their cellphone - which would
allow direct application protocols between these devices and
non-mobile networks at home, also with IPv6 space - then there would
be no problem providing this via TTR Mobility.

> Of course, should senior management become convinced to bear the
> expense to internally support IPv6 for patriotic or other
> non-business reasons, then this analysis will not pertain to that
> particular company.

In which case the shareholders will have some interesting questions
for them . . .

> Mergers and acquisitions are a special case that should worry the
> RRG because of the possibility of companies externally advertising
> multiple discontinuous PI address spaces. Of course, such issues
> already exist for corporations using pre-CIDR addresses.

>From the point of view of maximising IPv4 address utilization, we
might expect a pair of merged companies to consolidate their usage
and while leaving some room for expansion (and other acquisitions)
probably be able to return some of the PI space the previously two
companies were using.

However, it would be a lot of work - and maybe it is best to think of
large companies as modular units, not bonded together by
interdependence on shared address space and overly entangled networks.

Untangling a sub-division from the corporate address space and
preparing it for sale to another company sounds like a disruptive and
expensive process.

> Should governments push IPv6, then the issues confronting the RRG
> will become more problematic due to the need to externally route a
> combination of IPv4 and IPv6 for corporations that solely use IPv4
> today and who will probably continue to only use IPv4 internally in
> the future.

Yes, since a properly written regulation would surely require all
corporations, universities or whatever above some size limit to get
their own PI space, advertise it in the DFZ and demonstrate that
their networks (PCs? laser printers?) were fully IPv6 capable.

Affirmative action for IPv6 . . .

 - Robin

---

From: Paul Jakma <[email-masked]>
Date: Tue, 9 Feb 2010 14:34:20 +0000 (GMT)

> want and need to be where all the other people are.

If we're thinking in terms of analogies, it's important to note where 
they break down. The "build up, cause can't build out" analogy breaks 
down in 2 ways:

1. Earth isn't Trantor yet, many urban areas are still expanding.

    (Indeed, in europe, urban areas in many cases have gotten /less/
     crowded precisely through outward expansion and rehousing people
     from congested slums of the industrial revolution to developments
     and "new towns" in the country side, what we know think of as
     suburbs.)

2. There is no definite limit to building up, though it does
    become more costly to engineer whereas there are definite limits
    to 'building up' in IPv4.

    I.e. we /can/ make bigger and bigger buildings, for a given
    footprint. Whereas there are fixed, fundamental limits to IPv4
    that hard-constrain how many of todays ULPs can go through.

You might respond that the internet would adapt by changing the ULPs 
to better make use of IPv4. E.g. we could make NAT friendly 
transport protocols:

   - Add a "Internal host ID" field to ULP headers, like TCP.
   - Increase the range of the flow^Wport ID

So that a transport address on the internet then becomes some kind 
of 3 tuple:

  (public IP of the concentrator, network specific host ID, flow/service ID)

The core internet routing tables are then constrained to O(|public IP 
space|).

Concentrators can be stateful or stateless at their discretion. If 
concentrators are stateless, you could even have anycast 
concentrators and have a number of concentrators service a single 
concentrator ID.

So basically, your argument is that the locator/identifier split 
should be done by changing transport protocols - retaining IPv4 
compatibility - rather than done by shimming something in to make it 
all transparent to ULPs (map and translate/encap).

I have to say, I kind of agree that changing the transport is the 
best way to go in the long run. It's simpler than map-encap and it 
seems we could as easily upgrade the transport layer on the internet 
as we could move the internet to IPv6 (needed for some map 
translate/encap proposals). Plus, it'd work with either v4 or v6.

Anyway, I guess I'm stating the obvious :)

NB: The upper-layer adaption could also be done without touching the 
transport layer, through higher-level application specific 
multi-plexing. But then we definitely no longer have anything 
resembling the internet, and I assume no one here wants that.

regards,
-- 
Paul Jakma	[email-masked]	Key ID: 64A2FF6A
Fortune:
"The medium is the massage."
-- Crazy Nigel

---

From: Paul Jakma <[email-masked]>
Date: Fri, 12 Feb 2010 21:04:21 +0000 (GMT)

On Tue, 9 Feb 2010, Paul Jakma wrote:

> You might respond that the internet would adapt by changing the ULPs to 
> better make use of IPv4. E.g. we could make NAT friendly transport protocols:
>

I've expanded on this at:

  http://pjakma.wordpress.com/2010/02/12/making-the-internet-scale-through-nat/

Yeah, it's probably a bit crackful, but it's at least interesting to 
consider what could happen if NAT "sticks" -> what kind of "small" 
hacks could develop, and where could they go.

regards,
The MGs ran out of gas.