Subject: [Sdn] Timeliness WAS(Re: Thoughts on Defining control and Management Planes

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 11 Nov 2013 08:41:34 -0500

While I am enjoying Yaakov's postings, let me just zone in on
one issue:

On Thu, Nov 7, 2013 at 11:37 AM, David Meyer <[email-masked]> wrote:

> LB(convergence) = RTT(controller, switch) + PPT(controller) + PPT(switch)
>
> where
>
> RTT is the round trip time, and PPT is the packet processing time
> (note in the case of something like OF you might have sum this over
> the entire  path).

As Yaakov points out - there is the "human intervention" aspect in
what would be labelled as "management". This is the point that was being made
in our draft as well. If you have a human in the loop, the PPT(controller) is
typically designed to have tolerance in the seconds range.
Granted the new kids (and OSS/BSS) solutions have mostly removed the
human from the loop. The challenge as we express in the draft is that the
underlying protocols are not designed to handle the rates a machine can
generate. Execing a CLI via a bash script is still in the 100s of milliseconds.
Netconf being ascii has high PPT on both ends (the switch i am running has
a tiny ppc 32 bit processor which doesnt have any offload for string
processing).

On the other hand OF or ForCES were designed from ground up to be
able to handle large transaction rates.

Is someone making the contention that this is no longer important just
because we now have automation as a driver? If you are - please explain
why ;->

cheers,
jamal

---

From: "Russ White" <[email-masked]>
Date: Mon, 11 Nov 2013 08:59:09 -0500

> because we now have automation as a driver? If you are - please explain
why
> ;->

There's a difference between setting definitions to ground a conversation
and deciding what is important within the realms thus defined... In fact,
the point you're trying to make -- that automation makes the current
assumptions underlying "management plane" operations problematic -- is
specifically the situation you get when you start with what I consider to be
a bad definition. If you define the management plane as "those things which
interact with the device on which the control plane and data planes run, and
those things that interact with the way the control plane operates," (in
other words, what doesn't fit into the control or data planes fits into the
management plane), then you can quickly see there should be absolutely no
assumption about timing and speed of operation between the three.

When you define management plane as, "those things which interact with
humans," you imply timing -- and hence either are forced to consider
virtually everything that's automated part of the control plane (including
checking the temperature of the processors on the box, or the speed of a
hard drive spin, etc.), simply because it's automated. In this case, a bad
definition drives us to muddling two different sets of functionality into
one "thing," which makes it much harder to think through the problem of
centralized verses distributed control planes, what's needed where, etc. (a
distributed protocol to carry the cpu temp to all devices in the network,
anyone??).

The bottom line, to me, is that routers and switches process packets. Models
designed to help us understand and classify the operation of networks based
on routers and switches should, therefore, be based on processing packets --
not services, not collections of such devices, not naming systems, and not
use cases. Once we get the idea of what a forwarding system looks like, then
we can begin to fumble our way around trying to figure out what a group of
these things looks like, acts, can be controlled, etc.

:-)

Russ

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 11 Nov 2013 09:17:19 -0500

On Mon, Nov 11, 2013 at 8:59 AM, Russ White <[email-masked]> wrote:

> a bad definition.
> If you define the management plane as "those things which

[comments below]

> distributed protocol to carry the cpu temp to all devices in the network,

Timing is relevant IMO for the historical nature around it. Yaakov aptly
pointed out the human loop. Protocols and interfaces that are today
automated were designed with a human loop in mind. Usability for example
is an important factor.

> these things looks like, acts, can be controlled, etc.

We are sticking to the term "resources" in our doc for "things that you
interact with" - regardless of where they sit. Someone/thing that has
brains either controls or manages those "things".
Datapath is a packet processing resource; but: that is just one resource
that exists in a datapath.
If i am setting a debug level for logs on a line card then that is a different
resource altogether.
Your response that entering a static route makes it a control plane
activity i think needs to be qualified. If i enter a static route from a
CLI - that is a management activity because the timeliness is not
as important as if i exported the same route from an IGP.
I would argue this has nothing to do with whether the later is
automated or not; more to the urgency of it.

So, IMO - automation does not qualify something as "control";
the interfaces are not ready for it and timeliness is a factor.

cheers,
jamal

---

From: "Russ White" <[email-masked]>
Date: Mon, 11 Nov 2013 09:29:56 -0500

> > If you define the management plane as "those things which interact
> > with the device on which the control plane and data planes run, and
> > those things that interact with the way the control plane operates,"
> > (in other words, what doesn't fit into the control or data planes fits
> > into the management plane), then you can quickly see there should be
> > absolutely no assumption about timing and speed of operation between
> the three.
> 
> > virtually everything that's automated part of the control plane
> > (including checking the temperature of the processors on the box, or
> > the speed of a hard drive spin, etc.), simply because it's automated.
> > In this case, a bad definition drives us to muddling two different
> > sets of functionality into one "thing," which makes it much harder to
> > think through the problem of centralized verses distributed control
> > planes, what's needed where, etc. (a distributed protocol to carry the
> > cpu temp to all devices in the network, anyone??).
> 

So, should we put absolute times around it then?

- Data Plane -- timing less than 500ms
- Control Plane --timing greater than 501ms and less than 10 seconds
- Management Plane -- timing greater than 10 seconds

Do humans with faster reaction times now count as control planes rather than
management planes? Or are all managers just slower than all "control
humans?" :-)

Sorry, this doesn't work --I don't care what the "historical nature," is...
We don't want a definition that's "historically accurate," or etymologically
correct. We want a definition that will work to build a solid model of
device operation on which we can build a solid model of network and services
operation.

> interact with" - regardless of where they sit. Someone/thing that has
brains
> either controls or manages those "things".

Stick with whatever you'd like --but I'm going to continue to argue with
every revision of your doc that your definitions don't reflect a usable
model in the real world.

What is "brains?" Anything that does processing? And what happens when two
"brains" interact? Is the interaction itself another "resource?" These
definitions are so broad as to say just about anything.

> Your response that entering a static route makes it a control plane
activity i
> think needs to be qualified. If i enter a static route from a CLI - that
is a
> management activity because the timeliness is not as important as if i
> exported the same route from an IGP.

So a static route entered by an automated process which is faster than a
distributed control plane is not control plane simply because it's
traditionally entered by fingers on a keyboard? I suppose it all depends on
the typing speed of the person on the keyboard... ?? And setting up the
configuration to export a route from an IGP is somehow faster than
configuring a static route?

> So, IMO - automation does not qualify something as "control"; the
interfaces
> are not ready for it and timeliness is a factor.

You are trying to define something based on a mixture of requirements and
operations. "This operates that way, so it is x, but the timing of that is
y, so it is y." When you encounter overlaps, these definitions aren't going
to work --nor are they useful to describe timing requirements. If I want to
describe a requirement for a management plane operation that executes in
less time than you've allowed, must I now call it a data plane requirement,
no matter where it executes? In other words, "I need to check the cpu
temperature every 200 ms for the next 10 seconds," is now a data plane
requirement simply because it's fast, not because of the operation actually
being performed.

The only way towards clarity is to separate timing --traditional,
historical, modern, or future-- from the operations. Then you can describe
an operation, determine the timing requirements for performing that
operation, and then design a system that reaches those performance
requirements. Simply saying, "this is slower, so it's control plane, and
that's slower still, so it's management plane," won't help with that process
at all.

:-)

Russ

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 11 Nov 2013 10:00:35 -0500

> management planes? Or are all managers just slower than all "control

Picking some handwave numbers maybe sensible.

> device operation on which we can build a solid model of network and services

And isnt basing it on today's realities meet sufficiently meet those goals?
The interfaces for classical "management" were simply not designed to scale to
the latencies and throughput needs of "control". You would have to
perform acrobatics to get netconf in all its glory with yang to achieve 100K
table updates per second.

> definitions are so broad as to say just about anything.

The brain:
A human; a clever control plane application; an autonomous system of some
form.
On resource:
I see your point. My view of a resource is something that
can be scheduled by some manager aka "brain" above.
i.e not trying to go beyond what is defined today.

> configuring a static route?

I said if it is entered by a CLI, then the sense
of urgency is not as much as BGP would have updated the RIB/FIB.
This has nothing to do with how fast you can type;->
It is the intent of the interface. If you take that interface and automate it,
that still doesnt change its noun. It prefixes it with "automated"

> y, so it is y."

Precisely - because the doc is trying to provide some taxanomy. It is not
intended to be judgemental on why someone automating CLIs is saying
they are doing SDN. It is merely stating the differences in the different
things called SDN - and latency and throughput are a part of that.

>When you encounter overlaps, these definitions aren't going

Monitoring the temparature of the CPU is definetely a management activity.
Just because cpu i am monitoring sits on a line card is not important.
>From a timeliness/throughput lens:
I dont expect to be scaling it to 100K times/second. That defeats the intent.

> that's slower still, so it's management plane," won't help with that process

The important part is what existing interfaces do.
If you can tell me that you would automate FIB updates via a
CLI when resyncing 500K routes, or use netconf and achieve the same results
as using ForCES or whatever proprietary scheme used, then timeliness and
throughput are not important. I dont think that is the case.
And that was my starting point in this post.

cheers,
jamal

___________________________________
> sdn mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/sdn

---

From: David Meyer <[email-masked]>
Date: Mon, 11 Nov 2013 07:50:46 -0800

On Mon, Nov 11, 2013 at 6:29 AM, Russ White <<a
href="mailto:[email-masked]" target="_blank">[email-masked]</a>> wrote:>>>
If you define the management plane as "those things which interact>> >
with the device on which the control plane and data planes run, and>>
> those things that interact with the way the control plane
operates,">> > (in other words, what doesn't fit into the control or
data planes fits>> > into the management plane), then you can quickly
see there should be>> > absolutely no assumption about timing and
speed of operation between>> the three.>>>> > When you define
management plane as, "those things which interact with>> > humans,"
you imply timing -- and hence either are forced to consider>> >
virtually everything that's automated part of the control plane>> >
(including checking the temperature of the processors on the box, or>>
> the speed of a hard drive spin, etc.), simply because it's
automated.>> > In this case, a bad definition drives us to muddling
two different>> > sets of functionality into one "thing," which makes
it much harder to>> > think through the problem of centralized verses
distributed control>> > planes, what's needed where, etc. (a
distributed protocol to carry the>> > cpu temp to all devices in the
network, anyone??).>>>> Timing is relevant IMO for the historical
nature around it. Yaakov aptly>> pointed out the human loop. Protocols
and interfaces that are today>> automated were designed with a human
loop in mind. Usability for example>> is an important factor.>> So,
should we put absolute times around it then?>> - Data Plane -- timing
less than 500ms> - Control Plane --timing greater than 501ms and less
than 10 seconds> - Management Plane -- timing greater than 10
secondsIMO this is almost certainly the wrong approach, as whatever
times you decide to use will adjust to technology over time. That is,
the time required by whatever operations is not an architectural
feature; rather these are artifacts of current technology and
approaches.>> Do humans with faster reaction times now count as
control planes rather than> management planes? Or are all managers
just slower than all "control> humans?" :-)>> Sorry, this doesn't work
- --I don't care what the "historical nature," is...> We don't want a
definition that's "historically accurate," or etymologically> correct.
We want a definition that will work to build a solid model of> device
operation on which we can build a solid model of network and services>
operation.History is a good thing to understand, but I agree we need
extensible notions (I hesitate to use the word "models") of current
and if possible future usage.>>> We are sticking to the term
"resources" in our doc for "things that you>> interact with" -
regardless of where they sit. Someone/thing that has> brains>> either
controls or manages those "things".>> Stick with whatever you'd like
- --but I'm going to continue to argue with> every revision of your doc
that your definitions don't reflect a usable> model in the real
world.I've kinda lost which document you're talking about here.>> What
is "brains?" Anything that does processing? And what happens when two>
"brains" interact? Is the interaction itself another "resource?"
These> definitions are so broad as to say just about anything.>>> Your
response that entering a static route makes it a control plane>
activity i>> think needs to be qualified. If i enter a static route
from a CLI - that> is a>> management activity because the timeliness
is not as important as if i>> exported the same route from an IGP.>>
So a static route entered by an automated process which is faster than
a> distributed control plane is not control plane simply because it's>
traditionally entered by fingers on a keyboard? I suppose it all
depends on> the typing speed of the person on the keyboard... ?? And
setting up the> configuration to export a route from an IGP is somehow
faster than> configuring a static route?>> So, IMO - automation does
not qualify something as "control"; the> interfaces>> are not ready
for it and timeliness is a factor.>> You are trying to define
something based on a mixture of requirements and> operations. "This
operates that way, so it is x, but the timing of that is> y, so it is
y." When you encounter overlaps, these definitions aren't going> to
work --nor are they useful to describe timing requirements. If I want
to> describe a requirement for a management plane operation that
executes in> less time than you've allowed, must I now call it a data
plane requirement,> no matter where it executes? In other words, "I
need to check the cpu> temperature every 200 ms for the next 10
seconds," is now a data plane> requirement simply because it's fast,
not because of the operation actually> being performed.>> The only way
towards clarity is to separate timing --traditional,> historical,
modern, or future-- from the operations. Then you can describe> an
ogooleperation, determine the timing requirements for performing that>
operation, and then design a system that reaches those performance>
requirements. Simply saying, "this is slower, so it's control plane,
and> that's slower still, so it's management plane," won't help with
that process> at all.Agreed. In addition, if we really want to model
this we'll need to understand the "separation of timescales" (a key
idea in the study of complex systems, see e.g., [0], anything dealing
with the architecture of transcription networks, or google). I'm not
sure we want to go to this level of description here, but if you open
that can of worms...--dmm[0]
<a href="http://cmg.soton.ac.uk/research/projects/separation-of-timescales-in-models-of-complex-networks/"
target="_blank">http://cmg.soton.ac.uk/research/projects/separation-of-timescales-in-models-of-complex-networks/</a>>>
:-)>> Russ>

---

From: Konstantinos Pentikousis <[email-masked]>
Date: Mon, 11 Nov 2013 17:07:21 +0100

| Stick with whatever you'd like --but I'm going to continue to argue
| with every revision of your doc that your definitions don't reflect a
| usable model in the real world.

Should I read this as "no matter what is the (future/revised) text, we will have an argument"? Maybe I misunderstood, but that doesn't sound too open-minded, let alone suitable for a research group discussion. In fact, such kind of "argumentation" should convince even the most stern opponents of draft-haleplidis-sdnrg-layer-terminology of the need for a such a draft.

Perhaps you would be interested in proposing some text in the grand old IETF tradition:

OLD: ...

NEW: ...

Then we could be more concrete instead of arguing about which part of the text we do not agree upon today--and we will continue to do so till kingdom comes.

Many thanks,

Kostas

---

From: Yaakov Stein <[email-masked]>
Date: Mon, 11 Nov 2013 17:48:31 +0000

No, we needn't put absolute times.

The times of importance are those that would signify unavailability for the particular application.

Thus for interactive applications the times would be shorter than for progressive download ones.

In the old days when voice was the only application, 100 ms was the gold standard,
and so protection switching was required to be mo more than 50 ms.

Y(J)S

-----Original Message-----
From: Russ White [mailto:[email-masked]] 
Sent: 11 November, 2013 16:30
To: 'Jamal Hadi Salim'
Cc: 'David Meyer'; Yaakov Stein; [email-masked]
Subject: RE: Timeliness WAS(Re: [Sdn] Thoughts on Defining control and Management Planes

> > (in other words, what doesn't fit into the control or data planes 
> > fits into the management plane), then you can quickly see there 
> > should be absolutely no assumption about timing and speed of 
> > operation between
> the three.
> 
> > When you define management plane as, "those things which interact 
> > with humans," you imply timing -- and hence either are forced to 
> > consider virtually everything that's automated part of the control 
> > plane (including checking the temperature of the processors on the 
> > box, or the speed of a hard drive spin, etc.), simply because it's automated.
> > sets of functionality into one "thing," which makes it much harder 
> > to think through the problem of centralized verses distributed 
> > control planes, what's needed where, etc. (a distributed protocol to 
> > carry the cpu temp to all devices in the network, anyone??).
> 
> Timing is relevant IMO for the historical nature around it. Yaakov 
> aptly pointed out the human loop. Protocols and interfaces that are 
> today automated were designed with a human loop in mind. Usability for 
> example is an important factor.

Do humans with faster reaction times now count as control planes rather than management planes? Or are all managers just slower than all "control humans?" :-)

Sorry, this doesn't work --I don't care what the "historical nature," is...
We don't want a definition that's "historically accurate," or etymologically correct. We want a definition that will work to build a solid model of device operation on which we can build a solid model of network and services operation.

> We are sticking to the term "resources" in our doc for "things that 
> you interact with" - regardless of where they sit. Someone/thing that 
> has
brains
> either controls or manages those "things".

Stick with whatever you'd like --but I'm going to continue to argue with every revision of your doc that your definitions don't reflect a usable model in the real world.

What is "brains?" Anything that does processing? And what happens when two "brains" interact? Is the interaction itself another "resource?" These definitions are so broad as to say just about anything.

activity i
is a
> exported the same route from an IGP.

So a static route entered by an automated process which is faster than a distributed control plane is not control plane simply because it's traditionally entered by fingers on a keyboard? I suppose it all depends on the typing speed of the person on the keyboard... ?? And setting up the configuration to export a route from an IGP is somehow faster than configuring a static route?

interfaces
> are not ready for it and timeliness is a factor.

You are trying to define something based on a mixture of requirements and operations. "This operates that way, so it is x, but the timing of that is y, so it is y." When you encounter overlaps, these definitions aren't going to work --nor are they useful to describe timing requirements. If I want to describe a requirement for a management plane operation that executes in less time than you've allowed, must I now call it a data plane requirement, no matter where it executes? In other words, "I need to check the cpu temperature every 200 ms for the next 10 seconds," is now a data plane requirement simply because it's fast, not because of the operation actually being performed.

The only way towards clarity is to separate timing --traditional, historical, modern, or future-- from the operations. Then you can describe an operation, determine the timing requirements for performing that operation, and then design a system that reaches those performance requirements. Simply saying, "this is slower, so it's control plane, and that's slower still, so it's management plane," won't help with that process at all.

:-)

Russ

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 12 Nov 2013 06:31:06 -0500

> No, we needn't put absolute times.

Ok, agreed.
I was hoping there would be a good paper to what Dave Meyer pointed out
here:
http://cmg.soton.ac.uk/research/projects/separation-of-timescales-in-models-of-complex-networks/

> and so protection switching was required to be mo more than 50 ms.

I think what you say above may help me make the point.
Can we equate "consistency" to what you refer to as "unavailability"?
i.e. if i was able to update a FIB table at 10K/s then i would be able to reach
consistency faster than if i did it at 1K/s
Given that design intent of "management" interfaces was mostly
targeted for human
interaction both from usability  as well as response tolerances - does
automation
change the fact that they are still constrained to that original
design intent? I think not.
Just because i can automate a CLI with expect does not magically
change that it is slow
(my tie to sense of urgency that offended Russ ;->).
I can update tables a magnitude faster with OF than with netconf. Both
are called SDN technologies.

cheers,
jamal

---

From: David Meyer <[email-masked]>
Date: Tue, 12 Nov 2013 07:25:15 -0800

> http://cmg.soton.ac.uk/research/projects/separation-of-timescales-in-models-of-complex-networks/

I'm still not sure how to apply the idea of separation of timescales
here. In general the idea, at least in the systems literature, is that
if two processes operate on widely varying timescales (like an order
of magnitude or more)  then one or the other can be approximated  by a
constant (in "slower" case) or considered instantaneous (in the
"faster" case). This leads to vast simplification of mathematical
models. However, here we don't really have mathematical models of the
"planes" (save TCP).

> Can we equate "consistency" to what you refer to as "unavailability"?

If I understand what you are saying, I'm not sure you want to do that.
Consistency and Availability are different things, at least from the
CAP theorem perspective.

> are called SDN technologies.

I doubt that if OF had the same ACID properties that an interface like
netconf has that it would be faster. Whether you would want that is
another question.

--dmm

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 12 Nov 2013 11:01:55 -0500

>From an interface perspective, this is the interesting part to me:

PPT(controller) + PPT(switch) is much higher in what is traditionally
"management" vs "control" even if "management", using the same
existing interfaces, is automated. This is because design intents for
such _existing_ "management" interfaces is primarily not focussed
on optimizing those two parameters. One of the arguements for
netconf over SNMP was it was more human friendly for an operator
to see ascii than debug cascaded OIDs.

> Consistency and Availability are different things, at least from the

LB(convergence) points to the "C" in CAP. So yes.

> netconf has that it would be faster. Whether you would want that is

Well even if it did do ACID the PPT(controller) + PPT(switch) part
of OF vs netconf would be a magnitude better.

cheers,
jamal

---

From: David Meyer <[email-masked]>
Date: Tue, 12 Nov 2013 08:12:10 -0800

> of OF vs netconf would be a magnitude better.

Can you back that up with experimental data (or anything else)?

--dmm

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 12 Nov 2013 11:44:07 -0500

> Can you back that up with experimental data (or anything else)?

Lets start with "anything else" and talk about the 3 classical
schedule-able resources (cpu, bandwidth, memory):

1) OF is binary meaning simpler for a machine to parse, XML or
any other ascii format would be more expensive. So more cpu
resources needed. Modern X86 have built in string parsing built in
(making its way to the clibraries already), so overhead of a
netconf message indicating "set table foo value X" while high
is alleviated.
But your switch tiny 32 bit ppc will be much higher; handwave,
an order of magnitude.

2) OF being binary consumes less wire resources than ascii.
several orders of magnitudes less.

3) OF being binary consumes less RAM.
Probably not as a big deal since this would be a transient use.

As to "experimental" data - I think this would make an interesting
paper, so lets see if i can convince someone to work with me on it ;->

cheers,
jamal

---

From: Yaakov Stein <[email-masked]>
Date: Tue, 12 Nov 2013 18:20:54 +0000

No, consistency and availability are quite different

Consistency means that if you sent two packet in rapid succession,
they would be handled in the same (and hopefully correct) way.

Availability means that the packet is immediately handled, even if incorrectly.

So, if in OF you start a new flow, the packet is not matched and so is sent to the controller.
The flow now hangs until the controller receives the packet, figures out what to do with the flow, and configures all the match tables of all the switches.
This is a lack of availability.
>From now on the packets belonging to the flow are all handled consistently.

What happens if a link breaks ?
In a well-engineered network protection switching immediately compensates, 
thus guaranteeing availability, even if the packets are now handled differently (lack of consistency).
In a best-effort OSPF network, a link-down will cause the router before the break
to immediately Dijkstra the next packet along a new path - also availability at the expense of consistency. The new path may even lead to black-holing - really inconsistent behavior.

What happens if we need to send a packet belonging to a new flow to the controller
and the controller (or a link to it) fails ?
Then either we can wait until the controller CAN respond (unavailable)
or decide to forward according to conventional (pre-SDN) methods (inconsistent).

Y(J)S

-----Original Message-----
From: Jamal Hadi Salim [mailto:[email-masked]] 
Sent: 12 November, 2013 13:31
To: Yaakov Stein
Cc: Russ White; David Meyer; [email-masked]
Subject: Re: [Sdn] Timeliness WAS(Re: Thoughts on Defining control and Management Planes

Ok, agreed.

> In the old days when voice was the only application, 100 ms was the 
> gold standard, and so protection switching was required to be mo more than 50 ms.

I think what you say above may help me make the point.
Can we equate "consistency" to what you refer to as "unavailability"?
i.e. if i was able to update a FIB table at 10K/s then i would be able to reach consistency faster than if i did it at 1K/s Given that design intent of "management" interfaces was mostly targeted for human interaction both from usability  as well as response tolerances - does automation change the fact that they are still constrained to that original design intent? I think not.
Just because i can automate a CLI with expect does not magically change that it is slow (my tie to sense of urgency that offended Russ ;->).
I can update tables a magnitude faster with OF than with netconf. Both are called SDN technologies.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 12 Nov 2013 14:55:03 -0500

Yaakov,
Thanks for the insight.

So question: for lack of better wording, is the "accelaration" (or maybe it is
"velocity") towards consistency unimportant?
IOW, If i can update a 500K FIB table 100x faster with ForCES than with Netconf,
does that not make them different?

cheers,
jamal

PS:- I am not here to judge merits of the two, i think they have their place.
If updating the tables was unimportant and usability is more important i would
likely pick netconf. This is what we set out to do and i dont think the message
is getting across.

---

From: ramki Krishnan <[email-masked]>
Date: Tue, 12 Nov 2013 14:46:02 -0800

Hi Jamal,

I agree, "acceleration" is definitely important for certain use cases. There is some work going on in Netconf in the area of binary encoding (see draft below) to achieve the "acceleration" you mentioned.

https://datatracker.ietf.org/doc/draft-varga-netconf-exi-capability/

Thanks,
Ramki

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jamal Hadi Salim
Sent: Tuesday, November 12, 2013 11:55 AM
To: Yaakov Stein
Cc: Russ White; David Meyer; [email-masked]
Subject: Re: [Sdn] Timeliness WAS(Re: Thoughts on Defining control and Management Planes

Yaakov,
Thanks for the insight.

cheers,
jamal

> I can update tables a magnitude faster with OF than with netconf. Both are called SDN technologies.
_______________________________________________
sdn mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/sdn

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 12 Nov 2013 18:33:18 -0500

Hi Ramki,

Thanks for the pointer. The author well identifies the issues i was pointing to
with classical netconf (abuse of the three resources). The fact they mention
human readability is sacrificed even drives the point home more.
I think the proposal will improve bandwidth utilization without a doubt.

If i did not misunderstand - it will improve cpu utilization but not totally as
a protocol that is designed from ground up to  be binary. This is because
it seems to me it requires compression/decompression on both sides
(none which are free). But perhaps that is cheaper than string processing.
I am also not sure it is clever to have a protocol that now does both ascii and
binary - but that is a tangent discussion issue.

cheers,
jamal

---

From: Yaakov Stein <[email-masked]>
Date: Wed, 13 Nov 2013 07:02:39 +0000

No, a management plane operation doesn't become a control plane one just because it is faster.

In general, control plane operations are faster because they are local rather than having to suffer a round-trip time, or because they are automatic rather than having a human in the loop. 

But there are exceptions. Configuring using a terminal directly plugged into a box is management plane, although it is very local.

Control processes are automatic, triggered by events and timers,
and the logic behind them straightforward.
Management operations are initiated, and are directly motivated by high-level (business/mission related) concerns.

Y(J)S

-----Original Message-----
From: Jamal Hadi Salim [mailto:[email-masked]] 
Sent: 12 November, 2013 21:55
To: Yaakov Stein
Cc: Russ White; David Meyer; [email-masked]
Subject: Re: [Sdn] Timeliness WAS(Re: Thoughts on Defining control and Management Planes

Yaakov,
Thanks for the insight.

So question: for lack of better wording, is the "accelaration" (or maybe it is
"velocity") towards consistency unimportant?
IOW, If i can update a 500K FIB table 100x faster with ForCES than with Netconf, does that not make them different?

cheers,
jamal

PS:- I am not here to judge merits of the two, i think they have their place.
If updating the tables was unimportant and usability is more important i would likely pick netconf. This is what we set out to do and i dont think the message is getting across.

On Tue, Nov 12, 2013 at 1:20 PM, Yaakov Stein <[email-masked]> wrote:
> In a well-engineered network protection switching immediately 
> compensates, thus guaranteeing availability, even if the packets are now handled differently (lack of consistency).
> In a best-effort OSPF network, a link-down will cause the router 
> before the break to immediately Dijkstra the next packet along a new path - also availability at the expense of consistency. The new path may even lead to black-holing - really inconsistent behavior.
>
> What happens if we need to send a packet belonging to a new flow to 
> the controller and the controller (or a link to it) fails ?

---

From: Fernando Ramos <[email-masked]>
Date: Wed, 13 Nov 2013 10:49:04 +0000

Excellent explanation on the C. vs A. problem for networks. Thanks Yaakov.
One or two notes below.

>Consistency means that if you sent two packet in rapid succession, they
>would be handled in the same (and hopefully correct) way.
>
>The flow now hangs until the controller receives the packet, figures out
>what to do with the flow, and configures all the match tables of all the
>switches.

True. In practice things are not that easy, so you may need in fact
something like this:

http://www.cs.princeton.edu/~jrex/papers/sigcomm12.pdf
(and other recent related work in HotSDN and EWSDN etc.)

to have some consistency guarantees.

>compensates, thus guaranteeing availability, even if the packets are now
>handled differently (lack of consistency).
>In a best-effort OSPF network, a link-down will cause the router before
>the break to immediately Dijkstra the next packet along a new path -
>also availability at the expense of consistency. The new path may even
>lead to black-holing - really inconsistent behavior.
>

I also loved the bit "immediately Dijkstra the next packet". :-)

Crystal clear explanation. Thanks Yaakov.

Cheers,

Fernando

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 13 Nov 2013 08:50:41 -0500

> one just because it is faster.

Indeed.
I think the timeliness factor is already being proven of importance
by desire to make netconf move faster. Netconf is a protocol that
was designed to be management. And it is now used in some
cases to do control operations - it is natural to think its timeliness
needs to improve (as that draft tries to).

> In general, control plane operations are faster because they are local >rather than having to suffer a round-trip time, or because they are >automatic rather than having a human in the loop.
>

Is going a cross a small number of hops local (CE<-->FE)?

> But there are exceptions. Configuring using a terminal directly plugged into >a box is management plane, although it is very local.
>

Agreed.

> Management operations are initiated, and are directly motivated by
> high-level (business/mission related) concerns.
>

Dont agree 100% but it is the closest thing I have heard
to what we have been trying to say.

cheers,
jamal

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 14 Nov 2013 12:07:54 -0800

Yaakov,

Your note made me think about potential differences and similarities if 
we look at a single device (what we've classically done), and look at a 
network-wide view.

In the device view I see two different behaviors:
D1. Decision on the device - interface down, received routing update 
etc. The device makes a decision. We call that control plane.

D2. Decision off the device - the device is instructed to do something 
(provision a vlan, admin disable a port) from some external entity. The 
device doesn't really care whether that entity is a human, a script, or 
some more capable controller. We've called that management plane.

If we move up to a network view I see two similar, but network-level, 
behaviors:
N1. The network (controller) makes a decision based on some event (a 
router failing, some information received from a router, etc).

N2. The network (controller) is instructed to do something (configure a 
VPN from A to B) from some external entity.

(Note that for both N1 and N2 the device is likely to see the outcome as 
D2 i.e., the device being instructed to do something without the device 
making any decisions.)

I think that we are trying to use the terms we have for D1 and D2 in the 
network-level view, and that doesn't seem to fit well. Perhaps we need 
some new terms for N1 and N2?

    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 14 Nov 2013 14:17:55 -0800

Yaakov,

    Erik

On 11/12/13 11:02 PM, Yaakov Stein wrote:
 > In general, control plane operations are faster because they are 
local rather than having to suffer a round-trip time, or because they 
are automatic rather than having a human in the loop.
 >
 > But there are exceptions. Configuring using a terminal directly 
plugged into a box is management plane, although it is very local.
 > IOW, If i can update a 500K FIB table 100x faster with ForCES than 
with Netconf, does that not make them different?
 > If updating the tables was unimportant and usability is more 
important i would likely pick netconf. This is what we set out to do and 
i dont think the message is getting across.
 >> The flow now hangs until the controller receives the packet, figures 
out what to do with the flow, and configures all the match tables of all 
the switches.
 >> compensates, thus guaranteeing availability, even if the packets are 
now handled differently (lack of consistency).
 >> before the break to immediately Dijkstra the next packet along a new 
path - also availability at the expense of consistency. The new path may 
even lead to black-holing - really inconsistent behavior.
 >> i.e. if i was able to update a FIB table at 10K/s then i would be 
able to reach consistency faster than if i did it at 1K/s Given that 
design intent of "management" interfaces was mostly targeted for human 
interaction both from usability  as well as response tolerances - does 
automation change the fact that they are still constrained to that 
original design intent? I think not.
 >> Just because i can automate a CLI with expect does not magically 
change that it is slow (my tie to sense of urgency that offended Russ ;->).

---

From: Yaakov Stein <[email-masked]>
Date: Sun, 17 Nov 2013 11:48:53 +0000

Yes, "on-device" vs. "off-device" plays a big part, 
but you need to properly define what you mean by a "device".

Conventional switches are actually 2 devices:
one dumb but fast one (the switch fabric) and one smart but slow one.
So-called SDN switches only retain the first device,
replacing the second by a centralized management plane configuring the first one.

The SDN crowd calls this "separating the data and control planes",
while I call it removing the control plane and replacing its functionality by a management plane.

By the way, there is one thing the SDN academics got exactly right.
As I said in a previous email, BFD or ping or Y.1731 is a single OAM function for the entire data plane.
On the other hand, each control plane function has its own instantiation with its own OAM (keepalives) and other functionalities.

This leads to the unfortunate consequence that control plane functionality is not universal.
For example, LDP invented a great extension to the concept of TLVs - the U bit (which specifies whether one should silently ignore or notify upon receipt of an Unknown message type).
However, due to fragmentation of the control plane, this innovation never propagated to other control protocols based on TLVs.

What I think we should really learn from all this is not to insist on eradicating control planes entirely, but rather to start thinking about how we can unify the control plane. 
Actually, 802 tried this once (when they invented GARP - the Generic Attribute Registration Protocol) but it never took off.

Y(J)S

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]] 
Sent: 14 November, 2013 22:08
To: Yaakov Stein; Jamal Hadi Salim
Cc: Russ White; David Meyer; [email-masked]
Subject: Re: [Sdn] Timeliness WAS(Re: Thoughts on Defining control and Management Planes

Yaakov,

Your note made me think about potential differences and similarities if we look at a single device (what we've classically done), and look at a network-wide view.

In the device view I see two different behaviors:
D1. Decision on the device - interface down, received routing update etc. The device makes a decision. We call that control plane.

D2. Decision off the device - the device is instructed to do something (provision a vlan, admin disable a port) from some external entity. The device doesn't really care whether that entity is a human, a script, or some more capable controller. We've called that management plane.

If we move up to a network view I see two similar, but network-level,
behaviors:
N1. The network (controller) makes a decision based on some event (a router failing, some information received from a router, etc).

N2. The network (controller) is instructed to do something (configure a VPN from A to B) from some external entity.

(Note that for both N1 and N2 the device is likely to see the outcome as
D2 i.e., the device being instructed to do something without the device making any decisions.)

I think that we are trying to use the terms we have for D1 and D2 in the network-level view, and that doesn't seem to fit well. Perhaps we need some new terms for N1 and N2?

    Erik

On 11/12/13 11:02 PM, Yaakov Stein wrote:
> Control processes are automatic, triggered by events and timers, and 
> the logic behind them straightforward.
>> Then either we can wait until the controller CAN respond 
>> (unavailable) or decide to forward according to conventional (pre-SDN) methods (inconsistent).
>>
>> Y(J)S
>>
>>
>> http://cmg.soton.ac.uk/research/projects/separation-of-timescales-in-
>> m
>> odels-of-complex-networks/
>>