Subject: Requirements

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 16 Sep 2003 10:10:21 -0400

In conversations about one of the protocols that has been proposed, an
element of the requirements document was pointed out to me:

    10) In a ForCES NE, the FEs MUST be able to provide their topology
    information (topology by which the FEs in the NE are connected) to
    the CE(s).

I consider this a seriously broken requirement.  Cards have minimal (if
any) way to know what they are plugged into.  Even if a true backplane (as
opposed to ethernet switching) is used, the FEs do not generally have any
sensible way to know what is reachable over the backplane.

In fact, I would consider that this should be archtiecturally out of
scope.  Understanding / discovering what the links between FEs interconnect
is a CE task, not an FE task.

It may be too late to remove this from the document.
However, as a mandatory requirement on FEs it is a serious mistake.

I very much apologize for not having noticed and raised this issue earlier.

It is possible that something other than what I am reading is meant by this
requirement.  If so, it needs to be stated very differently.

Yours,
Joel M. Halpern

---

From: Alex Audu <[email-masked]>
Date: Tue, 16 Sep 2003 10:00:33 -0500

Hello Joel,

I agree; this type of information has global scope (in terms of the CE set and
its
FEs) and should be kept in the CE. However, all hope is not lost. An FE
should be able to keep information pertaining to whom it speaks with directly.
Frankly any object in any sort of list should be able to store pointers to its
next or
previous neighbors. And it can be argued that this is the best place to keep
that
information. The CE or some management entity can set the next neighbor
information up. The CE should be able to query the FEs for that information.

In this model, if the CE is involved in setting neighbors up, then CE will have

that knowledge also. If not, then the CE can request for the information. That
also may mean that we are keeping that information in two places.

Regards,
Alex.

---

From: Weiming Wang <[email-masked]>
Date: Wed, 17 Sep 2003 12:27:59 +0800

Hi Joel,

I was having the same feeling as yours regarding who should manage the FE connection, 
when I compared ForCES architecture with current popular backplane based machine box,
which support PnP function. I thought that FE might not be able to perform PnP by itself, instead
it should rely on CE to do this. 

But later I changed the mind, I found in ForCES, CE is even more away from performing
this.  CE has the power to find FEs connected with it but has little power to discover how FEs are 
interconnected with other FEs. Therefore, if we rely on CE to discover interconnections of FEs, only two ways are applicable,
one is manually to set it on CE, the other is to start the FE first then to command the FE to do some discovery, which should
be performed after pre-associationg phase. 

It is true that to let FE to auto-discover the connection is a tough task for FE, but I think that is the most we can do. 
Maybe manual ways are still important choices in many cases to do this. 

I also think a system composed of FEs and connected via a backplane is in fact a pure distributed system, with no master
 in it, which is different from current centralized backplane system, therefore, to design some L2  or even L1 protocol 
to support FE connection auto discovery is still possible. 

I agree with Alex that all FEs are better to maintain a global FE topology info as well as CE maintain it. 

Best regards,
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 17 Sep 2003 08:17:30 -0400

The more I think about this, the less I think that FE topology is ever a
natural FE function.
It may be an LFB function.  that is,
1) one could imagine that some of the topology discovery protocols are
suitable for running on an FE
2) If so, the enabling and activation of such a function would be up to the CE
3) The control of the processing of such a function, and the fetching of
the results, would use the normal forces protocol for (optionally)
creating, setting, and querying an LFB.
4) Thus, in that siutation, there is no FORCES protocol requirement to be met.

It has been suggested to me that in some situations FEs would be configured
with their adjacency topology.  I had initially agreed that as a corner
case that could happen.  The more I think about it, the less I like
it.  That kind of configuration is very error prone.  If one is using a
backplane with slots, or an ethernet switch, or ... then when one adds a
component to the system, all the other components must be
reconfigured!!  This is actually an invitation to non-functional solutions.

The requirement seems to be based on a view that it is somehow natural for
an FE to know its connectivity topology.  In reality, cards have a great
deal of trouble knowing what is reachable over each of their ports.  They
rely on applications (routing protocols, neighbor discovery protocols,
reachabbility discovery protocols) to find this out.  Well such protocols
MAY be delegated to the FE by the CE, if the FE happens to have such
support, that is an LFB not a FORCES protocol requirement.  And, given the
range of protocols and range of maturity for those protocols, I would not
want to mandate including such an LFB in the first release of the FORCES
LFB library.

And no, such an LFB does not meet the requirement as written, since the
requirement is a protocol requirement.

Yours,
Joel M. Halpern

At 12:27 PM 9/17/2003 +0800, Weiming Wang wrote:
>interconnected with other FEs. Therefore, if we rely on CE to discover
>interconnections of FEs, only two ways are applicable,
>one is manually to set it on CE, the other is to start the FE first then
>to command the FE to do some discovery, which should
>It is true that to let FE to auto-discover the connection is a tough task
>for FE, but I think that is the most we can do.
>I also think a system composed of FEs and connected via a backplane is in
>fact a pure distributed system, with no master
>  in it, which is different from current centralized backplane system,
> therefore, to design some L2  or even L1 protocol

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 17 Sep 2003 11:53:46 -0400

> 4) Thus, in that siutation, there is no FORCES protocol requirement to be met.

I think your premise #1) is the problem. An example would be useful:
Imagine two line cards on a single chasis, an incoming packet could
either be either switched/routed localy or across the backplane to the
second line card. There has to be knowledge where that second card is.
You could be simplistic and make that a basic L2 functionality or
intelligent and actually teach the line card how/where to send.
I am using L3 as an example but such knowledge would be useful for any
LFB.

> with their adjacency topology.

Calling it an adjacency topology is probably a little overkill. Refer to
the example above.

> I had initially agreed that as a corner
> reconfigured!!

The only time i see a reconfig happening is when new line cards are
plugged/removed or when links go down or up. The later may not be
necessary. If these events are gonna be happening all the time, then
the overall system may certainly be overloaded.

> This is actually an invitation to non-functional solutions.
>

You may have a valid point in that there maybe line cards that may not
be able to use that topology intelligence. Maybe we should add
capability bits so that FE can tell the CEs what they can do. Note as
pointed earlier, the CEs can teach the FEs about this topology.
You seem to be saying thiongs will be non-functional because you
disagree with any such knowledge to be known by the FE. For this i will
have to agree with the two other people who disagreed/responded to you.
Alex, note i agreed with you - theres hope for the world ;->

> want to mandate including such an LFB in the first release of the FORCES

Again this is based on the premise #1 that you made which is false.
CEs can teach FEs about the topology if they are unable to discover it.

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 17 Sep 2003 13:14:16 -0400

I presume I am misunderstanding something in your example.
The requirement is about inter-FE topology knowledge being delivered by FEs.
As such, to apply it to your example I must presme that the two line cards
are two FEs.  (At which point the fact that they are in one chassis is
almost irrelevant.)
I do not see how / why either line card would know, a priori, anything
about the other line card.
I do not see that the line cards can or should make the kind of decision
you refer to without the CE involvement.  It is the CEs job to set up the
processing of packets on the FE.  The CE needs to determine the topology,
using any means at its disposal.  Such means include configuration,
discovery protocols, and maybe other means.
I do not see why in the example you give there is any knowledge likely to
reside in each FE about the topology which connects it to other FEs.

Now, if you are describing a case where the chassis in question is
considered a single, two-port FE then we are talking about the capabilities
of an FE and the topology of the LFBs within the FE.  The knowledge that a
FE has two ports, and that an FE can accept a packet from one port and send
it out another port is indeed part of the FE knowledge, and should (and
will be) represented be the intra-FE information in FORCES.
The requirement is about topology between FEs.  That is why I object to it.

Yours,
Joel M. Halpern

---

From: Jamal Hadi Salim <[email-masked]>
Date: Thu, 18 Sep 2003 11:10:42 -0400

> are two FEs.  (At which point the fact that they are in one chassis is

nod.

> about the other line card.

They dont.

> you refer to without the CE involvement.
>
> It is the CEs job to set up the

nod again.

> reside in each FE about the topology which connects it to other FEs.

The knowledge would reside in the CE.
The CE teaches an FE about other FE "topologies". There are other ways
where the FE may get involved, but no need to go into them.

> The requirement is about topology between FEs.  That is why I object to it.

Ok, now i see why we are not syncing....
I think maybe the confusion (at least on my part) is what "topology"
means in this context. In the previous email i refered to what you
described as topology was overkill. I see "topology" as what you
describe as "intra-FE" information. I have a feeling this may also be
the impression of other people who posted their opinions earlier. At
least i hope thats what they meant ;-> If thats the case we should
clarify the text with "intra-FE" context. If thats not what was meant (
i dont remember the history behind this requirement - I think Todd added
this) then it is a showstopper.

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 18 Sep 2003 11:17:57 -0400

The requirements document in requirement 10 is clearly discussing inter-FE
connectivity / topology.  That is why I object to it.
Intra-FE topology (ie LFB topology) is clearly within scope and something
that must be communicated (and in some cases manipulated) by the protocol.
Yours,
Joel M. Halpern

---

From: Alex Audu <[email-masked]>
Date: Thu, 18 Sep 2003 12:42:29 -0500

Hello Joel,

I am trying to understand the reason for your objection to inter-FE topology.
Is it that this information is not useful, or is it that it is kept in the
wrong place?

Inter-FE topology in an NE is certainly useful. It lets us configure FEs that
cooperate in an NE to solve a particular problem by bringing more processing
power to bear on that problem.

Such topology can be dynamically configured according to the problem at
hand or service to be provided. CE can configure the topology, and hence
is aware of it. A scenario that may occur is the CE going out of service
momentarily.  CE should be able to query the FEs for this topology information

when it comes back in service.

Regards,
Alex.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 18 Sep 2003 16:15:33 -0400

I am not arguing that the inter-FE is not important.  It is vital.  Without
knowledge of this topology the CE can not do its job.

What I am saying is that the FE is not the component to look to for
assistance in getting this knowledge.  And the requirement mandates that
the protocol have mechanisms for having the FE provide this information.
This tends to lead to extra protocol mechanisms to solve a problem that
does not even need to be addressed.

Yours,
Joel M. Halpern

At 03:04 PM 9/18/2003 -0500, you wrote:
>It is clear that the inter-FE topology influences the beahvior of the NE.
>If that is the case, I think it makes sense for that topology to be queried
>so as to shed light on that behavior.
>
>This topology or connectivity information is soft state information. It could
>instruct FE-A to apply process X on packets ingressing on  port M and
>egress those packets on port N (onward to FE-B which in turn applies
>process Z before forwarding to the next router).
>
>Yes, CE keeps this configuration info for its records. But you have to provide
>a way for this information to be recovered just in case CE's record of that
>information is lost. I think this is important.
>
>Regards,
>Alex.

---

From: Alex Audu <[email-masked]>
Date: Thu, 18 Sep 2003 15:04:12 -0500

Regards,
Alex.

"Joel M. Halpern" wrote:

> There are multiple reasons for my objection.
>
> I start from the view that this information can almost never be accurately
> known by the FE.  If it is statically configured, it could have been better
> configured at the CE.  If it is known by a protocol, that is either a
> protocol the CE runs, or a protocol provided by an LFB, which should be
> configured and accessed (via FORCES) by the CE, not a special separate
> FORCES protocol mechanism.
>
> Note that information from the FE indicating that it is connected to a
> backplane in a specific chassis is not topology information, and is
> ptoentially useful.  I don't think we need special protocol mechanisms for
> that, but it is much more useful than an inter-FE topology information from
> the FEs.
>
> Remember that we are trying to support cases where as much of the
> intelligence as possible is in the CE.  neighbor discovery, and even
> nieghbor configuration, belongs with the intelligence.
>
> The CE is responsible for enabling the FEs to do their job.  It is
> absolutely true that the CE will need to know the inter-FE topology.  But
> that does not mean that there is a requirement on the FORCES protocol for
> FEs to be able to provide this information.
>
> While it is probably true that in some corner case the FE as an entity
> might have this information, it seems unreasonable to mandate additional
> complexity in the FORCES protocol to cover such a corner case.
> Note that with an LFB library and both FE and LFB attribute extensibility,
> it is possible for the FORCES system as a solution to represent such
> knowledge where it exists.  However, the requirement is much more demanding
> than that.
>
> Yours,
> Joel M. Halpern
>
> PS: I have an image of information going from an FE to a CE saying
>      "I have a serial port.  It is not connected to a multi-access converter,
>       and I think the entity on the other end is Harry."
>      The CE would be better off determining this itself rather than trying
> to determine what reliance to place on such configured information.
>
> At 12:42 PM 9/18/2003 -0500, you wrote:
> >Hello Joel,
> >

---

From: Weiming Wang <[email-masked]>
Date: Fri, 19 Sep 2003 09:19:02 +0800

I  get a little more understanding what Joel means. Maybe the focus point with the requirement

is the word "MUST". If it is a weaker word, things maybe better, since in many cases, neither 
CE nor FE can automatically discover the (inter-)FE
topology, therefore manual configuration is the last choice. In these cases, it is boring to ask so many  individual 
FEs to provide their topology information, with the information is in fact also manually keyed in in individual FEs or
FE managers. Instead, to let CE do it is easier.

If it is not a "MUST" requirement, the scenario may be like "CE query FE for topology info. If FE cannot provide, 
that's ok, CE will ask others(such as CE manager or administrator) to provide it. "

As a result, my opinion is the requirement is still necessary, but it should not be "MUST" one. 

Besides, I'm thinking (also reflected in GRMP protocol) if it is easier to understand FE topology processing by 
dividing the process into two scopes:
1) FE topology capability info collection; that is how FEs are physically interconnected. 
2) FE topology configuration(by CE); that is how CE would like to use the physical topology. It is not needed to be the same as physical 
topology, instead it can be a subset of the physical topology. This working topology can be  changed on the fly, within the scope of physical
scope. 

Another question raised by Joel: "If one is using a backplane with slots, or an ethernet switch, or ... then when one adds a
component to the system, all the other components must be reconfigured!!"  
I think this is not a problem. It's true the topology of other FEs will change, but it's not necessary to reconfigure it. 
It only needs to be informed of this change via some CE (broadcast) message, then  the individual FE can upgrade it.

Regards,
weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 18 Sep 2003 15:18:27 -0400

Yours,
Joel M. Halpern

---

From: Alex Audu <[email-masked]>
Date: Fri, 19 Sep 2003 10:26:19 -0500

I read that requirement differently.  Every FE object contains data which
stores its connectivity information with its neighbors. Null connectivity
will be valid in the case of a stand-alone FE. All the requirement says is
the FE must be able to report that information when queried. It doesn't
mandate any discovery mechanisms or any thing like that.

Regards,
Alex.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 20 Sep 2003 19:18:53 -0400

As judged by the protocols being brought forward, people are reading the
requirement as mandating special protocol mechanisms for reporting this
"connectivity".  As this information will almost never exist, I object
strongly to special mechanisms to support reporting such information.

Removing the requirement will
a) remove the incentive for special mechanisms
b) likely cause us to recognize that attributes about such information a
optional extensions, not core value.
Both of these are good results.

If there is anything we have learned in IETF protocol design it is that
extra mechanisms for rare cases are generally a bad idea.

Yours,
Joel M. Halpern

At 10:26 AM 9/19/2003 -0500, you wrote:
> > topology, therefore manual configuration is the last choice. In these
> cases, it is boring to ask so many  individual
> > FEs to provide their topology information, with the information is in
> fact also manually keyed in in individual FEs or
> > If it is not a "MUST" requirement, the scenario may be like "CE query
> FE for topology info. If FE cannot provide,
> > 2) FE topology configuration(by CE); that is how CE would like to use
> the physical topology. It is not needed to be the same as physical
> > topology, instead it can be a subset of the physical topology. This
> working topology can be  changed on the fly, within the scope of physical
> > scope.
> >
> > Another question raised by Joel: "If one is using a backplane with
> slots, or an ethernet switch, or ... then when one adds a
> > I think this is not a problem. It's true the topology of other FEs will
> change, but it's not necessary to reconfigure it.
> > It only needs to be informed of this change via some CE (broadcast)
> message, then  the individual FE can upgrade it.
> >
> > Regards,
> > weiming
> >