Subject: ForCES WG meeting minutes, IETF 63, Paris, Aug 1 2005.

From: Robert Haas <[email-masked]>
Date: Tue, 9 Aug 2005 13:54:55 +0200

Hi All,
Below are the minutes of the Aug 1st 2005 ForCES WG meeting in Paris.
Presentations are available at:
http://www.zurich.ibm.com/~rha/forces-ietf63-presentations.zip

Regards,
-Robert

IETF ForCES wg meeting, ietf63, Paris, Aug 1, 2005, 10h30-12h30.

Robert Haas and Alex Zinin chairing the meeting.

1) administratrivia (Robert Haas, 4 min)

Presentation: forces-ietf63-chairing-haas.ppt

Consider last call for protocol and model drafts before ietf-64.
Attendance/list is asked to participate in preparing the MIB draft(s)
and LFB library draft(s). Also, existing or in-progress implementations
should be reported to the list.

2) Flexinet presentation (Robert Haas, 6 min)

Presentation: forces-ietf63-flexinet-haas.ppt

Description of the basic and extendable function blocks in the Hitachi
modular router. Packets received by the NE are directed to one or more
other FEs for packet treatment, which requires metadata exchange among
FEs (future scope). Description of the ForCEG (ForCES CE Gateway) used
to gateway between services such as AAA Proxy and QoS and the FEs,
ensuring consistency and appropriate demultiplexing to those services.
Implementation status.

3) ForCES protocol updates presentation (Robert Haas, 33 min).

Presentation: forces-ietf63-protocol-updates-haas.ppt

Review of all technical issues in the tracker:
http://www.mip4.org/issues/tracker/forces/

Issue 6: DATARAW and PACKED-DATARAW. Necessary due to existence of
optional fields. Need to introduce to ILVs (Index-Lengh-Value). Question
by Alex Zinin to verify if the consensus was indeed agreed on the list.
Accept.

Issue 11: Heartbeat piggybacking used to skip sending heartbeat during
protocol activity (shows that both FE and CE are alive), need to add a
flag in the FE Protocol LFB. Accept.

Issue 12: Message obsoleting at TML. Not supported by most transports.
Reject.

Issue 22: Response formatting. Use of RESULT-TLV. Accept.

Joel comment: depending on the execution mode (do all, up to error),
each request in the config message can report its result, or a single
result can be reported if the execution is stopped at the first error.
Accept.

Issue 23: Common Header: no windowing at ForCES level. CE-FE is
master-slave relationship. Atomicity and batch flags remain in the
common header. Accept.

Issue 24: Abandon the CE LFB. Accept.

Issue 25: Association setup with FE ID left to 0 for CE to assign
dynamically does not cause issues with mcast IDs. Accept.

Issue 26: Teardown reason TLV. Accept.

Issue 28: Multiple LFB instances select (under the same LFB-select TLV).
Reject.

Joel comment: not needed at this time.

Jamal comment: corresponds to xcast. Can use multiple LFB-select TLVs
instead.

Issue 30: All event are subscribable. Accept.

Issue 31: special Notification Response Message (from CE to FE). Can use
ACK flag instead. Reject.

Issue 32: Packet redirect with two LFBs: sink and tap. Encapsulation of
redirected packet in a Redirect Message and TLVs. Static metadata
assignment, with metada transcoding. Accept.

Issue 33: Command pipelining is inherently permitted ny the protocol.
Correlators may be used to match responses with requests. Accept.

Issue 34: Header flags: text will be provided to set the precise meaning
of the ACK, priority flags. Throttle flag will be dropped. Pending.

Issue 36: FE/CE failover. CE failover policy determines if the FE should
continue or stop forwarding packets after an association is lost. FE
failover policy determines how the FE should behave after it regains an
association, currently: start from default state again (no fast
recovery). Accept.

Issue 37: Remove requirement for the FE Protocol LFB. Instead, global
defaults must be used. Heartbeat at 300 ms, with a Association Expiry
timer of 900 ms. Accept.

Issue 40: BLOCK operations. May include in a future release of the
protocol. Reject.

Issue 47: PL sequence number and correlator. Convert the sequence number
and the correlator into a 64-bit correlator. Correlator used in Config
and Query messages. Must be set to 0 for Notifications.

Joel comment: use a 64-bit correlator field, left to the CE to split it
how it wants.

Accept the introduction of correlator. Accept the 64-bit correlator to
replace sequence number and correlator.

Issue 52: Association Setup. Unsolicited information can be transmitted
in the Association Setup message from the FE, using GET-RESPONSE. CE can
set attributes with the Association Setup Response using SET. Accept.

Issue 55: Recovery of transactions in intermediate state after
association comes back up. Relates to issue 36. Reject.

Issue 56: Association Setup Result TLV. Accept.

Issue 57: Operation types.

Joel comment: No Event Notification Response message needed.

Avri comment: some of the operations are historically here still.

Joel coment: Need to have set and delete. The rest of the operations
(add, update, replace, del all, cancel are not needed).

Accept.

4) Model updates (Joel Halpern 11 min). No slides.

Comments about the draft from the working group are awaited. LFB 
definitions are needed, contributions from the working group are awaited.

Three noticeable changes:

a) addition of properties information (information about the elements in
the LFB): there is a flag in the protocol to reference properties
instead of data itself. Model includes property information such as is
the element readable/writable, array information (how many elements
exist, highest/lowest subscript). Properties allow the capabilities
information to be linked to the elements themselves.

b) aliases: for a given LFB instance to be able to reference information
in another LFB instance, so that data does not have to be duplicated.
For instance, the ARP LFB needs the MAC address field from the 
Ethernet-port LFB. Properties information tell which field of which LFB 
is referenced using PATH notation. Most LFBs won’t need aliases.

c) events: event declaration added to the model. Declares an ID for the
path to the base of all events in an LFB, and all events declared within
that. Allows a subscribe-to-all events. Event declaration contains:
event ID, target variable (to indicate which element in the LFB is
causing the event), condition, information to report (value that change,
subscript, etc). Base on a nested XML structure with field names or
variable names that can be used to represent array subscripts. Event
properties contain a threshold value, if the event is subscribed, and an
hysterisis (FE not required to support hysterisis, but important to
suppress oscillations and DoS’ing the CE).

Key and Keyref parts of the XML should be validated.

No questions.

5) LFB examples (Jamal Hadi Salim, 25 min).

Presentation: forces-ietf63-LFB-examples-jamal.pdf

Netdevice LFB (aka Port LFB). Abstracts layer 1 processing. Example of
Netdevice with ingress/egress parts, each with wire/upstream/downstream
LFBs ports.

Joel question: why does ingress has an upstream LFB port ?

Jamal: Used to route back packets to other ports, applicable if
netdevices are virtual ports.

Joel question: need of a loopback device in the model ?

Jamal: loopback is a bad example. Tunnel device that validate headers.

Description of the Netdevice LFB: capabilities, topology constraints,
MIB-derived info, etc. More specific description of the attributes of
the Ethernet Netdevice.

Robert question, Joel answer: events are created whenever an LFB is
instantiated. So the CE is aware of LFBs that are autonomously created
by the FE.

Sample topologies of Netdevices, with Ethernet, bridge, ipv4/v6,
forwarding LFBs.

XML model for the Netdevice LFB with capabilities, events, and
attributes. The LFB contains a table of netdevices. Existing
implementation of Ethernet netdevices.

Joel comment: change name from IPv4 LFB to IPv4 validation LFB (TTL
decrement, checksum, header validation. Only output is either valid or
non-valid) LFB.

Joel comment: The IPv4 LFB should not own the locally-terminated IPv4
addresses (the router’s IPv4 addresses). IPv4 addresses should not be
attributes of the IPv4 validation LFB. They should be handled by a
separate classifier LFB (model rule is to not fold a classifier into the
IPv4 validation LFB, use more LFBs instead). On the other hand, the
netdevice may include the classification based on the Ethertype, as it
is a desperately common case (alternatively, the netdevice may pull out
the Ethertype as metadata and give it to a classifier LFB). Use “thin” LFBs.

Short overview of the IPv6 LFB.

6) Updates to TCP TML (Jon Maloy, 19 min)

Presentation: forces-ietf63-tcp-tml-maloy.ppt

Jamal, Joel comment: multicast model for TCP is multi-unicast.

TCP no longer specified for the data channel, issues with transporting
TCP fragments over TCP. Possibility of using DCCP or GRE tunnels.

PL layer knows only FE and CE IDs, does not see channel descriptors anymore.

In the High Availability case, FE TML not responsible to decide which CE
to open channels to.

Complete removal of TML messaging and modifications to channel setup.

Multicast model: TML maintains mapping between mcast IDs and channels.

Description of the TML Service Interface used by the PL.

Robert question: Is the FE Protocol LFB used to set in the FE the mcast
IDs the FE belongs to ? Jon: a specific PL-message MC Grp Join Req is to
be used.

Examples of unicast channel setup, and mcast group join and leave.

Robert question: McId shown on slide 13 is a regular PL mcast ID ?

Jamal: This document contains more PL-TML APIs than before. Was the
PL-TML API not supposed to be published in a separate document,
according to consensus from previous ietf meeting ?

Joel: there is no other wg document now that contains this.

Jon: the PL-TML API can be moved to a separate document.

Open issues: which protocol to select for data channel ? Security:
IPSec, and/or TLS ?

Question by Toshiaki Suzuki: to ensure HA, one FE may be connected to
two CEs.

Joel: currently out of the scope of the charter to support multiple CEs
for load-balancing reasons.

Robert: mcast IDs at PL level can be used for such purposes. Failover to
a CE in standby mode is in the scope of the charter.

Jon: There is a Service Availability Forum model to make HA clusters.

7) TIPC TML (Jon Maloy, 14 min)

Presentation: forces-ietf63-tipc-tml-maloy.ppt

TIPC is a lightweight protocol, potentially better performance than TCP.
TIPC sses the same addressing as PL IDs.

A reliable connection for control and a best effort connection for data.

Description of the address mapping for unicast and mcast.

No security, only for closed LANs. Congestion controlled.

No Nagle (immediate delivery).

TIPC support for failover over parallel links.

4 priorities only in TIPC. Need to map from the up to 8 priorities.

Joel question: in case of congestion on the data channel, messages are
dropped by TIPC.

-- 
Dr. Robert R. Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-44-724-8698  fax +41-44-724-8578  http://www.zurich.ibm.com/~rha

---

From: Robert Haas <[email-masked]>
Date: Tue, 16 Aug 2005 17:58:09 +0200

Hi All,
I have updated the tracker for all issues discussed and resolved at the 
Paris wg meeting. The status remains "Pending" for most of them as long 
as the text has not been put in the draft. See below for the details.

Regards,
-Robert

> Issue 28: Multiple LFB instances select (under the same LFB-select TLV).

Updated in the tracker. Text in the draft must be updated (or removed).

> assignment, with metada transcoding. Accept.

Updated in the tracker. Remains "pending" until changes are reflected in 
the draft.

> Correlators may be used to match responses with requests. Accept.
Updated in the tracker. I proposed text for that section.

> of the ACK, priority flags. Throttle flag will be dropped. Pending.

Updated in the tracker.

> association, currently: start from default state again (no fast

Updated in tracker, text proposed. Must be added to the draft.

> defaults must be used. Heartbeat at 300 ms, with a Association Expiry

Updated in tracker. Text must be removed/changed in the draft.

> Issue 40: BLOCK operations. May include in a future release of the

Updated in tracker. Need to remove references to BLOCK operation tin the 
draft.

> and Query messages. Must be set to 0 for Notifications.

Updated in tracker. Need to make changes in the draft.

> set attributes with the Association Setup Response using SET. Accept.

Updated in tracker. Need text for the draft.

> association comes back up. Relates to issue 36. Reject.

Updated in tracker.

> Issue 56: Association Setup Result TLV. Accept.

Updated in tracker. Need to copy proposed text in the draft.

> Issue 57: Operation types.

Updated in tracker. Need to copy proposed text in the draft.

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 17 Aug 2005 00:15:04 +0800

Robert,

very clear description. I will update the text accordingly.

Thanks very much.

Weiming

----- Original Message -----
From: "Robert Haas" <[email-masked]>

Hi All,

Regards,
-Robert

Updated in the tracker.

Updated in tracker.

--
Dr. Robert R. Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-44-724-8698  fax +41-44-724-8578
http://www.zurich.ibm.com/~rha

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 17 Aug 2005 22:30:16 +0800

----- Original Message -----

[Weiming] Most of the text has already been updated in the current
version.  I just suggest the following minor changes more to reflect the
transcoding necessity. After this, I think the issue can be completely
closed.

<Original>
Meta Data ILV:
      This is an Identifier-Length-Value format that is used to describe
      one meta data.  The ILV has the format as:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Meta Data ID                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Length                                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Meta Data Value                        |
   .                                                               .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Where, Meta Data ID is an identifier for the meta data, which is
      usually defined by FE-Model[FE-MODEL].
 </Original>

<Supposed Change>
Meta Data ILV:
      This is an Identifier-Length-Value format that is used to describe
      one meta data.  The ILV has the format as:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Meta Data ID                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Length                                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Meta Data Value                        |
   .                                                               .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Where, Meta Data ID is an identifier for the meta data, which is
      defined by FE-Model[FE-MODEL]. This actually implies
a Meta Data ID is statically assigned by FE-Model.  This also implies a
Meta Data
ID transcoding mechanism may be necessary if a metadata traverses
several LFBs
while these LFBs define the metadata with different Meta Data IDs.

</Supposed Change>

Comments please.

thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 17 Aug 2005 22:06:29 +0800

----- Original Message -----

[Weiming] The text has already been updated in the previous update
version by weiming.  This issue can be completely closed.

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 17 Aug 2005 22:48:27 +0800

Robert,

----- Original Message -----

 [Weiming] I see the current Hormuzd's  version has included the
correlator in some way, but not sure if it's all. I'm now out of town
and my lap-top only reaches latest two weeks list posts, making me hard
to locate your proposed text. Could you try to send the proposed text
again? I then can compare to see if anything is included.

Thanks a lot.
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 18 Aug 2005 22:10:43 +0800

Updated in the tracker.

[Weiming] Hormuzd has provided text for priority flags. I suppose to
provide text on ACK flags  as below:

       - ACK indicator(2 bits)
         The ACK indicator flag is used by a protocol message sender to
indicate the message receiver whether a response message is required
back by the sender.

               'NoACK' (00) - to indicate the message receiver not to
send any response message back to this message sender.

               'SuccessACK'(01) - to inidcate the message receiver to
send a response message back only when the message has been successfully
processed by the receiver.

               'UnsuccessACK'(10) - to indicate the message receiver to
send a response message back only when the message has been found unable
to be successfully processed. In other words, if the message can be
processed successfully, the sender will not expect any response from the
receiver.

               'ACKAll' (11) - to indicate the message receiver to send
a response message back in any case.

Note that the ACK indicator can only take effect  after  the message has
been successfully received, while successful delivery of a message is
always a resposibility of protocol TML.

Also note that some messages defined in the following sections may have
default definitions for requiring response messages. In this case, the
ACK indicator will also not take effect. We summarize these definitions
as below:
.  Association Message will always expect a response message.
. Teardown Message, Redirect Message, and Heartbeat Message will always
not expect a response message
. Any kind of response messages will never expect further responses to
these messages.

Any comments?

p.s., Jamal, Could you provide some text on the Batch flag and the
Atomicity flag? They are still blank in the draft. Another question is,
I'm not sure if the flags description is precise enough or not, because
the bits location has not been provided.

Thanks,
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 18 Aug 2005 10:33:45 -0400

I think that this needs some tuning.
A single "message" with a single set of flags can contain multiple operations.
We have in other places in the document stated that there are three 
different modes of operation:
1) All-or-none.  In this case either all the operations succeed, or none 
are performed.  It is clear how that relates to the ACK flags you describe.
2) Operate-until-error.  In this case, all operations are performed 
sequentially until one fails.  If they all succeed, then the relationship 
with the flags is clear.  But if some succeed and then one fails, should 
that result in an ACK if the setting was "SuccessACK"?  Should it result in 
an ACK if the settings was "UnsuccessACK"?
3) Perform-individually.  In this case, each operation is attempted, 
whether the earlier operations succeeded or failed.  Given that there can 
be multiple success and multiple failures, what should happen if the 
setting is either "SuccessACK" or "UnsuccessACK"?

I believe we can pick and describe behaviors.  But we need to do so.

Yours,
Joel M. Halpern

---

From: Robert Haas <[email-masked]>
Date: Thu, 18 Aug 2005 16:39:40 +0200

Weiming,
Please take a look at the tracker with my proposed text (copied below):
http://www.mip4.org/issues/tracker/forces/issue33

Consensus of the Paris wg meeting is that pipelining is inherently supported 
by the protocol. Correlators may be used to match responses with requests.

I propose to put this sentence in the "Command Pipelining" section:

"The protocol allows any number of messages to be issued by the CE before the 
corresponding acknowledgments (if requested) have been returned by the FE. 
Hence pipelining is inherently supported by the protocol. Matching responses 
with requests messages can be done using the correlator field in the message 
header."

Regards,
-Robert

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 18 Aug 2005 23:17:40 +0800

Hi,

The section "Command Pipelining" has been added in the draft with the
text proposed by Robert as below.

This issue 33 can be closed.

Thanks,
Weiming

----- Original Message -----

Weiming,

Consensus of the Paris wg meeting is that pipelining is inherently
supported
by the protocol. Correlators may be used to match responses with
requests.

"The protocol allows any number of messages to be issued by the CE
before the
corresponding acknowledgments (if requested) have been returned by the
FE.
Hence pipelining is inherently supported by the protocol. Matching
responses
with requests messages can be done using the correlator field in the
message
header."

Regards,
-Robert

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 18 Aug 2005 23:57:42 +0800

uniformed  all "FE LFB" to"FE Obejct LFB".

Weiming

----- Original Message -----
From: "Robert Haas" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, August 16, 2005 11:58 PM
Subject: Re: ForCES WG meeting minutes, IETF 63, Paris, Aug 1 2005.

Hi All,

Regards,
-Robert

Updated in the tracker.

Updated in tracker.

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 00:20:43 +0800

[Weiming] I think Hormuzd has added some in his revision as below:

      *  FE restart policy - This specifies the behavior of the FE
         during a restart.  For example, this would specify if the FE
         should start from a default state after a CE failure in the NE.

      *  CE failover policy - This specifies the behavior of the FE
         during a CE failure and restart time interval.  For example,
         this would specify if the FE should continue running or stop
         operation during a CE failure in the NE.

I'm based on you two and propose the following:

     *  FE restart policy - This specifies the behavior of the FE
         during a restart.  For example, this would specify if the FE
         should start from scratch or from a default state after a CE
failure in the NE. Default value for this attribute should be to let the
FE to start from scratch.

      *  CE failover policy - This specifies the behavior of the FE
         during a CE failure and restart time interval.  For example,
         this would specify if the FE should continue running or stop
         operation during a CE failure in the NE. Default value for this
attribute should be to let the FE to continue running.

Comments please.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 00:30:04 +0800

[Weiming] The heartbeat default and Association Expiry default have been
added by Hormuzd, but I think this issue has not been totally solved.
Description of some of the attributes need to be added, and the defaults
also.  I also suggest this issue combines FE Object attributes also.

I hope this issue can be discussed more on the next call or someone can
provide more text on this.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 00:34:40 +0800

Updated in tracker. Need to remove references to BLOCK operation tin the
draft.
[Weiming] checked the file and assured all BLOCK operation has been
removed. This issue can be closed.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 00:48:37 +0800

Hi

The text has been updated according to the proposed change.

This issue can be closed.

Thanks,
Weiming

----- Original Message -----
From: "Wang,Weiming" <[email-masked]>

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 11:37:04 +0800

Joel,

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>

> different modes of operation:
First of all, we should note that ACK flag does not interfere into any
operations. Then, we noted that ACK flag also does not affect the
response format of operations. The only effect of ACK flag is to choose
a response at the time.  I see the question you raised as that we may
need some definitions and specifications in the operations regarding ACK
flag meaning, whereas the ACK flag still keep its clearness in meaning.

> that result in an ACK if the setting was "SuccessACK"?
It should reasonably be no response then.

>Should it result in
It should reasonably be of a response then.

I suggest to add a text in operation-until-error description that
"ACKAll" be recommended to use for such operation.

> setting is either "SuccessACK" or "UnsuccessACK"?

ACK flag in the header is defined to be overall of operations in the
message, therefore, one failure can be reasonably taken as unsucces of
the whole message operation.

> I believe we can pick and describe behaviors.  But we need to do so.
By your above comments, I just feel that a 'Success' or 'Unsuccess'
definition should be added to the text I proposed to make it clearer.

Thank you very much.

Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 13:07:58 +0800

Robert,

I don't think to remove the Sequence Number is the consensus we reached.
I have for several times noted that SN will be used in several cases.
Note that SN will be the only key to order back messages for messages
transported over unreliable TMLs.  I can currently think of that  the
redirect message will definitely need the SN to order back redirect
data. I also think of the future, when SN may be used by CE and FE to
syncronize the messages. It's a little too early to deliminate the
necessity, I'm  afraid.

How others think?

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 13:12:17 +0800

Hormuzd,

Could you provide an updated text regarding this issue to let me put in?
Then we can close it.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 19 Aug 2005 13:26:00 +0800

[Weiming] Hormuzd has added the error codes in the association setup
response message in the revision. I'm not sure if this issue has also
related to more than this. If not, then this issue can be closed.

Thank,
Weiming

---

From: Robert Haas <[email-masked]>
Date: Mon, 22 Aug 2005 17:40:41 +0200

Weiming,

>always a resposibility of protocol TML.
I don't understand the sentence above. Did we mean instead: "Note that 
the ACK indicator can only take effect after the message has been 
successfully PROCESSED, [..] " ?
And what do we mean exactly by "take effect" ?

> the whole message operation.

I agree: anything else than a complete success is a failure, i.e., for 
the all-or-none, operate-until-error, and perform-individually schemes, 
if any single operation in the message fails, then it will result in a 
response only if the UnsuccessACK is set (or AckAll obviously too).

Regards,

---

From: Robert Haas <[email-masked]>
Date: Mon, 22 Aug 2005 18:19:07 +0200

Weiming,
Well, this was the consensus among the participants at the meeting, at 
least. But let's discuss this again:

First, note that the protocol as we designed it until now _relies_ on a 
reliable TML.
For instance, if the response to a operation was lost over an unreliable 
TML, the CE would reissue the operation. The FE would have no way to 
realize that this second operation is actually a repetition (remember 
that the correlator field is completely opaque to the FE). Depending on 
the kind of operation, this could be fatal to the FE.

Now, if you want to use an unreliable TML only for certain message types 
(such as event notifications or redirected packets, nothing fatal can 
happen if messages are lost), then we need to decide whether we want to 
guarantee ordering at the PL level at all. This is debatable.

Regards,
-Robert

Wang,Weiming wrote:

>>Issue 47: PL sequence number and correlator. Convert the sequence
>>    
>>
>number
>  
>

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 01:57:40 +0800

Hi,

the update related to the issue is as below:

>Consensus from Paris wg meeting on the messages and operations required:
>(need to be put in the draft)

>CONFIGURATION message:
> Set (also used for event subscription), Delete
>CONFIGURATION-RESPONSE message:
> Same as above with *-response
[Weming] has been updated and has made comparatively large scale text modifying to this section for text consistency with other sections, see [editorial 4] post.

QUERY message:
 Get
QUERY-RESPONSE message:
 Get-response
[Weiming]Has been updated and made minute modification to whole section

EVENT-NOTIFICATION message:
 Report
[Weiming]Updated. Is going to completely remove the section of event notification response?

ASSOCIATION-SETUP message:
 Get-response (optional, used to advertise fields from the FE LFB 
and/or FE Protocol LFB in an unsolicited manner)

[Weiming]haven't made changes. Questions:
Do we really need an operation in the association setup message? It will make flooding attack more easy, I think. Why don't we let CE later to query then?

ASSOCIATION-SETUP-RESPONSE message:
 Set (optional, only to set fields in the FE LFB and/or FE Protocol LFB)
[Weiming]Haven't made changes. Questions:
Does this really mean we allow a SET operation that never be able to get SET response? Why don't we use config message to SET then?

PACKET REDIR, HEARTBEAT do not have operations.
[Weiming]has made minute text modification to these sections.

Thanks,
Weiming

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 02:15:42 +0800

Hi,

Attached is the supposed modifying to Section 6.5 for the purpose of consistency. Pls send your comments.

Thanks,
Weiming

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 02:33:53 +0800

Robert,

----- Original Message ----- 

[Weiming] maybe theres some languge problem with me, but what I really want to say is:

the ACK indicator can play its role only under the condition that the message with this indicator has been successfully received, though it doesn't have to wait for the message to be successfully processed(executed).  

To mention this, I just want to express the idea that except setting to "ACKall", the ACK indicator is of no use to inhance transport layer reliability. 

Let me modify this sentence.

Thanks,
Weiming

---

From: Robert Haas <[email-masked]>
Date: Mon, 22 Aug 2005 21:22:34 +0200

Weiming,
The idea of the get-response and set operations in the Association Setup 
and Response messages, respectively, was mainly to allow unsolicited 
information to be advertised by the FE. I agree with you that the set 
operation is not very fortunate as it means that a set-reponse might be 
needed, this time in a CONFIG message type.
Therefore I propose that we only allow the get-response (or can be 
called anything else, such as SHOW) in the Association Setup messages, 
and nothing in the Response.

Regards,
-Robert

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 11:01:25 +0800

Robert,

----- Original Message ----- 

Weiming,
[Weiming] I did not follow the discussion well before. Maybe there are other reasons, but I think why  we do not set the association setup operation just as REPORT?

Thanks,
Weiming

Regards,
-Robert

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 11:38:46 +0800

----- Original Message ----- 
From: "Weiming Wang" <[email-masked]>
 
> [Weiming]Updated. Is going to completely remove the section of event notification response?
There is a more question about event notification message. Have we reached any consensus that we don't need any event notification sent by CE? If we allow CE events, there will be problem with the message representation, because we do not have any PATH-DATA defined for CE after we have decided that we abandoned CE LFB.

Comments please!

Thanks a lot.
Weiming

---

From: Weiming Wang <[email-masked]>
Date: Tue, 23 Aug 2005 13:20:35 +0800

Hi,

Here is the supposed update for event notification message.

Thanks,
Weiming

6.7  Event Notification Message

   Event Nitification Message is used by FE to asynchronously notify CE
   of events happend in the FE.

   All events happened in FE are subscribable by CE.  A config message
   is used by CE to subscribe/unsubscribe for an event in FE.

   As usual, an Event Notification Message is composed of a common
   header and a message body that consists of one or more TLV data
   format.  Detailed description of the message is as below.

   Message Transfer Direction:
      FE to CE

   Message Header:
      The Message Type in the message header is set to
      MessageType = 'EventNotification'.  Because we do not expect any
      response for event notification, the ACK flag in the message
      header SHOULD be reasonably set to 'NoACK'.  If the ACK flag is
      set to any other values, the meaning of the flag will then be
      ignored.

   Message Body:
      The message body for an event notification message consists of (at
      least) one or more than one TLVs that describe the notified
      events.  The TLV is also a LFBselect TLV (with the TLV
      Type="LFBselect"), and a generic format definition of which has
      been given in Section 6.1.1.  Detailed data format of a LFBselect
      TLV for event notification message is as below:

     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |        Type = LFBselect       |               Length          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                          LFB Class ID                         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        LFB Instance ID                        |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        Operation TLV                          |
    .                                                               .
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ~                           ...                                 ~
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        Operation TLV                          |
    .                                                               .
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                                  Figure 24

   Operation TLV:
      This is an operation TLV that describes the event to be notified,
      as follows:
 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Type = REPORT              |               Length          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    PATH-DATA-TLV for REPORT                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                                  Figure 25

   Type:
      Only one operation type is defined for the event notification
      message:

             Type = "REPORT" --- this type of operation is for FE to
             report something to CE.

   PATH-DATA-TLV for REPORT:
      This is generically a PATH-DATA-TLV format that has been defined
      in "Protocol Grammar" section(Section 6.1) in the PATH-DATA BNF
      definition.  The restriction on the use of PATH-DATA-TLV for
      REPORT operation is it MAY contain DATARAW-TLV(s) but SHALL NOT
      contain any RESULT-TLV in the data format.

   To better understand the above PDU format, we show a tree structure
   for the format as below:

   main hdr (type = Event Notification)
        |
        |
        +--- T = LFBselect
        |        |
        |        +-- LFBCLASSID = target LFB class
        |        |
        |        |
        |        +-- LFBInstance = target LFB instance
        |        |
        |        |
        |        +-- T = operation { REPORT }
        |        |   |
        |        |   +--  // one or more path targets
        |        |        // associated with DATARAW-TLV(s)
        |        +-- T = operation { REPORT }
        |        |   |
        |        |   +--  // one or more path targets
        |        |        // associated with DATARAW-TLV(s)

                                 Figure 26

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 23 Aug 2005 10:52:42 -0400

The text below is an excerpt from a proposed methodology for a particular 
PL message.
It is about event notification messages, and the fact that no responses are 
expected to them.  (They are transmitted reliably, and the FE which sends 
them has no use for an ACK.

However, if we are going to take this approach, we render the ack flags 
much more complex.  We should either define the ack flags globally and use 
them everywhere, or we should not have them in the common message header.
If we choose to keep them, then a message receiver should be prepared to 
generate an ACK for ANY message it receives, if the flags tell it to do so.

Yours,
Joel M. Halpern

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:19 -0400

I think it is an issue of semantics.
If we say we have a 64 bit field to play with and call it correlator it
doesnt matter how that field is used by the CE. The important detail in
my opinion is that the 64 bit is opaque to the FE and is returned as is
to the CE. So my suggestion is to call it a corellator and have
perhaps(?) some text around it to say how its usage by the CE could be
implemented.
In the case of reliability of messages, I think thats the responsibility
is one that is owned by the TML (and the TML API describes how to
indicate a message from the PL that needs reliability).

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:03 -0400

Weiming,

Comment below ...

> ----- Original Message -----
[..]
> a Meta Data ID is statically assigned by FE-Model.  

This is not realistic. I think that way after the FE-Model becomes RFCs
metadata will continue to be defined.

> This also implies a Meta Data ID transcoding mechanism may be necessary 
> if a metadata traverses several LFBs

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:22 -0400

> [Weiming]Updated. Is going to completely remove the section of event notification 

Lets review after you complete your update.

> Do we really need an operation in the association setup message? It will make 
> flooding attack more easy, I think. Why don't we let CE later to query then?
> 

I think making it optional is fine. Optionally it can be used to cut
down on the number of messages issued.

> Does this really mean we allow a SET operation that never be able to get SET 
> response? Why don't we use config message to SET then?

I think the challenge in the naming convention as we discussed in the
meeting is that Config goes from CE->FE and response goes back. 
However you do bring up a good point: the CE would need to know if the
SET was successful or not. A GET following the SET should be sufficient.
Which actually invalidates the need to SET in the ASSOCIATION-SETUP
message since eventually the number of messages is going to be
equivalent. Thoughts?

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:16 -0400

> flag meaning, whereas the ACK flag still keep its clearness in meaning.

Could we specify that the ACK flags override the All-or-none,
Operate-until-error etc?

> > >Atomicity flag? 

Why dont i wait until you are done since i think youve done a lot of
cleanup and this may be related to the execution text. i.e i want to
correlate with the rest of the text. Maybe we can open an issue in the
tracker so it is not forgotten.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:31 -0400

On Tue, 2005-23-08 at 11:38 +0800, Weiming Wang wrote:
> There is a more question about event notification message. Have we reached any consensus 
> that we don't need any event notification sent by CE? If we allow CE events, there will 
> be problem with the message representation, because we do not have any PATH-DATA defined 
> for CE after we have decided that we abandoned CE LFB.
> 

I think we decided IIRC that messages from CE to FE will be of type
CONFIG? Events from the CE to FE could be masquareded as CONFIG, no?

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:34 -0400

>    Event Nitification Message is used by FE to asynchronously notify CE
           ^^^^^^^^^^^  typo

>    of events happend in the FE.
               ^^^^^^^ probably rewrite as "of events that happen in the FE"

>    All events happened in FE are subscribable by CE.  A config message
                ^^^^^^^ "happen"
>    is used by CE to subscribe/unsubscribe for an event in FE.

Perhaps mention that to subscribe to an event is usually by specifying
to the path of such an event as described by FE-Model and defined by
specific LFB.

>       set to any other values, the meaning of the flag will then be

I think once you suggest a SHOULD to use NoACK then the rest is a
SHOULD;
i.e if you set an ACK then you expect a response back even if it is 
an event.

>       TLV for event notification message is as below:

My comments as before in the config. The Operation and LFBSelect should
be defined elsewhere and just used in these messages. Clearly you are
repeating this text and diagrams relative to the config message.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 23 Aug 2005 10:29:25 -0400

Hello Weiming,

> Attached is the supposed modifying to Section 6.5 for the purpose of consistency. 

Comments attached.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 19:51:52 +0800

Hi,

The draft has been updated in Section 6.2.1(FE Protocol LFB) according to
the text proposed below.

The issue 36 can be closed!

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 20:57:00 +0800

Hi,

I modified the text as below:

<Supposed Update>
       - ACK indicator(2 bits)
         The ACK indicator flag is used by a protocol message sender to
indicate the message receiver whether a response message is required
back by the sender. The values are defined as below:

Note that in above definitions, a success to process a message actually
means the message is completely processed(excuted) without any failure. This
also means anything else than a complete success is a failure and results in
an unsuccess of the message processing. As a result, for the execution
modes(defined in Section 3.3.1.1) like execute-all-or-none,
execute-until-failure, and continue-execute-on-failure, if any single
operation in the message fails, then it will result in a response only if
the ACK indicator is set to 'UnsuccessACK' or to 'ACKAll'.

Also note that some messages defined in the followed sections may have
default requirements for responses. In these cases, the
ACK indicators will be ignored by message receiver. We summarize these
defaut requirements
as below:
.  Association Message will always expect a response.
. Teardown Message, Redirect Message, and Heartbeat Message will always
never expect responses.
. Any kind of response messages will never expect further responses.
</Supposed Update>

I'l update this soon if theres no more comments, and then the issue 34 can
be closed.

Jamal, pls see more reply inline below.

Thanks,
Weiming

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, August 23, 2005 10:29 PM
Subject: Re: [issue 34] Header flags

> Could we specify that the ACK flags override the All-or-none,
I still prefer to define what 'Success' and 'Unsuccess' for ACK indicator
mean. If we say override as above, we then also need to define what
'override' actually mean.
> tracker so it is not forgotten.
That's fine. I don't think we have to open a speccial issue for this. The
blank is obvious and I think you can notice it easily. I'l also remember
it:)

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 24 Aug 2005 10:58:53 -0400

Weiming,
Dont close it yet since I am not sure if i commented on this or not:
a) Default behaviors for the two restarts are contradictory. If the
Default FE restart behavior is to start from scratch, that implies an
element of trust involved i.e the FE doesnt trust the state it has
during the disappearance of the CE. OTOH, the CE restart default seems
to say the FE should continue to use the policies it has on
disappearance of the CE which implies it trusts the state it has as
being valid ;-> Is it fair to say we need to choose similar default
policies?
b) Should we enumerate all possibilities in the text for the two restart
policies?

cheers,
jamal

[..]

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 21:54:49 +0800

Jamal,

> metadata will continue to be defined.
How about adding a mention as 'by FE-Model and LFB library'? Do we have
other documents related to the definition?

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 22:09:59 +0800

Joel,

Pls allow me to change the titile to issue 34.

I agree with you some what. I also noticed that actualy the ACK flag is only
useful for Config message after we removed event response function.

The question is, if we don't put it in the common header, as a global flag
for this message, where can we put then and how we can make it specifically
for config message only?

I thought of puting the flag in an LFBselect TLV, the operation TLV,  (or
even in a Path flag). In this way, it makes the flag only restricted to the
LFB selector, the operation,  (or even the path).  It means some LFBselect
or operations inside wants response, while others may not. I didn't think it
further.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 22:57:47 +0800

Jamal,

Thanks for the intensive review.

Some have updated accordingly and some replied inline.

Thanks,
Weiming

Hormzud, could you check inline also?

----- Original Message -----
> i.e if you set an ACK then you expect a response back even if it is
Can I understand in this case, SHOULD means it's not an error, while if we
use MUST, it becomes an error.
> repeating this text and diagrams relative to the config message.
Yes, I noticed this. Just not very sure where should it be put, in the BNF,
or using another section? After we decide this, I can update the text
actually.
Hormuzd, could you give a reply on this?

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 24 Aug 2005 11:10:36 -0400

On Wed, 2005-24-08 at 20:57 +0800, Wang,Weiming wrote:

>                'UnsuccessACK'(10) - 

Do we wanna call this a FailureACK instead?

> 'override' actually mean.

My comment was mostly to Joel's concern in that there are contradictory
statements i.e:
for example, consider when you say "Operate-until-error" but then you
set a flag to say "noACK":
 
statement1) "Operate-until-error" says to the receiver "when the error
occurs stop and let me know"
statement2) "noACK" says "dont bother telling me anything".

In other words we have ambiguity between the two statements. Which one
should the receiver belive?;->
Typically conflict resolution requeires to specify a rule of which among
X possible statement wins/overrides. This could be done in the
text. Hope that makes sense.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 24 Aug 2005 11:43:13 -0400

> other documents related to the definition?

LFBs will continue to be defined in the future even post LFB Library
document; for example if Forces is still around in 10 years, someone
will define an LFB and its associated metadata. 
So i would suggest just "A metaid is staticaly assigned by the LFB
definitions" 

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 24 Aug 2005 11:52:38 -0400

> use MUST, it becomes an error.

SHOULD is a recommendation. Ok, so let me change a little what i said.
If you say "the header SHOULD be reasonably set to 'NoACK'" then a
response MUST not be issued to such a message.
If you say "SHOULD set the ACK flag" then a response MUST be provided to
the message.

> or using another section? After we decide this, I can update the text

I think another subsection of the message description - maybe just
before individual ones are done, this would be the first one.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 25 Aug 2005 00:05:15 +0800

Jamal, Hormuzd, and all,

Based on this Jamal's and last Robert's comments, can we reach an agreement
as below:

1. Association-setup message with REPORT operation TLV.
2. Association-setup response message without operation followed, but with a
TLV followed to return result only(note not a RESULT-TLV in PATH-DATA). We
can name the TLV as something like ASRresult.
3. Association-teardown message with a TLV to show reason, as TEAreason
(note we have already had the TLV)

In this way, at the message body first level, we have three TLVs:
LFBselect
ASRresult
TEAreason

thanks,
Weiming

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, August 23, 2005 10:29 PM
Subject: Re: [Issue 57] Inconsistent operation types

> > [Weiming]Updated. Is going to completely remove the section of event
notification
> > response?

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 23:31:10 +0800

----- Original Message -----

> Do we wanna call this a FailureACK instead?
OK, I also hope to change ACKAll to AlwaysACK.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 24 Aug 2005 23:53:13 +0800

Jamal, and Hormzud, etc,

I also strongly tend to rewrite the message sections with the idea to strip
off the LFBselect TLV off and put it in a Section 6.1.1 or elsewhere with a
special section, but I still want to keep operation TLV description in every
message text, for it does differ from each other regarding the Type and the
Path-data-TLV description.

Can we decide it, Hormuzd? then I can update quickly.

Thanks,
Weiming

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, August 23, 2005 10:29 PM
Subject: Re: [Edtorial 4] Configure Message (Section 6.5) modifying, also
related to issue 57

> > Attached is the supposed modifying to Section 6.5 for the purpose of
consistency.
> >Pls send your comments.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 24 Aug 2005 16:51:09 -0400

Your proposed clarification of the interaction of the ACK flags and 
multiple operations makes sense to me.
(I doubt if it ever makes sense to set the flags to "SuccessACK", but we 
can await implementation experience.)

However, the text below leaves me uncomfortable.  As I said before, if we 
are going to define the flags as global, they should be global.
I could understand text which said "setting the flags improperly in the 
following will cause problems."
In fact, I believe that an Association Request message like, the others in 
the list, should not normally be sent with an ACK expected.
Heck, a GET request should not normally be sent with an All-ACK.  The 
normal success response to a GET is not an ACK, but rather is the get response.

Is it possible we have some ambiguity here between response messages and 
application level acknowledgements?
Is the only message with a range of behaviors a SET request?  If so, these 
flags belong in the SET request, not in the common message header?

Yours,
Joel M. Halpern

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 24 Aug 2005 16:55:47 -0400

Looking at your more recent not, it seems that these flags only apply to 
the SET (Config) Request).  If that is so, then they belong in the SET 
Operation TLV, or in the format of the Config message.

Yours,
Joel M. Halpern

---

From: Weiming Wang <[email-masked]>
Date: Sat, 27 Aug 2005 00:15:41 +0800

Hi,

Attached is the supposed update for all messages, with the following trace:

1. LFB select TLV is in a specific section.

2. TLVs and Operations as below:

                      TLVs            Operations

AssoSetup           LFBselect           REPORT

AssoSetupResp       ASRresult           none

AssoTeardown        ASTreason           none

Config              LFBselect           SET, DEL

Config Response     LFBselect           SET-RESPONSE 
                                        DEL-RESPONSE

Query               LFBselect           GET

Query Response      LFBselect           GET-RESPONSE

Event Notification  LFBselect           REPORT

Redirect            Redirect            none

Heartbeat           None                none

I haven't modified any text related to ACK, hope to do it after the issue resolve. Pls also comment on this issue.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 11:03:10 +0800

Joel,

After some consideration, I think this ACK flag is for acknowledgement
rather than for response control.  It is only applied to Config message
(note not only SET by also DEL) because other messages have to be constantly
defined for there acknowledgement requirements. While for config message, we
use such flag only try to give the message acknowledge with some choices
like" If you can do this, please let me know", or "If you cannot do this,
please let me know".

Now, if we then put such flag in Operation TLVs in config message, we may
not able to reach the goal above, because it is still vague if a Config
Response is required or not when we set all Op TLVs insde with
NoACK(NoResponse).

As a result, I think to consider a flag in every Operation TLV for response
control may still be an issue separated to the ACK flag used in the header.

One proposed solution for the ACK flag is, I suppose, still put it in the
header, with the define that the flag is only applied to Config message. For
all other messages, the flag is ignored, and for all other messages,
whether responses are required are expicitly defined in the message
definitions.  In this way, I can just save the paragraph that describe the
individual ACK flag usage for all other messages.

Any thoughts?

Thanks,
Weiming

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
To: "Wang,Weiming" <[email-masked]>; <[email-masked]>
Sent: Thursday, August 25, 2005 4:55 AM
Subject: Re: [issue 34] Header flags

> > > It is about event notification messages, and the fact that no
responses
> >are
> > > generate an ACK for ANY message it receives, if the flags tell it to
do
> >so.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 16:42:13 +0800

Jamal,

I understand the two policies default values a little differently regarding
the trustness.

For FE (failover-then-)restart policy, we set the default policy being to
start from scratch. This means CE does not trust FE after FE has been
failover, therfore does not let it restart from other intermediate state.

For CE failover plicy, we set the default policy being to continue running.
This means CE trust the FE for its running when it has lost association with
the CE. The reason the CE can trust the FE here is because the FE still work
properly.

I don't think it leads to some contradictory.

I have to say the name 'FE restart policy' may not properly describe the
case here. I prefer it as the name 'FE failover-then-restart policy' for
more fitness.

Not sure if it makes sense or not.

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 22:27:42 +0800

Issue 45,  the new text was already updated by Jamal in the previous
version. can be closed.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 22:50:37 +0800

Note sec 6.1.1.1.6 is now sec 6.1.1.1.9.
The question raised by the issue has been solved. I just suggest to add a
word 'whole' for the sentence and makes it as below:

2.  When dumping a whole table in a GET, the GET-RESPONSE, merely editing
the T to be SET will endup overwritting the (WHOLE) table.

Jamal, just make a notice here that the SET-REPLACE TLV has been agreed to
just SET. some of the text, like in the sec 6.1.1.1.9 and in the appendix
examples, where SET-REPLACE may need modify to SET. I haven't made this
change, leaving it for your turn may be better :)

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 19:38:00 +0800

was already fixted.  can be closed.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 23:24:48 +0800

The issue was already solved by weiming's new text about Packet redirect msg
in the previous version.

The issue can be closed.

Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 23:28:49 +0800

This issue is totally a overlap of issue 47, which is now under discussion.

This issue can be closed by only leaving issue 47.

Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 23:22:07 +0800

I'v updated the whole text for Association Setup Message as shown in issue
57, by defining the (Optional) Operation TLV for Association Setup msg as
REPORT.

This issue can be closed if this propose can be agreed.

Thanks,
Weiming

---

From: Jamal Hadi Salim <[email-masked]>
Date: Sun, 28 Aug 2005 11:59:35 -0400

> I don't think it leads to some contradictory.

Ok, I think thats a good way to position it. Perhaps we should say that
in the text.

> Not sure if it makes sense or not.

It does - maybe just leave it as is then with the explanation from
above. I would have suggested "FE Failover-restart-policy" but then that
would be misleading because this just describes how you restart and not
necessarily that you are restarting because you have failed over.
Again, I think we need to enumerate the possible (known) schemes.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Sun, 28 Aug 2005 12:01:19 -0400

> change, leaving it for your turn may be better :)

Well, if we can tag it somewhere either in the text or tracker - that
would make sure its not forgotten.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 22:30:01 +0800

Issue 48,  Seems Jamal already updated with the text modification, not sure
if enough or not. can be closed after Jamal's double check.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 22:28:52 +0800

Issue 46,  I think Hormuzd has updated the text according to Joel's
propose. can be closed after Hormuzd's double check.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 22:26:01 +0800

Issue 44, the new text was already updated by Jamal in the previous version.
can be closed.

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 28 Aug 2005 23:16:44 +0800

Note the Fig.11 has been to Fig. 13.

I think to modify the figure by adding some KEYs in the first two
PATH-DATAs,as Robert suggested, may be a good way to solve the issue, which
makes the figure more close to reality, I suppose.

Jamal, could this issue be left for your turn?

Thanks,
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Tue, 30 Aug 2005 18:31:19 +0800

Jamal and Robert,

We'v currently put the FE protocol LFB definitions in Appendix C. When going
through current Section 6.2 (Core ForCES LFBs), I feel that we may need to
make it clear how these two places address the LFBs separately. I'm not sure
if now we follow the way that Section 6.2 only mentions the names (and the
default value) for all attributes while detailed defintions are in the
Appendix (or other document later when we decide).

Anyway, I'd like to put the following text in the Section 6.2 for the FE/CE
policy attributes first. If necessary, I hope Robert can move it to Appendix
C during your turn of editing or manage it in other way, is it ok?

<supposed text>

      *  FE restart policy - This specifies the behavior of the FE
         during a FE restart. The restart may be from a FE failure
         or other reasons that have made FE down and then
         need to restart.

          0(default)     just restart the FE from scratch.
                         In this case, the FE should start from pre-
                         association phase definitely.

          1              restart the FE from an intermediate state.
                         In this case, the FE itself decides from
                         which state it restarts. For example, if
                         the FE still keeps enough infomation of
                         pre-association phase after some failure,
                         it then has the right just to
                         start from post-associatin phase with this
                         policy case.

        Others          Reserved

  *  CE failover policy - This specifies the behavior of the FE
     during a CE failure and restart time interval, or when
     the FE loses the CE association. A timeout parameter is
     associated with this attribute. (The default value for the
     timeout is set to 30 seconds).

     0(default)     The FE should continue running and do
                    what it can even without an associated CE.
                    This basically makes the FE support CE
                    Graceful restart. Note that if the CE still
                    has not been restarted or hasn't been
                    associated back to the FE, after a time interval
                    specified by the timeout parameter, the FE
                    will go down completely.

     1              FE should go down to stop functioning
                    immediately.

     2              FE should go inactive to temporarily stop
                    functioning. If the CE still has not been
                    restarted after a time interval of specified
                    by the timeout parameter, the FE will go
                    down completely.

     Others         Reserved
</supposed text>

The reasons to set above policy values are some from the requirements from
RFC3746 Section 4.3.1 (CE graceful restart) and Section 4.3.2 (FE restart).

For FE restart policy, the text in Section 4.3.2 as below makes sence:

   When FE1 decides to rejoin again, or when it restarts again after the
   failure, FE1 needs to re-discover its master (CE).  This can be
   achieved by several means.  It may re-enter the pre-association phase
   and get that information from its FE manager.  It may retrieve the
   previous CE information from its cache, if it can validate the
   information freshness.  Once it discovers its CE, it starts message
   exchange with the CE to re-establish the association, as outlined in
   Figure 9, with the possible exception that it might be able to bypass
   the transport of the complete initial configuration.  Suppose that
   FE1 still has its routing table and other state information from the
   last association.  Instead of re-sending all the information, it may
   be able to use a more efficient mechanism to re-sync the state with
   its CE, if such a mechanism is supported by the ForCES Protocol.  For
   example, CRC-32 of the state might give a quick indication of whether
   or not the state is in-sync with its CE.  By comparing its state with
   the CE first, it sends an information update only if it is needed.
   The ForCES Protocol may choose to implement
   similar optimization  mechanisms, but it may also choose not to, as
   this is not a requirement.

For CE Failover policy, the following text in Section 4.3.1 makes sense:

   ForCES should be able to support a CE graceful restart easily.  When
   the association is established the first time, the CE must inform the
   FEs what to do in the case of a CE failure.  If graceful restart is
   not supported, the FEs may be told to stop packet processing all
   together if its CE fails.  If graceful restart is supported, the FEs
   should be told to cache and hold on to its FE state, including the
   forwarding tables across the restarts.  A timer must be included so
   that the timeout causes such a cached state to eventually expire.
   Those timers should be settable by the CE.

Thanks,
Weiming

----- Original Message -----

> > > > The draft has been updated in Section 6.2.1(FE Protocol LFB)
according
> > to
> > > > > failure in the NE. Default value for this attribute should be to
let
> > the
> > > > >          operation during a CE failure in the NE. Default value
for
> > this

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 31 Aug 2005 12:35:40 +0800

Hi,

the Section of Operation SUmmary has been updated as below. Pls show your
comments.

Thanks,
Weiming

6.10  Operation Summary

   The following table summarizes the TLVs that compose messages, and
   the applicabiity of operation TLVs to the messages.

+----------------------------+-----------+----------------------------+
|          Messages          |    TLVs   |         Operations         |
+----------------------------+-----------+----------------------------+
|      Association Setup     | LFBselect |           REPORT           |
|                            |           |                            |
| Association Setup Response | ASRresult |            None            |
|                            |           |                            |
|    Association Teardown    | ASTreason |            None            |
|                            |           |                            |
|           Config           | LFBselect |          SET, DEL          |
|                            |           |                            |
|       Config Response      | LFBselect | SET-RESPONSE, DEL-RESPONSE |
|                            |           |                            |
|            Query           | LFBselect |             GET            |
|                            |           |                            |
|       Query Response       | LFBselect |        GET-RESPONSE        |
|                            |           |                            |
|     Event Notification     | LFBselect |           REPORT           |
|                            |           |                            |
|       Packet Redirect      |  Redirect |            None            |
|                            |           |                            |
|          Heartbeat         |    None   |            None            |
+----------------------------+-----------+----------------------------+

   The following talbe summarises the applicability of the DATARAW TLV
   and the RESULT TLV to the Operation TLVs.

              +--------------+-------------+------------+
              |  Operations  | DATARAW TLV | RESULT TLV |
              +--------------+-------------+------------+
              |      SET     |     MUST    |  MUST NOT  |
              |              |             |            |
              | SET-RESPONSE |     MAY     |    MUST    |
              |              |             |            |
              |      DEL     |     MAY     |  MUST NOT  |
              |              |             |            |
              | DEL-RESPONSE |     MAY     |    MUST    |
              |              |             |            |
              |      GET     |   MUST NOT  |  MUST NOT  |
              |              |             |            |
              | GET-RESPONSE |     MUST    |     MAY    |
              |              |             |            |
              |    REPORT    |     MAY     |  MUST NOT  |
              +--------------+-------------+------------+

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 31 Aug 2005 17:05:49 +0800

Hi,

The following is the latest definition for ACK flag, which has been put in the draft. Pls continue comments on:

<text>
   Flags(32 bits):
       Identified so far:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |ACK|  Pri  |Bat|A|             Reserved                        |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       - ACK: ACK indicator(2 bit)
           The ACK indicator flag is only used by a Config Message
           sender to indicate the message receiver whether or not a
           config response is required by the sender.  Note that all
           other messages than Config Message MUST ignore this flag.

           The flag values are defined as below:

               'SuccessACK'(01) - to inidcate the message receiver to
               send a response message back only when the message has
               been successfully processed by the receiver.

               'FailureACK'(10) - to indicate the message receiver to
               send a response message back only when there is any
               failure for the receiver to process(execute) the message.
               In other words, if the message can be processed
               successfully, the sender will not expect any response
               from the receiver.

               'AlwaysACK' (11) - to indicate the message receiver to
               send a response message back in any case.

Doria (Ed.), et al.       Expires March 4, 2006                [Page 27]

Internet-Draft                   ForCES                      August 2005

           Note that in above definitions, the success means a complete
           excution without any failure of the message.  While anything
           else than a complete successful execution is defined as a
           failure for the message processing.  As a result, for the
           execution modes (defined in Section 3.3.1.1) like execute-
           all-or-none, execute-until-failure, and continue-execute-on-
           failure, if any single operation among several operations in
           the same message fails, it will be treated as a failure and
           result in a response if the ACK indicator has been set to
           'FailureACK'.

           Also note that other messages than Config Message, whose
           requirements for responses are statistically prescribed by
           the message definitions in the followed Sections.  The ACK
           indicators in these messages MUST be ignored.

           We in advance summarise the defaut requirements of these
           messages, their requirements for responses as below, while
           detailed descriptions can be found in the individual message
           definitions:

           +   Association Setup Message always expects a response.

           +   Association Teardown Message, Packet Redirect Message,
               and Heartbeat Message never expect responses.

           +   Query Message always expects a response.

           +   Any kind of response messages never expect further
               responses.
</text>

Thanks,
Weiming

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 31 Aug 2005 07:21:30 -0400

Hi Weiming,

Since you MAY have this already in your text.

>    the applicabiity of operation TLVs to the messages.

So what happens if someone adds a new TLV or operation that goes in
addition with the already defined one? I think we need to specify the
behavior. Example it could be "Ignore". Actually i think it should be
"Ignore" for backward compatibility reasons.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 31 Aug 2005 07:50:40 -0400

> The following is the latest definition for ACK flag, which has been put in the draft. Pls continue comments on:

I apologize to bring this up at the last minute; I missed following up
this discussion that you had with Joel.  Also, in your discussion with
Joel you said the flag was for "acknowledgement rather than response"; 
I dont see the difference between the two.
Must we use such strong language above? Example, while probably not very
likely, a) I could send a heartbeat (which is not a Config msg) that i
request a response for - setting the ACK flag allows me to and b) I may
not wish to recieve a response for a certain config.   
Shall we reduce it to a recommendation?:
"The ACK indicator flag is used by the sender to indicate to the
receiver how it wishes to see a response. As an example Config Message
SHOULD always set this flag while other message types SHOULD NOT"

I used the word "example" because you have more clarifying text below.

[..]

>            indicators in these messages MUST be ignored.

Another use of the term MUST above.

cheers,
jamal