Subject: Re: [issue54] correlator in the PL message header

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Sun, 12 Jun 2005 12:22:51 -0700

Jamal,

Do you see a good reason to keep a SN? I don't see one since the TML
provides reliability. If there is another reason, let us
know...otherwise we should get rid of the SN.

Thanks
Hormuzd

-----Original Message-----
From: Forwarding and Control Element Separation
[mailto:[email-masked]] On Behalf Of Jamal Hadi Salim
Sent: Sunday, June 12, 2005 8:37 AM
To: [email-masked]
Subject: Re: [FORCES] [issue54] correlator in the PL message header

On Sun, 2005-12-06 at 11:00 -0400, Joel M. Halpern wrote:

> But we can not simply treat the sequence number as the correlator.
> 

Agreed.
In the some cases we could - but i think allowing for such polymorphic
behavior is something we may live to regret (it has been trickery-hell
in netlink to ensure backward compatibility because the seq # is used as
a correlator as well) so lets just add a second 32 bit number in the
main header. 

cheers,
jamal

---

From: "Deleganes, Ellen M" <[email-masked]>
Date: Mon, 13 Jun 2005 10:34:58 -0700

I should have been clearer. When I voted for option 1, I didn't mean to
imply the sequence number field is overloaded to mean the same thing as
the correlator. I agree that the correlator should not be tied to the
sequence number. It should be up to the issuer of the requester to
assign a correlator to the request, which is echoed in the response.

There seems to be some question as to whether a sequence number is
needed. If the sequence number is required, then I would change my vote
to option 3 (both fields).

Regards,
Ellen

-----Original Message-----
From: Forwarding and Control Element Separation
[mailto:[email-masked]] On Behalf Of Jamal Hadi Salim
Sent: Monday, June 13, 2005 10:29 AM
To: [email-masked]
Subject: Re: [issue54] correlator in the PL message header

On Mon, 2005-13-06 at 08:36 -0700, Deleganes, Ellen M wrote:
> I also agree that a correlator field looks to be useful. Given that at
> least a few people are arguing that a requirement of the TML is to
> guarantee in order delivery, the correlator field appears to be more
> useful than the sequence number field.
> 
> The argument seems to be more whether two fields are needed, not
whether
> a correlator field is useful.
> 
> Perhaps the better way to ask is which option is preferred, to
determine
> if there is consensus:
> 1) Correlator field only
> 2) Sequence number field only
> 3) Both
> 
> (I vote for option 1)

A sequence number could be used as a correlator. That's what we are
doing today in our implementation for example. 
But that in itself is making some assumption about the implementation
details and to a small degree OS level details[1].
Infact, given our implementation today we may never need the corellator
number - although on occasion i do worry about transactional issues and
Ha when we get to it. It would certainly ease the state  maintainance we
do if we had it; OTOH, if forces was already released as an RFC i could
live without it. We are not at that stage yet and
i dont wanna impose how we implement onto Robert.

Despite the above, I think Joel put it best in one of his emails, to
quote:

"A Sequence number should have a different sequential value has I hand 
messages from the PL to the TML.
A correlator should have distinct values for distinct requests, and
replies should have the value from the request.
These are not the same property.  They may even be in conflict."

The TML sequence number is a transport level one and really has nothing
to do with PL level. A PL level sequence could in fact be multiple
transport level sequence numbers. 
The correlator is, to be precise, opaque to the receiver (typically FE)
whereas the sequence number could be intepreted.

cheers,
jamal

Unfortunately, such assumptions could get us in trouble and i would
rather give up 32 bits now than regret later. [To compare with the
callback model and the associated correlators in the NPF APIs model
where we are currenlty working around assumptions (such as a flat memory
space OS or the implementation using multithreading).]

---

From: "Putzolu, David" <[email-masked]>
Date: Mon, 13 Jun 2005 11:32:42 -0700

I believe that FE should not make any assumptions about the correlator
values.  While the CE may make them sequential (or choose them using any
other algorithm it wishes), the FE should not rely on it.  This is both
to allow the CE implementor freedom and in order to make the FE more
robust - if the spec mandated that the correlators be sequential, and
the FE relied on that property, a CE with a bug in the sequence would
cause FE failure.  A more robust FE would not rely on the sequence
(under the be liberal in what you accept guideline).  Given this, it is
better to simply remove the constraint that the CE make them sequential.

I've found correlators/cookies to be useful in similar contexts as this,
so I support putting one in.

For sequencing, I prefer to rely on the TML for this, as it is not clear
to me why one would want to lift it out of the TML.

Cheers,
David

-----Original Message-----
Sent: Monday, June 13, 2005 6:01 AM

Ok, folks lets put closure to this: so far the stick-finger-in-the-air
answer i am seeing to the question "should we add a correlator field in
the main header in addition to existing sequence number?"

Jamal: 100% yes
Robert: 100% yes
Joel: 65% yes - Joel can we get a boolean from you?
Hormuzd: 100% no
Weiming:Not sure where you stand.

cheers,
jamal

On Mon, 2005-13-06 at 07:31 -0400, Joel M. Halpern wrote:
> Actually, I would prefer to put as few restrictions as possible on the

> creator of the correlator, so as to allow the most implementation
freedom.
> 
> Hence, the only MUST would be that a responder MUST put the correlator
from 
> a request into the response.
> 
> It seems like "A requestor SHOULD NOT reuse the correlator from
outstanding 
> requests for different requests." makes sense.  I would not want to
make 
> that a MUST.
> 
> An receiver MUST NOT apply any semantics to received correlators.
> 
> Sequence information is either derivable from the TML or should be in
a 
> separate field.
> Yours,
> Joel
> 
> At 11:30 PM 6/12/2005, Wang,Weiming wrote:
> >Firstly , I think even if we use it only as a correlator, the number
should
> >sitll not be generated randomly, rather sequencially. Secondly, it
therefore
> >menas we actually doesn't pay too much by introducing the Sequential
Number in
> >while without loosing the correlator function. Thirdly, it maybe too
early to
> >say a SN is not at all useful for the protocol PL level. For this
point, I
> >suppose we should distinguish the idea of the protocol message
sequence 
> >from the
> >TML transport sequence a little. I can curretnly see two cases where 
> >protocol SN
> >may make senses to PL thought not to TML:
> >
> >1. For messages like Packet redirect messages,  transport order of
which 
> >may not
> >strictly requred, but the sequence of the redirected packets still
make 
> >sense to
> >the applications of the packets.
> >
> >2. Usually, core PL process runs in a multi-thread mode, in this
case, a 
> >SN can
> >act well as a tracker for the process. This can be used at PL-TML
interface PL
> >side as well as the TML side.  This may also be used for CE or FE
failover 
> >case
> >like graceful restart.
> >
> >Cheers,
> >Weiming
> >
> >----- Original Message -----
> >From: "Khosravi, Hormuzd M" <[email-masked]>
> >
> >I agree, this was along the lines of my previous suggestion. I don't
see
> >the reason for having both a sequence number and a correlator....SN
> >should not be needed by the PL since the TML provides reliability
which
> >guarantees in-order delivery (acc to the definition). Would you
agree?
> >[mailto:[email-masked]] On Behalf Of Joel M. Halpern
> >Sent: Sunday, June 12, 2005 8:00 AM
> >Actually, I would prefer to change the name.
> >
> >A Sequence number should have a different sequential value has I hand
> >These are not the same property.  They may even be in conflict.
> >
> >Note that a request originator is capable of remembering the sequence
in
> >
> >which it sends requests.
> >Thus, for most purposes a correlator will do more than a sequence
> >number.
> >What a correlator does not provide is the ability to compare the
order
> >of a
> >received even and a received response.  If some TML may disorder
things,
> >
> >that would be lost information.
> >If that is a big deal, we need two fields.
> >
> >Yours,
> >Joel
> >At 11:19 PM 6/11/2005, Khosravi, Hormuzd M wrote:
> > >Ok, so you're saying we should use the Sequence number as a
Correlator.
> > >If that is the case then we are in agreement, I think. We should
update
> > >the text.
> > >
> > >Let me know if I misunderstood you,
> > >Hormuzd
> > >
> > >[mailto:[email-masked]] On Behalf Of Wang,Weiming
> > >Sent: Saturday, June 11, 2005 8:14 PM
> > >----- Original Message -----
> > >From: "Khosravi, Hormuzd M" <[email-masked]>
> > >
> > >Yes, a Correlator is useful and currently we have been using the
> > >Sequence number as correlator. I don't see any other use of the
> >Sequence
> > >number since reliability is provided by the TML. I think we should
> > >change Sequence number to correlator in the draft and update the
> > >description accordingly unless there is some use for a sequence
number.
> > >
> > >[Weiming] I think it's too early to say the SN is only for
correlation.
> > >currently I don't think we should a hundred percent to rely on TML
for
> > >thinks
> > >like sequential ordering.   As a safer mode, I still prefer to have
it
> > >as its
> > >original  name of Sequence Number. Correlating request/reply
messages
> > >should
> > >only be one of its functions.
> > >
> > >thanks,
> > >Weiming
> > >
> > >BTW, issue 47 looks like the same one...we should close one of
these
> > >issues.
> > >regards
> > >[mailto:[email-masked]] On Behalf Of Wang,Weiming
> > >Sent: Friday, June 10, 2005 9:12 AM
> > >Robert,
> > >----- Original Message -----
> > >From: "Robert Haas" <[email-masked]>
> > >
> > >New submission from Robert Haas <[email-masked]>:
> > >
> > >Adding a correlator seems useful: response messages can be matched
to
> > >the
> > >request messages even if pipelining and reordering occurs.
> > >It seems that some are currently using the sequence number as a
> > >correlator:
> > >this can be dangerous, and not suitable for event notifications
> >messages
> > >(nothing to correlate with).
> > >[Weiming] I'm not sure if you mean to use the SN as a correlator is
to
> > >let the
> > >response have the same SN as the sender.  If so, I cannot follow
well
> > >why it is
> > >dangerous and not suit for event notifications.  Could you clarify
this
> > >more?
> > >
> > >Thanks.
> > >Weiming
> > >
> > >But this introduces the risk of flooding an FE with too many
commands
> >as
> > >there
> > >is no windowing mechanism in place (yet).
> > >
> > >----------
> > >category: Technical
> > >draft: draft-ietf-forces-protocol
> > >messages: 129
> > >nosy: rha
> > >priority: Should fix
> > >status: No discussion
> > >title: correlator in the PL message header
> > >
> > >___________________________________________________
> > >Forces issue tracker <[email-masked]>
> > ><http://www.mip4.org/issues/tracker/forces/issue54>
> > >___________________________________________________

---

From: Weiming Wang <[email-masked]>
Date: Sat, 25 Jun 2005 01:58:34 +0800

Susan,

Very glad to see you.  

----- Original Message ----- 
From: "Susan Hares" <[email-masked]>
Subject: RE: [FORCES] [issue54] correlator in the PL message header

Weiming:

Great news that we can have 2 fields.  Are you firm on the SN/correlator
values or would you entertain a discussion of parallelization of
updating streams using the SN/correlator?
[Weiming] Sorry, but I just cann't catch well on the parallelization of updating streams here.  Could you show more of your thoughts on this?

Thanks,
Weiming

Sue Hares

-----Original Message-----
From: Forwarding and Control Element Separation
[mailto:[email-masked]] On Behalf Of Weiming Wang
Sent: Friday, June 24, 2005 12:14 PM
To: [email-masked]
Subject: Re: [FORCES] [issue54] correlator in the PL message header

Jamal,

Thanks for the mention, whereas I think I'v expressed my opinion in my
previous posts. I can live with two fields (SN and correlator) solution,
thought I do think one field is enough for our current use for the
message sequencing purpose, that is, sequencing of messages without
response messages, which already enough for the purpose like redirect
message ordering back. One more thing is, for two fields solutions, I
suggest to let the correlator field usually have the same value as the
SN field, except for response messages, which should have the correlator
values as the messages they reply, so as to simplify the correlator
value generating.

Thanks,
Weiming
----- Original Message ----- 
From: "Jamal Hadi Salim" <[email-masked]>
Subject: Re: [issue54] correlator in the PL message header

> On Thu, 2005-23-06 at 00:17 -0700, Khosravi, Hormuzd M wrote:
> > Thanks for the input, Sue.
> > I believe we have decided to leave the SN as is and add a Correlator
as
> > well to the header. So that should address any such requirements.
> > 
> 
> I dont know if we have heard from Weiming on this so we can close it?
> 
> cheers,
> jamal

---

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Sat, 25 Jun 2005 14:37:32 -0700

Robert,

I don't understand the need for a transaction id. I believe SN and Correlator are more than sufficient for the protocol. It almost seems like a terminology issue, if calling a correlator a transaction id makes you happy, I am fine with that. But other than terminology I don't see a need for yet another ID.

regards
Hormuzd

-----Original Message-----
From: Forwarding and Control Element Separation [mailto:[email-masked]] On Behalf Of Robert Haas
Sent: Friday, June 24, 2005 6:35 AM
To: [email-masked]
Subject: Re: [FORCES] [issue54] correlator in the PL message header

I haven't heard much reactions to my last post on the topic, so let me
reformulate it in a shorter way:
- what is currently called SN is in fact used as a Transaction ID. One
can (ab)use it for sequencing a subset of messages (see details in my
previous post)
- for full sequencing capability, a distinct field from Transaction ID
should be introduced.
- in addition, yet another field for higher-level correlation should be
introduced.

So now that we agreed we need (at least) two fields, how shall we call 
them and what meaning/semantic do we give to them ?

field #1: Transaction ID ? 16 bits ?
field #2: Correlator ? 32 bits ?
possibly field #3: true Sequence Number 16-32 bits ?

Regards,
-Robert

Jamal Hadi Salim wrote:

>>I believe we have decided to leave the SN as is and add a Correlator as

-- 
Dr. Robert R. Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-44-724-8698  fax +41-44-724-8578  http://www.zurich.ibm.com/~rha