Subject: RE: [rohc] [SigComp]Shared mode and some other SigComp-state rela ted stuff.

From: "Surtees, Abigail" <[email-masked]>
Date: Thu, 14 Apr 2005 08:58:11 +0100

Hi Zacco,

Once again, apologies for the delay in responding to this.

I am somewhat confused by some of the statements in the discussion below and the level of indentation isn't helping.  Consequently, I'm going to state my understanding and you can tell me where you disagree or I am wrong.  I hope that's ok.  I've used text here and put some ASCII diagrams in the attached file (they get messed up if they're in the body of the email).

Let's start with basic SigComp operation (to ensure we're all clear on the terminology and interpretation of pictures - using a 2 byte partial hash to identify state):

Scenario 1:
Endpoint A sends a message to endpoint B requesting two pieces of state to be created and feedback to be returned:

Bytecode with hash bcbc
Buffer state with hash bff1

The state handler at B has no idea what the pieces of state are - they are just chunks of bytes.  Endpoint B includes the feedback in its return message, thus endpoint A can refer to both pieces of state in its next message.

Scenario 2:

Endpoint A knows about a dictionary (with hash d1c7).  It sends a message to endpoint B requesting state creation and feedback as in scenario 1, but also advertising d1c7.

If endpoint B doesn't know anything about d1c7, then the advertisement will be ignored and the return message will not refer to it.

If endpoint B does know about d1c7 and has the capacity to use it, then the return message will refer to it.

Scenario 3:
Endpoint A knows about dictionary (with hash d1c7) and wants to be able to use it for compression itself, so it uploads it to endpoint B as another piece of state.

Scenario 4:
Endpoint A knows about dictionary (with hash d1c7) and wants to be able to refer to it at endpoint B AND it thinks B may be a sophisticated implementation.  Therefore endpoint A not only uploads the dictionary (as in scenario 3) but also advertises its presence at endpoint A, allowing B to refer to it.  N.B.  By advertising it, endpoint A is saying that the dictionary will be available for the duration of the compartment (this may be in memory external to the compressor SMS, or it may be created within the compartment e.g. at sufficiently high priority in the compressor A SMS that it will not be pushed out by other state - compressor A has full control of both alternatives).

The simplest implementation of B on receipt of the advert for d1c7 would decide it knows nothing about d1c7 as is the case with any other state advert about which it knows nothing.

A more sophisticated implementation may notice that the state being advertised has also been created in the decompressor SMS.  Therefore, because it has the bytes of the state, the compressor at B *could* make use of the state.  However, the compressor at endpoint B does not know what sort of state it is - it's just a bunch of bytes.  Some heuristic (e.g. comparison with the next message to be compressed) could be used to decide whether or not the piece of state is useful as a reference to improve the compression ratio of the next message.  

Endpoint B can refer to d1c7 as long as it knows what it is (i.e. it might get pushed out of the decompressor SMS at B, at which point, B no longer has it).  As mentioned above, once the state has been advertised by endpoint A, endpoint A must keep it for the duration of the compartment.

So far, all of the above is standard rfc 3320 behaviour.  When compressor A advertises the dictionary, where it stores the bytes of dictionary (inside or outside SMS) is implementation specific, though in scenarios 3 and 4 it must be accounted for in the SMS monitoring because it is mirroring the SMS at endpoint B.  At endpoint B there is no requirement to do anything other than store state according to rfc 3320.

I think that placing 'usd1' as the first 4 bytes of a dictionary is then a simple heuristic optimisation.  If endpoint A puts 'usd1' at the beginning of the state then endpoint B has a simple way of recognising that the state will be useful as a reference.  If, however, A doesn't do that, then B (if it wants to) can use some other heuristic, as it would without the optimisation.  Equally, if endpoint A does put 'usd1' in the state and endpoint B doesn't look for it, both endpoints continue as if it were not there.

The key point is that, whatever happens, the actual state creation and handling at both endpoints is the same, regardless of whether or not the state contains the bytes 'usd1'.  

Is my interpretation correct?  If not, please explain my misunderstanding.  Whether or not  my understanding is correct, the text describing this mechanism needs clarification in the PoC specification.  (I realise this is not the place for that discussion - I'm just trying to make sure I understand correctly).

Best regards,

Abbie

> > 
> > Sorry for the delay.  I've got some questions about the PoC 
> > related use of
> > usds.
> > 
> > > 
> > > > 3. USD. I fully understand the usage of a USD. However, I do 
> > > > not understand 
> > > > if a USD-state should be treated any differenty then any 
> > > > other state created 
> > > > at the remote end. Is a USD per definition a shared state? 
> > > 
> > > No, it isn't. RFC 3321 says it's a regular state. POC USD is 
> > > something different. I would say it's a regular and a shared 
> > > state at the same time.
> > 
> > What does that actually mean?  As you say 3321 says it's a 
> > regular state, so
> > in SigComp terms how can it be a shared state as well?
> 
> [zacco] In PoC, the terminal uploads the USD after saving it 
> in the state memory. The special advertisement lets the 
> server know it received a USD that it also saves in the state 
> memory (or outside the state memory, which must be signalled 
> backwards). Thus, both endpoints will have the USD and can 
> use it for compression. That is why I say PoC USD is a 
> dynamic (or regular) and shared state at the same time.
> 
> > 
> > > 
> > > > I'm guessing that 
> > > > it must be if the sending compressor is to have any 
> > > advantage of the 
> > > > USD-state at the remote end. Then, again, should the 
> > > > USD-state (local state) 
> > > > at A be considered to occupy State-memory? Should it be 
> > > > considered any 
> > > > differently then any other shared state?
> > > 
> > > It depends on which USD you are talking about. A regular 
> > > state is not saved in a state memory at the compressor side, 
> > > so if you go for the RFC 3321 USD, it decreases the available 
> > > memory at B in the state memory of compresssor at A. In case 
> > > POC USD is used, it must be saved as a local state, otherwise 
> > > the compressor at B would not be able to access it.
> > 
> > However, we believe that the POC definition of using USD as 
> > shared state is
> > not well defined and can lead to loss of synchronisation 
> > between the two
> > endpoints.
> > 
> > When you say it must be saved as 'local state', do you mean 
> > in the state
> > memory of the compressor at B or not in a compartment at all?
> 
> Not in a compartment at all (in A). However, in B it is up to 
> the endpoint if it saves in the state memory or not in the 
> compartment.
> 
> > > 
> > > > 5. (Little bit off-topic, PoC specific question): In the PoC 
> > > > NEMS Signalling 
> > > > Spec. "Signalling flows (UNI) v2.06", a procedure for 
> > > > establishing a USD is 
> > > > described. Among others it states that the four first 
> > bytes of the 
> > > > state-value should be "usd1". Can any helpful individual 
> > > > explain to me what 
> > > > the purpose might be? Should either local or remote 
> > > endpoint take any 
> > > > specific action here?
> > > > The spec says:
> > > > "For USD identification it is important that the identifier 
> > > > string is placed 
> > > > in the beginning of the state_value of the USD, therefore 
> > > > special attention 
> > > > must be paid in the case of large USD as the UDVM has a 
> > > > circular buffer."
> > > > I do not understand the purpose here..
> > > 
> > > Suppose Endpoint 2 supports USD (without the optional 
> > > requirement of moving it to the local state location) and the 
> > > explicit acknowledgement specified in RFC 3321, too. If 
> > > Endpoint 2 explicitlly acks the uploaded USD state ID, 
> > > Endpoint 1 will think USD is moved to the local states 
> > > location (indication is the same as if it was an explicit 
> > > ack) and will not update (decrease) its remote available SMS. 
> > > Later on it may result in decompression failure if a state is 
> > > saved deleting another one in use, due to the remote 
> > > available SMS out of synchron.
> > > 
> > 
> > I'm a bit confused here:
> > 
> > By 'moving it to the local state location' I take it that you 
> > mean that the
> > state created at Endpoint 2's decompressor would be moved to 
> > Endpoint 2's
> > compressor.  Is this correct?  Would it be duplicated (i.e. 
> > still exist in
> > the decompressor) or actually moved (and therefore the 
> > compressor at A has
> > no control over or knowledge of when it is deleted)?
> > 
> It is not duplicated. I meant it was moved to the 
> "non-compartment" state memory. It must be remembered somehow 
> (implementation stuff) that the state actually belongs to a 
> specific compartment, so when the compartment is deleted, the 
> USD must also be deleted.
> 
> > Are you saying that the presence of the string "usd1" is an 
> > indicator to
> > endpoint 2 that it should move/copy the state into the compressor
> > compartment?
> 
> Besides the state is saved and advertised at the same time. 
> The reason why the usd tag was added is that this situation 
> might easily happen with a well-known bytecode.
> 
> > 
> > If so, this seems to be at odds with the SigComp conceptual 
> > model of state
> > that it is just 'some bytes'.  
> > 
> 
> Why do you say that? 'usd1' is not something that appears at 
> the beginning of a state, especially in a first message, and 
> especially not in a bytecode.
> 
> > If my understanding above is correct, then the piece of state 
> > presumably has
> > minimum retention priority?  Care must once again be taken 
> due to the
> > possibility that the creation of multiple pieces of minimum 
> retention
> > priority state can lead to loss of synchronisation between 
> endpoints.
> > 
> 
> USD is just one state, which is absolutely in line with the 
> "one shared state at a time" concept.
> 
> > How does this mechanism interact with the use of shared mode 
> > as defined in
> > 3321?
> > 
> 
> Nothing special, as the shared state is not saved as a state 
> in B, only in A. 
> 
> > I also think there are some implications for 
> interoperability between
> > endpoints that do this and endpoints that don't.  As you 
> > showed in your
> > example above, if there is disagreement between endpoints 
> > about whether or
> > not state is moved to 'local state location' (whatever that 
> > is) it can lead
> > to loss of synchronisation between endpoints.
> > 
> > Is this concept mandatory in PoC?  If not, or if a non PoC 
> > endpoint receives
> > a state create request and an explicit advertisement, it will 
> > not know that
> > it should check the first 4 bytes for "usd1" and move the state so
> > synchronisation will be lost as you explained.
> 
> As far as I know it is mandatory, but this I should check.
> 
> > 
> > Best regards,
> > 
> > Abbie
> > 
> > -- 
> > 
> > Visit our website at www.roke.co.uk
> > 
> > Roke Manor Research Ltd, Roke Manor, Romsey, Hampshire SO51 0ZN, UK.
> > 
> > The information contained in this e-mail and any attachments 
> > is proprietary to
> > Roke Manor Research Ltd and must not be passed to any third 
> > party without
> > permission. This communication is for information only and 
> > shall not create or
> > change any contractual relationship.
> > 
> > 
> 

_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: "Lajos Zaccomer \(IJ/ETH\)" <[email-masked]>
Date: Fri, 15 Apr 2005 07:49:09 +0200

HI Abbie,

No problem. See my comments inline ...

Best regards,
Zacco

> -----Original Message-----
> From: Surtees, Abigail [mailto:[email-masked]]
> Sent: Thursday, April 14, 2005 9:58 AM
> To: Lajos Zaccomer (IJ/ETH)
> Cc: [email-masked]
> Subject: RE: [rohc] [SigComp]Shared mode and some other 
> SigComp-state rela ted stuff.
> 
> 
> Hi Zacco,
> 
> I am somewhat confused by some of the statements in the 
> discussion below and the level of indentation isn't helping.  
> Consequently, I'm going to state my understanding and you can 
> tell me where you disagree or I am wrong.  I hope that's ok.  
> I've used text here and put some ASCII diagrams in the 
> attached file (they get messed up if they're in the body of 
> the email).
> 
> Let's start with basic SigComp operation (to ensure we're all 
> clear on the terminology and interpretation of pictures - 
> using a 2 byte partial hash to identify state):
> 
> Scenario 1:
> Endpoint A sends a message to endpoint B requesting two 
> pieces of state to be created and feedback to be returned:
> 
> Bytecode with hash bcbc
> The state handler at B has no idea what the pieces of state 
> are - they are just chunks of bytes.  Endpoint B includes the 
> feedback in its return message, thus endpoint A can refer to 
> both pieces of state in its next message.
> 
> Scenario 2:
> 
> Endpoint A knows about a dictionary (with hash d1c7).  It 
> sends a message to endpoint B requesting state creation and 
> feedback as in scenario 1, but also advertising d1c7.
> 
> If endpoint B doesn't know anything about d1c7, then the 
> advertisement will be ignored and the return message will not 
> refer to it.
> 
> If endpoint B does know about d1c7 and has the capacity to 
> use it, then the return message will refer to it.
> 
> Scenario 3:
> Endpoint A knows about dictionary (with hash d1c7) and wants 
> to be able to use it for compression itself, so it uploads it 
> to endpoint B as another piece of state.
> 
> Scenario 4:
> Endpoint A knows about dictionary (with hash d1c7) and wants 
> to be able to refer to it at endpoint B AND it thinks B may 
> be a sophisticated implementation.  Therefore endpoint A not 
> only uploads the dictionary (as in scenario 3) but also 
> advertises its presence at endpoint A, allowing B to refer to 
> it.  N.B.  By advertising it, endpoint A is saying that the 
> dictionary will be available for the duration of the 
> compartment (this may be in memory external to the compressor 
> SMS, or it may be created within the compartment e.g. at 
> sufficiently high priority in the compressor A SMS that it 
> will not be pushed out by other state - compressor A has full 
> control of both alternatives).

[zacco] In PoC, state d1c7 is called a USD and is stored in memory external to the compressor.

OMA-POC-2004-0655-CP-SigComp 5.X.1.2 says:
"The USD is a locally available state, [RFC3320], at the PoC Client and to improve the compression efficiency in the uplink the PoC Client MAY upload the USD to its SIP/IP Core compartment. To increase the compression efficiency in the downlink, the PoC client MAY advertise the locally available state to the SIP/IP core. 
The first four bytes of the USD state value SHALL be the specific identifier string; "usd1" in order to distinguish the USD state from the possible bytecode uploaded for the Universal Decompressor Virtual Machine (UDVM)."

> 
> The simplest implementation of B on receipt of the advert for 
> d1c7 would decide it knows nothing about d1c7 as is the case 
> with any other state advert about which it knows nothing.
> 
> A more sophisticated implementation may notice that the state 
> being advertised has also been created in the decompressor 
> SMS.  Therefore, because it has the bytes of the state, the 
> compressor at B *could* make use of the state.  However, the 
> compressor at endpoint B does not know what sort of state it 
> is - it's just a bunch of bytes.  Some heuristic (e.g. 
> comparison with the next message to be compressed) could be 
> used to decide whether or not the piece of state is useful as 
> a reference to improve the compression ratio of the next message.  
> 
> Endpoint B can refer to d1c7 as long as it knows what it is 
> (i.e. it might get pushed out of the decompressor SMS at B, 
> at which point, B no longer has it).  As mentioned above, 
> once the state has been advertised by endpoint A, endpoint A 
> must keep it for the duration of the compartment.

Not quite if d1c7 is moved to external memory in B, which is optional and up to B. If so, it must be indicated with explicit acking this state. (The only problem with this solution that it fails, if you advertise e.g. your bytecode at the same time, and B explicitly acks the bytecode instead of d1c7. In this case, A thinks d1c7 is in external memory, and will most probably overwrite d1c7 sooner or later.

> 
> So far, all of the above is standard rfc 3320 behaviour.  

Yes. My example is PoC. If we talk about 3320-22 only, I have nothing to add or modify.

> When compressor A advertises the dictionary, where it stores 
> the bytes of dictionary (inside or outside SMS) is 
> implementation specific, though in scenarios 3 and 4 it must 
> be accounted for in the SMS monitoring because it is 
> mirroring the SMS at endpoint B.  At endpoint B there is no 
> requirement to do anything other than store state according 
> to rfc 3320.

Unless you want to implement some PoC standard features, as well.

> 
> I think that placing 'usd1' as the first 4 bytes of a 
> dictionary is then a simple heuristic optimisation.  If 
> endpoint A puts 'usd1' at the beginning of the state then 
> endpoint B has a simple way of recognising that the state 
> will be useful as a reference.  If, however, A doesn't do 
> that, then B (if it wants to) can use some other heuristic, 
> as it would without the optimisation.  Equally, if endpoint A 
> does put 'usd1' in the state and endpoint B doesn't look for 
> it, both endpoints continue as if it were not there.

Right.

> 
> The key point is that, whatever happens, the actual state 
> creation and handling at both endpoints is the same, 
> regardless of whether or not the state contains the bytes 'usd1'.  
> 
> Is my interpretation correct?  If not, please explain my 
> misunderstanding.  Whether or not  my understanding is 
> correct, the text describing this mechanism needs 
> clarification in the PoC specification.  (I realise this is 
> not the place for that discussion - I'm just trying to make 
> sure I understand correctly).
> 
> Best regards,
> 
> Abbie
> 
> > >

---

From: "Surtees, Abigail" <[email-masked]>
Date: Fri, 15 Apr 2005 16:59:25 +0100

Hi Zacco,

We've got crossed wires somewhere and I'm afraid I'm still confused.
What I was trying to say is the following:

1) In standard rfc 3320 behaviour, if an endpoint advertises a piece of
state to the remote endpoint, it is saying that it has that piece of
state available for use for the duration of the compartment*.  It is up
to the implementer whether the bytes of the advertised state are
uploaded to the remote endpoint and created as high priority state (i.e.
stored in the compressor compartment) or are stored outside the
compressor SMS in the endpoint (more likely e.g. dictionary).

*Shared mode state is an exception to this.

2) The remote endpoint *can*, if it is sophisticated, notice that a
piece of state has been both uploaded (i.e. the bytes are in the
decompressor compartment) and advertised and, therefore, choose to use
this piece of state as reference state for its return message.

3) There is no requirement on the remote endpoint to do this.

4) If the remote endpoint does this, neither endpoint has done anything
other than use standard rfcs 3320 - 22 behaviour.

5) An optional mechanism that would make it easier for the remote
endpoint to recognise that a piece of state is useful as a reference is
to include the bytes 'usd1' as the first 4 bytes.  However, addition of
these bytes DOES NOT change the state processing at either endpoint.
The state is stored in exactly the same way (implementer's choice at the
compressor, in the SMS memory at the decompressor) as if the bytes are
not there.

6) If the remote endpoint chooses to move the state from its
decompressor compartment and advertise it back to the local endpoint,
that is an implementation choice and is still perfectly compliant with
rfcs 3320 - 22.  (An endpoint choosing to do this must carefully
consider the security implications.)

7) If there is anything in the intention of what is written in the PoC
specification that is not covered here, please could you explain (e.g.
with use of diagrams similar to the ones in my previous email) what it
is that I'm missing.

8) As it stands at the moment, this part of the PoC specification is
unclear, ambiguous and could lead to loss of synchronisation.

Best regards,

Abbie

> "The USD is a locally available state, [RFC3320], at the PoC 
> Client and to improve the compression efficiency in the 
> uplink the PoC Client MAY upload the USD to its SIP/IP Core 
> compartment. To increase the compression efficiency in the 
> downlink, the PoC client MAY advertise the locally available 
> state to the SIP/IP core. 
> The first four bytes of the USD state value SHALL be the 
> specific identifier string; "usd1" in order to distinguish 
> the USD state from the possible bytecode uploaded for the 
> Universal Decompressor Virtual Machine (UDVM)."
> 
> > 
> Not quite if d1c7 is moved to external memory in B, which is 
> optional and up to B. If so, it must be indicated with 
> explicit acking this state. (The only problem with this 
> solution that it fails, if you advertise e.g. your bytecode 
> at the same time, and B explicitly acks the bytecode instead 
> of d1c7. In this case, A thinks d1c7 is in external memory, 
> and will most probably overwrite d1c7 sooner or later.
> 
> >