Subject: [Tsvwg] Concerns about SCTP

From: "Williams, Jim" <[email-masked]>
Date: Fri, 27 Dec 2002 13:53:41 -0500

These concerns arose as a result of looking
at SCTP as a potential transport for RDDP.

My view is perhaps a bit different in that
I look at the issues from a hardware, or
low level embedded, perspective wherein
resources must be micro-managed.
System software folks sometimes take what
I regard as a sloppy view or resource
allocation; simply call malloc (or the
kernel equivalent) when resources are
required, and abort the association if
the malloc fails.

For an embedded product that claims to
support 1024 associations, if a test
program is able to cause any of those
associations to fail with a resource
error, the product is defective, and
must be fixed.

I regard the following as two basic
rules of protocol design.

Rule 1:  Two implementations that are fully
         compliant with the protocol definition
         must interoperate.  If they don't,
         then that is a defect in the protocol
         definition.

Rule 2:  One end of an association should not
         be able to consume an unbounded amount
         of resources on the other end without
         negotiation and agreement from the other
         end.

I believe in this respect, the multi-homing feature
of SCTP comes up short.  Each remote address that
is supported consumes resources on the local end.
SCTP places no limit and requires no negotiation
as to the number addresses that can be included
in an INIT.  The receiver must either allocate
the resources needed to support this number of
addresses (violating rule 2), or must abort
the association if too many are requested
(violating rule 1).

There may be some "tricks" an implementation
can use to work around the above problem, but
this state of affairs is not good.

Another concern, although less serious, is the
use of a_rwnd as a mechanism to manage receive
queue resources.   The number of bytes or 
received data is not generally a good measure
of the receive resources consumed.  Received
data is generally not copied into packed buffers,
since copies are usually minimized.  Also
state is required to track TSNs and stream
info.  1000 one byte data chunks consume
a lot more resources that one 1000 byte chunk.
The concern is less serious because incoming
packets can always be dropped as a last 
resort for managing receive resources.

Preferable would have been to make implementation
of ECNE and CWR mandatory.  This mechanism, even in
the absence of ECN is useful for a receiver 
to indicate congestion and throttle back the senders.
It works more effectively than a_rwnd, because
and advertised window can't be retracted once
granted, and because the receiver doesn't generally
know the amount of data outstanding from the 
sender, and therefore doesn't know how to size
a_rwnd for optimum flow control.  ECNE, however,
does exactly the right thing.

Dropping packets is usually exactly
the wrong thing to do, because it prevents
the subsequent packets from being processed,
acknowledged, and delivered.  This is exactly
what you don't want to do in case of receiver
congestion, because it makes the congestion
worse.  The sender can be "faked out"
by not dropping a packet, but pretending
to do so as indicated in the SACK.  This 
is messy, however, and causes an unnecessary
retransmission.

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Qiaobing Xie <[email-masked]>
Date: Fri, 27 Dec 2002 14:54:30 -0600

Hi, Jim,

>          then that is a defect in the protocol

Depending on how you define "interoperate", it can strictly mean that
the two ends MUST be able to carry out successful data communications,
or it can be more broadly defined to even include that one end can
nicely say "no, I choose not to work with you". 

>          negotiation and agreement from the other

But it certainly should be allowed to ask. Whether the other end will
grant the request is a different issue and is pretty much implementation
specific.

> the association if too many are requested

The four way handshaking in SCTP _is_ a form of negotiation, in which
either end can indicate its multi-homing request and has an opportunity
to say "no, you are asking for too much" to the other. And if it chooses
to, the other end can adjust its request and try again, and eventually
reach a point that both feel comfortable with the resource. You may say
this is not your favorite way of negotiation mechanism. But it is a form
of negotiation that is simple and effective.

> since copies are usually minimized.  

How the a_rwnd value is derived by the receiver is out of scope of SCTP
protocol. I.e., the receiver is free to use whatever method it views as
best to reflect its resource availability to calculate a_rwind. 

> Also

Not sure I understand your comments here. But isn't it the same in TCP
if the sender for whatever reason is sending you one byte long segments?
The ability of reneging acked data should really be viewed as a big plus
for SCTP if you are so concerned about receiver resoure management.

-Qiaobing

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 27 Dec 2002 13:49:51 -0700

Jim,

> the association if too many are requested

This was discussed on several occassions.  As the INIT must be
placed into one IP packet, its size (and thus the number of
addresses in the INIT, and cookie) are bound by the maximum size
of an IP packet for the route to the destination to which the
INIT is being sent.  Therefore, your "rule 2" is not violated:
the amount of resources is bounded.

> this state of affairs is not good.

No tricks required.  For your micro-managed embedded system,
fixed allocate resource for the maximum number of addresses that
can fit in an INIT chunk, for each association, if you want to
never face the situation where malloc comes up empty.  Of
course, fixed allocation will result in alot of wasted resource
under normal conditions, but that's what you always get from
fixed allocation: wasted resource.
> resort for managing receive resources.

In implementation this is not much different than TCP.  TCP also
uses receive buffer space for flow control, and also has a large
amount of state (more so with SACK and such) associated with
1000 packets of 1 byte each.  But, what's good for the goose is
good for the gander.  If this situation needs fixing, might I
suggest that you "fix" it for TCP first?

> does exactly the right thing.

Huh?  a_rwnd is for flow control, ECN is for congestion
control.  These are two different things.  Even with ECN
a_rwnd is still required for flow control, as ECN provides
no flow control.

I think you are confusing a_rwnd for cwnd here.

> is messy, however, and causes an unnecessary

SCTP requires that a TSN be accepted and acknowledged unless the
a_rwnd is zero (0).  An SCTP receiver is not allowed to accept
TSNs and not acknowledge them, nor is it allowed to acknowledge
TSNs without accepting them.  It now (see the imp guide) also
includes a zero-window probe procedure at both the sender and
received when a_rwnd is zero (0) to reliably re-open the window.

If you have (dynamic allocated) resource pressure at the receiver,
advertize an a_rwnd of zero (0) and discard packets until the
resource pressure abates.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Randall Stewart <[email-masked]>
Date: Fri, 27 Dec 2002 15:32:17 -0600

Some comments for you to think upon... :>

> the association if too many are requested

I don't see how this violates rule 1. SCTP provides a
basic "I can't do that .. not enough resources" in response
to something that can't be done. Yes the association will
not come up until the peer reduces the resources request...

There is only a couple of resources an INIT/-ACK can consume:

1) Addresses
2) a TCB

If it is #2 then you are out of luck and can never get
an assoc up.

If it is #1 retrying with less addresses may solve the
problem.. but this usually takes the application cooperation
since the address list is bound by the application not
by SCTP itself (with the notable exception of the wild-
card address).

When we designed SCTP we attempted to stay away from
a setup negotiation aka PPP. It was thought that doing
a negotiation can lead to loops of offered/refused and excess
delay in association setup.

It is arguable that one could define more error message aka
error cause "to many addresses" for example. If such were
defined then one could conceivably do the negotiation vis a via
INIT--->
<---abort(to many addresses)
INIT---->
... etc until an agreed upon number could be reached..

But the issue that we also tried to avoid here is defining error causes
for everything in the world. Yes I understand your concern but there
are arguably a lot of other error's that I have been asked about as
well.. and this (adding lots of error codes) too was attempted to
be avoided..

Do we need more error codes? maybe.. but I am as yet unconvinced...

> resort for managing receive resources.
The same is true for any protocol. There is some overhead
within the basic structure of how data is maintained. This
is true for TCP as well as SCTP. Since SCTP is record
oriented (a feature I believe RDDP actually wants to use)
there will be a larger overhead in this. Even TCP has constraints
normally somewhere around a 2k mark per mbuf-cluster at least in
BSD systems.. and you can get packets in 500 byte chunks
if that so happens to be what the peer is doing.. and now the
trade off becomes do you keep the data in records (mbufs) or
do you pack it down... Most BSD TCP implementations do a packing
when this occurs ... trading off the structures you mention
for cpu overhead of data copies...

One thing you must remember is that rwnd can collapse for
more than just data bytes.. on BSD systems I know both
the number of bytes that is being used AND the count
of mbuf resources come in to play and can actually
collapse the rwnd faster... in other words the
protocol overhead is accounted for... this can be
done in any implementation (I know my SCTP impl
also does this just like the TCP impl).

> to indicate congestion and throttle back the senders.

Two things here

1) ECN is defined to identify network congestion not
    receiver congestion. Using it for other purposes such
    as you propose is not according to the ECN RFC's

2) ECN WOULD have been mandatory for SCTP implementations if
    it would have been a standards track document. Unfortunately
    it was still experimental when SCTP went to the RFC editors
    queue so we had to "stand on our heads" so to speak to even
    get it in place at all... one of the things on my list
    as been to write a ECN for SCTP draft... sigh never enough
    time...

> does exactly the right thing.

I think you need to be a bit more specific here... you
are proposing something major aka using ECN for
flow control. A draft to clearify your idea would
be most appropriate I think.  I know, for me, I truely have
a hard time understanding how ECN would work better for
flow control than the rwnd. aka

------DATA-----> (rcver realizes its is running low
                   on resources (chunks/structures/?)
  <------SACK(rwnd=0)

or

-----DATA------> (same as above)
<-----ECNE (fabricated)----
-----CWR--------->

In the case of the rwnd=0 the sender will begin to probe
for space with a single chunk of data.

In the case of the ECN the sender cuts his cwnd but still
can send more data (in its view) based on the ECN RFC's. It
is thinking that there is NETWORK congestion NOT receiver
congestion.

I think I might understand this better if you had a draft to
futher describe this.. also I wonder how this interacts with
the ecn nonce...  It may be fine but I feel one needs to
think this through and make sure it does not cause some unexpected
problems...

> is messy, however, and causes an unnecessary

If a receiver has no resources to accept a new packet
I don't see how clamping a rwnd to 0 and possibly dropping
packets is the wrong thing.. this is precisely what TCP does
as well. Sending a ECN at this point will not improve things
if you have run out of resources... or am I missing your point?

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 27 Dec 2002 16:36:26 -0600

>the association if too many are requested

There are the limits Brian's response cited. But even
if an implementation wishes to set a lower limit, that
is quite acceptable. Limitations on "rule 1" are quite
common, and usually quite acceptable. For example, one party
to an InfiniBand connection can request a very large send
credit. That does not mean that all, or even most,
responders will accept that request.

But multi-homing is indeed where SCTP and DDP are not in
allignment. That is why the in-process draft specifically
allows a DDP/SCTP implementation to limit the number of IP
addresses supported on a single SCTP endpoint. It is almost
a requirement for efficient RNIC based DDP/SCTP.

However, if the other end has Software or Motherboard based
DDP/SCTP then they should be allowed to use multi-homing.
The multi-homing feature as defined for SCTP is ultimately
fairly similar to InfiniBand's Automatic Path Migration.
When used to that extent it should be just as compatible
with RDMA applications.

It would certainly be valid to argue that DDP/SCTP should
be limited to single-address endpoints. The first draft
probably will not make that recomendation, sticking with the
goal of being cleanly layered on top of SCTP. As currently
drafted a software DDP client running in user mode on top of
an existing SCTP stack is fully feasible. This is not
something that we would want to lose unless it has a severe
impact on hardware implementations.

>resort for managing receive resources.
SCTP credits act as the equivalent of link layer credits for
InfiniBand. They are credits governing only lower layer
buffering. The ULP is responsible for assuring that there
are sufficient DDP buffers to accept delivery.

A standard mapping of those DDP buffers to the advertised
SCTP window would be very difficult and undesirable. DDP
deals with two very distinct buffering models: tagged and
untagged. Neither the DDP nor SCTP layers have sufficient
information to know how much tagged data can be received.
Any attempt to predict untagged availability will run into
the problem that untagged buffer resources are stream
dependent, while the SCTP window is not.

While InfiniBand link layer credits are end-to-end, they
have a somewhat analogous relationship with the end-to-end
transport layer credits: i.e., one that should be ignored.

The SCTP layer should advertise its available raw buffer
capacity (which is larger than its buffers, because they can
be turned over rapidly). The SCTP layer must receive and
buffer an entire SCTP packet before delivering it to the DDP
layer. It must validate the packet or individual Data Chunks
first. Once it has done so it delivers them to the DDP
layer.

If the DDP layer determines that a tagged message is invalid
because it specified an invalid target, the DDP stream is
torn down. But this does not concern the SCTP layer.

If the DDP layer determines that an untagged message does
not have a pre-posted receive operation for it, the DDP
stream is torn down. Once again, this does not concern the
SCTP layer.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Qiaobing Xie <[email-masked]>
Date: Fri, 27 Dec 2002 18:34:05 -0600

...
> a requirement for efficient RNIC based DDP/SCTP.

SCTP does not force the use of multi-homing. It would be perfectly legal
if someone builds an SCTP stack on a tiny card on which there is only
enough hardware resource to support a limited number of singly-homed
associations. Such an implementation will only initiate associations
with its peers with one IP address and will always say "sorry, no
resource" to any attempt from a peer for a multi-homed association. I'd
never consider such an implementation a violation to the protocol. In
the contrary, I'd think this is the kind of flexibility a protocol
design should naturally have.

-Qiaobing

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 27 Dec 2002 19:57:29 -0600

On 12/27/02, Qiaobing Xie wrote:

>....
>> allignment. That is why the in-process draft
>> specifically allows a DDP/SCTP implementation to limit
>> the number of IP addresses supported on a single SCTP
>> endpoint. It is almost a requirement for efficient RNIC
>> based DDP/SCTP.
>
>SCTP does not force the use of multi-homing. It would be
>perfectly legal if someone builds an SCTP stack on a tiny
>card on which there is only enough hardware resource to
>support a limited number of singly-homed associations.
>Such an implementation will only initiate associations
>with its peers with one IP address and will always say
>"sorry, no resource" to any attempt from a peer for a
>multi-homed association. I'd never consider such an
>implementation a violation to the protocol. In the
>contrary, I'd think this is the kind of flexibility a
>protocol design should naturally have.
>

I think I fall somewhere between the position you're
advocating and the one Jim took that any two valid
implementations should be able to interoperate no matter
what resource requests each chooses to make.

Supporting an association with a multi-homed peer does
require more resources than one with a single-homed peer.
Some status has to be maintained for each remote address.

But "insufficient resources" should not be a cop-out for "I
didn't both to implement that". I would be extremely
reluctant to claim that it is proper for a host to claim
that it had "insufficient resources" to handle a single
association with a dual-homed peer when it was willing
to accept 100 new associations with single-homed peers.

Any implementation that does not have the resources to
accept an association may reject it (even if it is a matter
of resource-preserving policy rather than an actual
shortage), but claiming insufficient resources for something
that you totally lack the capability of doing is just
misleading the other host.

Would it be acceptable to implement SCTP without
multi-streaming and simply reject all associations
requesting more than a single stream in each direction?

Caitlin Bestler

---

From: Qiaobing Xie <[email-masked]>
Date: Fri, 27 Dec 2002 22:19:52 -0600

It is so common in any design that one needs to make trade-off between
the available resource and the capability of the implementation. In
fact, if you take a look at _any_ code, you would find a bunch of
#define MAX_NUM_SOMETHING here or there.

This is totally reasonable since after all, the _implementor_ is the one
who should be ultimately responsible to the usefulness of the final
implementation (unless the implementor has some malicious motivation for
not doing something in the spec, that would be a different story). I
believe an implementor will always try his best to make the right
trade-off and not to "fool" himself - if he is unwilling to commit
resource to support something, he must have his own good reason to do so
and he must understand that the usefulness of the implementation will be
impacted. 

As long as he does not call his limited-resource implementation general
purpose and does not tell people that that is the standard way to
implement SCTP stack, I do not sea a reason to prohibit it. 

-Qiaobing

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 28 Dec 2002 01:41:52 -0700

Qiaobing,

I agree that the protocol should provide the flexibility, but what you
describe hardly seems necessary.  Additional memory overhead for an additional
destination is less than 30 words of memory and could be made far less than
that with optimization.  This is far less memory than it takes to hold a
single MTU of a single SCTP packet.  If memory is so constrained as to not
allow a fraction of an MTU to be allocated to a destination, the
implementation will be sacrificing far more than multi-homing: that is, it
will be sacrificing fragmentation of data chunks greater than an MTU (because
not even one MTU can be removed from receive buffers), it will sacrifice flow
control performance (becaues a_rwnd and SWS avoidance will make flow control
non-linear), thus throughput would be dismal, etc...

It is fortunate that there is no such system (or, if you don't have enough
memory, wait a month or two).

I think the original point was that the number of IP addresses was unbounded.
As the INIT must fit in one IP packet, this is just not the case.

--brian

---

From: Caitlin Bestler <[email-masked]>
Date: Sat, 28 Dec 2002 10:18:14 -0600

On 12/28/02, Brian F. G. Bidulock wrote:

>
>I think the original point was that the number of IP
>addresses was unbounded. As the INIT must fit in one IP
>packet, this is just not the case.
>
Minor quibble: RFC2960 specifically warns that INITs and
INIT ACKs may be "quite large" (specifically larger than an
Ethernet MTU), but does not explicitly say that the receiver
should accept the maximum IP size. Realistically, "more than
1500 bytes" will hold a lot of remote addresses. The RFC
even cites 8000 bytes as an example. I think most would read
this as an implicit SHOULD, and support at least that size.

The bottom line for DDP over SCTP is that it is very
reasonable for an RNIC based DDP/SCTP to choose for itself
not to be multi-homed, but it should not make that choice
for its peer. Unless by some fluke it really is on its last
few bytes, it should not reject an INIT merely because the
prospective peer is multi-homed.

Caitlin Bestler

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 28 Dec 2002 14:48:56 -0700

Caitlin,

There is a maximum size to an IP packet regardless of
the MTU.

"A large number" is not "unbounded".

Therefore, Jim can design to his principles and you can
design to yours.

--brian

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 30 Dec 2002 15:55:51 -0600

Brian,

Besides memory, people working with hardware often need to worry about
other constraints, such as the number of logical gates used, etc. But in
general, when you build a high performance implementation for a specific
application area (such as the case of RDDP), part of the designer's task
is to optimize the resource utilization. This means that, depending on
what is important to that specific app area, some features/functions may
be supported fuller, while some may be minimally supported.

Back to my original scenario. If the h/w implementor feels there really
aren't enough logical gates available on the chip to fully implement
multi-homing support of the SCTP stack and the app that will use the
stack will only need singly-homed associations, it would be absurd to
tell the implementor "Sorry, you cannot use SCTP since you have to have
full support to multi-homing even you won't use it."

-Qiaobing

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 30 Dec 2002 17:18:18 -0600

On 12/30/02, Qiaobing Xie wrote:

>
>Back to my original scenario. If the h/w implementor feels
>there really aren't enough logical gates available on the
>chip to fully implement multi-homing support of the SCTP
>stack and the app that will use the stack will only need
>singly-homed associations, it would be absurd to tell the
>implementor "Sorry, you cannot use SCTP since you have to
>have full support to multi-homing even you won't use it."
>

The problem is that this limited NIC is not choosing to
limit itself, but rather to limit NICs that talk to it.

Allowing individual implementations to restrict their peers
in use of legitimate protocol options is a dangerous idea.

You could probably save a *lot* of gates for a SAN/SCTP
device that dealt only with 512 byte Data Chunks. Would that
be legitimate? I'm quite liberal on variant local
interfaces, so i'd have no problem with a specialized NIC
that chose to limit the Data Chunks it sent that way -- but
would you really allow an "SCTP" NIC to save gates and only
accept 512 byte Data Chunks?

If you really think that supporting multi-homing at the
other end is a severe imposition that is not appropriate for
all RNICs then lobby to have multi-homing support downgraded
to a SHOULD. Otherwise, don't go restricting your peer's use
of legitimate protocol options unless you are legitimately
out of *dynamic* resources.

Caitlin Bestler

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 30 Dec 2002 18:18:46 -0600

> limit itself, but rather to limit NICs that talk to it.

You are splitting a hair :-) If I lock myself in a room, does that means
I am not limiting myself, but I am limiting other's accessibility to me?
Yes, but what's the issue?

> accept 512 byte Data Chunks?

A) if the community consensus is to use variable size chunks but you
choose to go against community consensus and build a NIC that only
accepts 512 byte chunks. Then your NIC will have big problem inter-work
with others (and I will not buy your NICs :-). If you still want to do
that for whatever reason, I think you are always allowed. But you will
not call it standard since you do not have consensus on your side.

B) if the community consensus is to use 512 byte chunks only, then you
have no problem and you get a standard. This is what RDDP WG is supposed
to do. In such a case, RDDP would have a RFC saying something like:
"...the layer about SCTP MUST only pass 512 byte data messages to SCTP
stack..." When building a RDDP NIC, one then could choose to either use
a general purpose SCTP stack or create a customized SCTP stack on the
NIC that only handles 512 data chunks, with the understanding that this
SCTP stack is NOT general purpose and it only works with RDDP
applications, and you will not call this customized implementation full
compliant to RFC2960 since it isn't. 

Either way I don't see any issue here.

> to a SHOULD. 

IMO, to be called a full compliant RFC2960 stack, you MUST do the MUSTs
in the spec. But this does not mean one is banned from building his/her
own not-full-compliant (i.e., customized) SCTP stack for his/her own
special app uses. But such implementations should never be put on the
"big I" Internet.

> Otherwise, don't go restricting your peer's use

This sounds like "you are not allowed to say 'no' for your own reasons,
but only for reasons I allow you." I think this is quite unreasonable,
unenforceable and unnecessary. Whenever a peer rejects some options, I'd
always trust there is a good reason behind the rejection.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

I agree with you, and I wasn't saying that.  What I was saying is
that the number of destinations in not "unbounded".

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

Qiaobing,

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

<<< No Message Collected >>>
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 30 Dec 2002 15:06:10 -0700