Subject: [Tsvwg] Re: [SCTP]Congestion Control Parameters

From: Andrea Senatore <[email-masked]>
Date: Mon, 28 Oct 2002 20:05:21 +0100

Tuexen Michael wrote:

>Hi Andrea,
Hi Michael,

>
>
>this would mean that you have m * n path if one peer has
>n the other m IP addresses. Using the current terminology
>has only n + m. This was considered less complexity.
>
You could use fixed paths associating every node of interface to one 
destination.
So you has only min(n,m) paths...

>
>
>The other reason is that normally the IP layer selects the
>source address of the packet according to its forwarding table.
>Selecting the source address at the SCTP layer (called source
>selection) is not very easy and even if you select an IP-address
>as a source address the IP layer may send it out on an interface
>which has not bound that address. In the bakeoff some implementations
>showed up with using source address selection and it came out
>that it is a difficult task to get that stuff right.
>
Sorry I did not know these issues...but I think that a good congestion 
control mechanism
should keep congestion parameters for each used path.

>
>
>In general, the forwarding in an IP network is mostly based on
>the destination address, not on the source address.
>
>These are some of the reasons why this notion was choosen.
>
....... :-/

>
>
>I know of some people working on the path notion which is defined
>as a pair of source and destination addresses and it is not a trivial
>part. 
>
>Best regards
>Michael
>
>Michael Tuexen   Tel.:   +49 89 722 47210
>Siemens AG       Fax:    +49 89 722 48212
>ICN WN CC SE 7   e-mail: [email-masked]
>
>>-----Original Message-----
>>From: Andrea Senatore [mailto:[email-masked]]
>>Sent: Monday, October 28, 2002 5:05 PM
>>To: Tuexen Michael
>>Subject: Re: [SCTP]Congestion Control Parameters
>>
>>
>>Tuexen Michael wrote:
>>
>>>Hi Andrea,
>>>
>>>it is a definition, so it is not correct or incorrect. That
>>>is the term used.
>>>
>>>What 'behaviour' of SCTP are you thinking to be not correct?
>>>
>>IMHO  "congestion control parameters"  would be kept for each 
>>source-destination pair  (i.e. sender interface <----------> receiver 
>>interface) and not for
>>each destination as written in RFC.
>>These parameters depends on destination only when the sender has a 
>>single interface.
>>Best Regards,
>>
>>-- Andrea
>>========================================================
>>Andrea Senatore,     
>>Mail:    [email-masked],
>>Address: CoRiTeL c/o Università degli Studi di Salerno,
>>         Facoltà di Ingegneria,  via Ponte Don Melillo,
>>         84084 - FISCIANO (SA) - Italy
>>Tel:     +39 089 964012
>>Fax:     +39 089 964010
>>========================================================
>>
>>
>>
>
>
Thanks Michael,
best regards ,

-- Andrea
========================================================
Andrea Senatore,     
Mail:    [email-masked],
Address: CoRiTeL c/o Università degli Studi di Salerno,
         Facoltà di Ingegneria,  via Ponte Don Melillo,
         84084 - FISCIANO (SA) - Italy
Tel:     +39 089 964012
Fax:     +39 089 964010
========================================================

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 28 Oct 2002 23:35:23 -0700

Andrea,

What definition do you use for "path"?  And, how do you ensure
that such a definition of "path" has meaning in the underlying
network?

This was one of the difficulties in the original SCTP work.  Do
you use destination IP address?  Do you use source-destination
pairs?  Do you use interfaces?  Interface pairing?

In general there is not a definition of path which suits all
networks and all circumstances.  The approximation to treat all
paths as a single path with variable characteristics is a usable
compromise.

--brian

On Mon, 28 Oct 2002, Andrea Senatore wrote:
> control mechanism should keep congestion parameters for each used path.
> 

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 29 Oct 2002 03:51:16 -0700

Andrea,

On Tue, 29 Oct 2002, Andrea Senatore wrote:
>
> I believe that source-destination IP pairs identifies the path better 
> than only destination IP address.

How so?  Source address typically has no bearing on the path
taken by a packet through an IP network (nor on which interface
the outgoing packet is sent).

> Why need I use different congestion control parameters for different  IP 
> destination in SCTP ?

Because the only valuable use of SCTP multi-homing is to have
each destination IP address correspond to a different interface
on the destination host, which is, indeed, a different path for
each destination address.

> Because if I change the IP destination address, probably the path 
> changes, and then all congestion control parameters MUST change.

The useful application of SCTP multi-homing means that the path
does change when the destination address changes and thus, per
RFC 2960, the congestion control parameters MUST change.

> But the path changes also if  I change IP source address (if is it 
> possible..)  and, for the same reasons,  all congestion control 
> parameters MUST change.

No.  Path does not change when IP source address changes.  The
IP source address is merely a return address.  In a typical IP
stack or router, changing the source address has no bearing on
the path taken by the packet.  Only destination address does.
Source address has a bearing on the return path for packets, but
CC on the return path is the business of the peer.

> This is the reason for which I say that a good congestion control 
> mechanism should keep congestion parameters for each used path..

The host will chose the outgoing interface based on the
destination address and will not consider the source address
when selecting the outgoing interface.  Some IP implementations
permit routing based on source address, but this is atypical.
Regardless, it is outgoing interface and destination address
that determines path, not source address.

In the absence of failures and network reconfigurations, it is
true that destination address alone determines path.

--brian

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 29 Oct 2002 15:05:33 -0600

On 10/29/02, Brian F. G. Bidulock wrote:

>> I believe that source-destination IP pairs identifies
>> the path better than only destination IP address.
>
>How so?  Source address typically has no bearing on the
>path taken by a packet through an IP network (nor on which
>interface the outgoing packet is sent).
>

The definition of 'Primary Path' in the RFC appears to
disagree:

> o  Primary Path: The primary path is the destination and
> source address that will be put into a packet outbound to
> the peer endpoint by default.  The definition includes
> the source address since an implementation MAY wish to
> specify both destination and source address to better
> control the return path taken by reply chunks and on
> which interface the packet is transmitted when the data
> sender is multi-homed.

The wording implies that an implementation is free to
perform local routing on the combination of source and
destination address, rather than merely on the destination
address.

Clearly, performing local routing based upon destination
alone is an acceptable implementation. So clearly,
congestion based upon destination alone must be an option.

That still leaves the seperate question of whether an
implementation that does differentiate physical interfaces
by source address SHOULD also track congestion seperately.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 29 Oct 2002 16:34:03 -0700

Caitlin,

> > which interface the packet is transmitted when the data

This definition has little to do with the "path" taken
through the network from host to host.

> destination address, rather than merely on the destination

Source routing is the stuff of research.

> congestion based upon destination alone must be an option.

In fact it is a requirement.

> by source address SHOULD also track congestion seperately.

A multi-homed host is performing part of the functions of
a router.  A transport protocol can no more be expected to
track the routing changes in a router local to the host than
it can be expected to track routing changes in a router remote
from the host.   A router internal to the network could change
the "path" at any point in time.  Exponential closing of the
congestion window is sufficient for remote routers (even in TCP),
why would it not be sufficient for local routers?

Compare to TCP which on a multihomed host can change the forward
path by routing packets out an alternative interface.  (The only
difference is that TCP might not be capable of changing the
reverse path (it only has one source address bound at the time
that the connection is formed).)  TCP does not alter its congestion
window when an alternate outgoing interface is selected by the
underlying routing functions in the host.

Nevertheless, RFC 2960 already provides for congestion procedures
more conservative than those specified.  If your implementation can
detect change of outgoing interface and apply slow-start, or at
least a tighter congestion window than previously assigned to the
destination, then go ahead.

--brian

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 29 Oct 2002 18:59:40 -0600

On 10/29/02, Brian F. G. Bidulock wrote:

>> >path taken by a packet through an IP network (nor on
>> >which interface the outgoing packet is sent).
>> >
>> 
>> > o  Primary Path: The primary path is the destination
>> > and source address that will be put into a packet
>> > outbound to the peer endpoint by default.  The
>> > definition includes the source address since an
>> > implementation MAY wish to specify both destination
>> > and source address to better control the return path
>> > taken by reply chunks and on which interface the
>> > packet is transmitted when the data sender is
>> > multi-homed.
>

No, but it recognizes that it is a valid practice to
have a multihomed host that has different interfaces to
different local subnets. It provides redundant subnets, not
merely redundant interfaces.

>Source routing is the stuff of research.

Source routing on the first hop (i.e. which interface the
packet exits via) is far from research, it is a standard
firewall practice. Only packets with the correct source
address will be sent to a given subnet.

There are many reasons for differentiating interfaces. The
most common is not relevant here: when a host connects to
two networks that are otherwise isolated (i.e. external and
internal).

But other SCTP relevant reasons include differential
capacity. For example, high priority applications could
prefer the source IP address associated with the higher
speed interface.

>track the routing changes in a router local to the host
>than it can be expected to track routing changes in a
>router remote from the host.   A router internal to the
>network could change the "path" at any point in time. 
>Exponential closing of the congestion window is sufficient
>for remote routers (even in TCP), why would it not be
>sufficient for local routers?
>

Yes, and some routers are statically configured for very
valid security and policy enforcement purposes. It is even
more common for a multi-homed host to have logically
differentiated interfaces. Some examples: a web server that
has an interface leading to clients and anther leading to
file servers, a CMTS or DSLAM that has an interface leading
to the net and another leading to end-client homes.

The RFC is fully compatible with these scenarios as is. But
in debating congestion you are arguing that all hosts are
somehow obligated to follow a interface-indifferent
strategy.

The Source Address can be used by many applications as a
"virtual interface". It has a stable meaning, which can be
mapped to the appropriate physical interface. This is a
valid mode of operation for any application, and it is the
default for RDMA applications.

>Compare to TCP which on a multihomed host can change the
>forward path by routing packets out an alternative
>interface.  (The only difference is that TCP might not be
>capable of changing the reverse path (it only has one
>source address bound at the time that the connection is
>formed).)  TCP does not alter its congestion window when
>an alternate outgoing interface is selected by the

TCP is bound to a single source IP address. To switch
between alternate source IP addresses the ULP would have to
create the equivalent of an association out of multiple TCP
connections. It is totally different.

Caitlin Bestler

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 29 Oct 2002 18:08:46 -0700

Caitlin,

> connections. It is totally different.

It is not different in that TCP can route outgoing packets
via an interface which does not have the source address bound
to the connection (routing on destination address).  This results
in a different path out of the host, but TCP does not adjust nor
accomodate this change in path when considering congestion.
SCTP is (intentionally) no different than TCP in this regard.

--brian

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 29 Oct 2002 19:26:30 -0600

On 10/29/02, Brian F. G. Bidulock wrote:

>via an interface which does not have the source address
>bound to the connection (routing on destination address). 
>This results in a different path out of the host, but TCP
>does not adjust nor accomodate this change in path when
>considering congestion. SCTP is (intentionally) no
>different than TCP in this regard.
>

Under TCP, a packet has a fixed source IP address.
Therefore, no matter which interface it exits via the first
place it goes is the same subnet (at least logically). It is
quite logical to assume that the set of subnets it will
traverse to reach the destination will be the same from
there. So keeping the same congestion information makes a
lot of sense.

Under SCTP, a shift in the source address can mean that the
first hop will be a different subnet (logicqally and
physically). The entire purpose of shifting the source
address can be to force a different path to be selected.

Caitlin Bestler

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 29 Oct 2002 18:59:03 -0700

Caitlin,

Although SCTP provides for interface redundancy, is no more
intended to provide network redudancy than is TCP.  This is
because the Big I is one network.  If you have two separate
and distinct subnets between hosts, it is not an internet,
in which case, according to the applicability statement, CC
is permitted to be defeated or modified to the implementation's
liking.  The Internet, however, is one network, and the
CC procedures and requirements in RFC 2960 are well suited to
this environment.

--brian

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 30 Oct 2002 09:18:39 -0600

On 10/29/02, Brian F. G. Bidulock wrote:

>Caitlin,
>
>Although SCTP provides for interface redundancy, is no
>more intended to provide network redudancy than is TCP. 
>This is because the Big I is one network.  If you have two
>separate and distinct subnets between hosts, it is not an
>internet, in which case, according to the applicability
>statement, CC is permitted to be defeated or modified to
>the implementation's liking.  The Internet, however, is
>one network, and the CC procedures and requirements in RFC
>2960 are well suited to this environment.
>
>--brian
>

SCTP is an IP protocol not a "Internet" protocol. IP is
widely deployed on private networks as well "the Big I".
Indeed those "distinct seperate subnets" you seem to be so
dismissive of are more likely to find uses for SCTP. Two
osts within a data center or server farm are far more likely
to have a large number of logical connections with each
other than two machines connected to "the Internet". 

Multi-tiered architectures, featuring levels of segregated
networks, are commonly deployed for high volume web serving
farms. There is a network for dealing with clients ("the Big
I"), one for dealing with application servers, frequently a
third one for storage and sometimes a totally distinct one
for administrative control.

Distinquishing packets as originating on "trusted" or
"untrusted" networks is invaluable for doing this without
locking functions to physical ports, which is clearly
undesriable. This can be done if source IP addresses are
taken seriously -- which means no host should be originating
packets with a Source IP address that should not be
originating there.

None of these solutions work if you rigourously stick with
the fiction that the Internet is "one network" and that all
routing, even the first hop, is opaque and off limits to the
application layer.

More importantly, these server cluster environments that
feature multiple  segregated networks are precisely the
environments where the outer ring machines (Web Servers)
have multiple concurrent sessions with the inner ring
machines (applications, database and file servers) and could
greatly benefit from not having the overhead of a TCP
connection for each and every session. But there would be no
chance of obtaining the potential benefits of SCTP in these
environments if its deployment required loosing distinction
between the different local networks.

No change is required, it is compatible now. But when
discussing possible new features it is important to remember
that "IP" does not alway mean "The Internet".

Caitlin Bestler

---

From: Randall Stewart <[email-masked]>
Date: Wed, 30 Oct 2002 10:06:49 -0600

Caitlin:

I have read through this thread and have just a few comments...

1) Brian is correct in that source address routing is more
    of a research issue.

2) Brian is also correct in that SCTP requires you to have
    a cwnd per destination address... not per source/destination
    address pair.

3) I don't see how keeping a seperate cwnd for two different
    sources would help you much.. The source is going to
    allow you to get a different return path for the SACK...SACK's
    are not limited by cwnd... so what benefit do you get by
    attempting to keep two sets of cwnd parameters for say a
    address IP-A?

4) You are most likely to get edge filtered if you select a
    source address that is different than your interface that you
    emit the packet on... most all ISP's do filtering at the
    edge to help protect against DOS attacks.

5) In a private network (such as you allude to) you could emit a
    packet to IP-A with differing source addresses.. this may help
    you with the return path but how does it help you with the forward
    path. Your kernel will lookup the route and emit the packet via the
    way your routing table is indicating... so for example if you have
    IPX and IPY on intf-1x and intf-2y and you send to a peer with a
    destination that the route will yeild out intf-2y.. it will not
    matter which source address you stamp in.. IPX or IPY it is still
    going to go out intf-2y. Yes your sack will come back a different way
    (assuming you are not stuck with #4 above.. in which case the packet
     would be dropped ..edge filtered).. but the "path" through the
    network is going to go out the intf-2y way .. and routers in-between
    will follow their destination as well... placing IP-X or IP-Y should
    in all but a few specialized cases not change the way the packet goes
    through the network...

6) The few specialized cases I mention in 5 would be if you were playing
    with source address routing in your network... something that is
    generally turned off in any production network. If you are
    worried about firewalls then yes... the src address may dictate
    if the sender is trusted... but it is far more likely that the
    arrival interface is what will determine this... I know my firewall
    rules are setup to look at arrival interface as well as source...
    after all I would not want a packet from the internet arriving on
    intf-inet to be put in my network if its source was a 10
    based address

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Andrea Senatore <[email-masked]>
Date: Tue, 29 Oct 2002 10:21:58 +0100

>Andrea,
Hi Brian,

>pairs?  Do you use interfaces?  Interface pairing?
An endpoint cannot control completely the path used to send packets 
(i.e. all routers), but can choose the source and destination IP address.

Best Regards,

-- Andrea

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 31 Oct 2002 16:40:55 -0600

>1) Brian is correct in that source address routing is more

Source routing was never an issue. The question was whether
the source address would have an affect on the path
selected. Brian implied that it would not.  I belive this is
a very incorrect argument. If source addresses are for
different subnets then the set of interfaces that can
transmit them can easily be different, and presumably they
normally are.

In clustering terminology: it is important to deal with
redundant access networks, not just redundant interfaces.

> a cwnd per destination address... not per
> source/destination address pair.
>

Certainly that is how it is drafted. I also agree with it.

The ultimate justification is that even the full combination
of source and destination does *not* specify a path over an
IP network. The path is intrinsically fuzzy.

Consider that it is not a 'real' path, and that in many
configurations the paths will actually be identical (because
the host is only configured for redundant interfaces), and
in most the bulk of the path will be the same after the
first hop anyway.

The benefits of tracking the *seperate* source/destination
pairs are just not likely to result in valuable data.
Splitting the available data by source address will result
in less usable data. Working from destination only makes
sense.

It *might* make sense to optionally supplemnent this by
attempting to measure first hop congestion when there
multiple distinct options on the first hop and the choice of
source IP address would affect the first hop. Am I correct
that an implemtation would be free to do so?

>3) I don't see how keeping a seperate cwnd for two
> different sources would help you much.. The source is
> going to allow you to get a different return path for the
> SACK...SACK's are not limited by cwnd... so what benefit
> do you get by attempting to keep two sets of cwnd
> parameters for say a address IP-A?
>
Agreed, for the reasons given in the prior section.

> source address that is different than your interface that
> you emit the packet on... most all ISP's do filtering at
> the edge to help protect against DOS attacks.
>

Yes, which is one reason that I reacted negatively to the
implication that consumer specification of a preferred
source address would have no impact on the path.

>5) In a private network (such as you allude to) you could
> emit a packet to IP-A with differing source addresses..
> this may help you with the return path but how does it
> help you with the forward path. Your kernel will lookup
> the route and emit the packet via the way your routing
> table is indicating... so for example if you have IPX and
> IPY on intf-1x and intf-2y and you send to a peer with a
> destination that the route will yeild out intf-2y.. it
> will not matter which source address you stamp in.. IPX
> or IPY it is still going to go out intf-2y. Yes your sack
> will come back a different way (assuming you are not
> stuck with #4 above.. in which case the packet would be
> dropped ..edge filtered).. but the "path" through the
> network is going to go out the intf-2y way .. and routers
> in-between will follow their destination as well...
> placing IP-X or IP-Y should in all but a few specialized
> cases not change the way the packet goes through the
> network...
>
Firewall/packet filter rules are fully capable of specifying
the disposition of a packet based upon the source address.
In fact they SHOULD block any packet with an incorrect
source address from being transmitted.

>6) The few specialized cases I mention in 5 would be if
> you were playing with source address routing in your
> network... something that is generally turned off in any
> production network. If you are worried about firewalls
> then yes... the src address may dictate if the sender is
> trusted... but it is far more likely that the arrival
> interface is what will determine this... I know my
> firewall rules are setup to look at arrival interface as
> well as source... after all I would not want a packet
> from the internet arriving on intf-inet to be put in my
> network if its source was a 10 based address
>
With TCP and UDP the application simply binds the source
address, and that dictates the interface that packets can be
sent from. Automatically "routing" the packet ot the other
port would not be allowed unless the host had enabled
routing. That is precisely what you do *not* want a host
that is connected to two seperate networks to do. If routing
is allowed between the two networks it will be handled by a
router, not by a multi-homed application server.

I'm not claiming these are typical network designs, but they
have been true of the last two industries I've done IP
networking in. By contrast I've never actually had a project
where the "network" was the vague amoprhous "Internet" that
was interfaced to with multiple ports and no further
information was required.

Caitlin Bestler