Subject: Multicast draft: Group/Join delay

From: [email-masked] (Kevin Dubray)
Date: Mon, 9 Mar 1998 17:10:50 -0500

Group,

With respect to item 3.5.1, Group Join Delay of the 
Multicast benchmarking terminology draft <draft-ietf-bmwg-
mcast-02.txt>, I've received some comments.

One set of comments articulated that the Join/Leave Delay metrics
are of limited utility because they essentially measure a protocol
response.  As such, these metrics are not a good basis for comparison because
there is a perception that, in a controlled environment, there will be
negligible differences between implementations.

The other set of comments reflects that because of the many variables
(e.g. protocol configuration, physical topology, "tree" topology, etc,) 
that could possibly affect the outcome of such a metric, the metric
itself becomes meaningless.

Comments?  Should the Group Join/Delay metrics be deleted from the
draft?  If not, is there a way that these metrics can be made more meaningful.

In an unrelated comment to item 3.5.1, a suggestion was made that it would
be useful to have an opening remark about the nature of multicast 
characterization.  It should go along the lines of, "if you thought unicast
benchmarking is tough, multicast benchmarking is even tougher.  Here's why..."

Would an introductory paragraph in the spirit of the above be useful?  If
so, what would you cite as some of the issues that distinguish multicast
benchmarking from its unicast analog?

Thanks,
Kevin

---

From: Zohar Levy <[email-masked]>
Date: Wed, 11 Mar 1998 16:37:27 +0200

Kevin,

-----Original Message-----
From:	[email-masked] [SMTP:[email-masked]]
Sent:	Tuesday, March 10, 1998 00:11
To:	[email-masked]
Subject:	Multicast draft: Group/Join delay

> negligible differences between implementations.

I think that the differences are not necessarily negligible when the 
DUT is offered with high load. If the
DUT is processing traffic at very high load, and during that time it 
receives an IGMP request, the time
it will take until it processes this message may highly depend on 
whether the DUT can recognize such a message
at an early stage and process it. If there is no special priority to 
IGMP messages and the device  for example
implements some sort of input queueing, the high load may cause 
problems such as HOL blocking and even
packet loss which may delay the treatment of IGMP packets. This might 
happen both when the load is offered on the
same port as the incoming port of the IGMP packet  (the IGMP packet 
might be dropped like other packets) or if
the load is offered on other ports, in which case the time until the 
IGMP packet is processed depends
on the input-queue scheuling of the DUT (fairness).

To summarize, although input-queueing, HOL blocking or any type of 
packet loss are problems not specific to
IGMP processing, different devices that suffer such problems may be 
differentiated according to whether they are
able to recognize IGMP packets in advance and treat them in a special 
manner. Therefore, I think it is meaningful
to test this metric under heavy (or full) load conditions.

> itself becomes meaningless.

Perhaps the test of the metric should be limited to cases where the 
orginator of the request is directly connected
to the DUT (on a physical link). If a more general system needs to be 
tested, then the topology should be specified
as a test parameter.

I would also like to note that the join/leave delay tests should 
consider only requests that change the forwarding state
of the port towards the request originator. For example, if a join 
request for a group G is originated from a device that resides
in a LAN,  and there are already other devices receiving draffic for 
group G, then the request may have no effect on the
DUT, since it will already be forwarding traffic towards the 
reqeuster's LAN. In this case, the test is meaningless. Same goes
for leave requests.

> benchmarking from its unicast analog?

Regards,
      Zohar

===============================
     Zohar Levy,
     Digital Semiconductor Design Center,
     Jerusalem, Israel.
     e-mail: [email-masked]
       Phone: ++972-2-5892418
       FAX: ++972-2-5892600/1

---

From: [email-masked] (Kevin Dubray)
Date: Thu, 12 Mar 1998 15:12:17 -0500

>From [email-masked] Thu Mar 12 14:06:03 1998
To: [email-masked], [email-masked]
Subject: RE: Multicast draft: Group/Join delay

> draft?  If not, is there a way that these metrics can be made more
	I agree that measuring these delays per se may not add much
value. A background set of join/leave operations though is needed to
ensure that implementations able to sustain fast forwarding rates are
also able to process the membership and route management overhead when
scaling to larger groups and networks. The analogy in unicast
benchmarking is a background SNMP and routing protocol overhead.