Subject: [icnrg] Comments on the recently posted draft-anilj-icnrg-dnc-qos-icn-01

From: "David R. Oran" <[email-masked]>
Date: Sun, 15 Sep 2019 09:53:14 -0400

These are all with <chair hat off>

Many thanks for the continued work on this. It helps a lot to inform our 
development of QoS capabilities for NDN and CCNx in ICNRG. I have three 
sorts of comments. First are general/architectural comments. These are 
directly below in the email. Second are detailed technical comments, 
some of which emerge directly from the architectural questions; others 
are about specifics of the proposed design. Third are some editorial 
comments, typos, and nits. The latter two sets are embedded in the copy 
of the draft itself, which I append to the message.

##General/Architectural comments##
I doubt you will be surprised by these comments, as I’ve made most of 
them multiple times on both the earlier version of this draft, and in 
general discussions at ICNRG. I do think they are important enough to 
keep repeating since they cut to the core of what a QoS design for NDN 
and CCNx should look like. They also relate closely to the material in 
the recent draft I posted: 
https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/ so reading 
that first will help folks (not just the authors) understand better 
where I’m coming from.

Here you go:

1. The design couples flow classification and QoS treatment into a 
single mechanism, the *QoS Marker*. I think this is a quite serious 
mistake. I cannot find a single good reason do do this, because flow 
classification is a property of the named objects in the namespace and 
how they relate to each other, while QoS treatment is a property of a 
particular fetch operation by an individual consumer. The flow 
classification persists independent of who the consumers are and what 
they are doing. The QoS treatment for the same content object could 
differ substantially when fetched by different consumers. Combining 
these just seems wrong.

2. Placing the QoS marker (as currently defined) as part of the name is 
also a mistake in my view. It complicates name parsing and involves what 
I consider to be problematic changes to both PIT and CS data structures 
(and possibly the FIB as well). I give what I think are convincing 
arguments in the QoSArch draft for why QoS treatment should be entirely 
independent of names. For flow classification, the most important 
question is whether the flows are bound into the security envelope of 
the content object or not. The flow classification draft presents both 
approaches and arguments for and against each. Your draft doesn’t seem 
to capture these issues in a way that adds to the understanding or 
informs the tradeoffs. It could be that both mechanisms of flow 
classification might be needed. By being clear about what is bound 
securely to a name, what is hierarchically decomposable (if you need 
flows and sub flows) and what can be aggregated by routers seems a key 
architectural decision that you can’t defer to some later detailed 
design discussion by defining QoS markers the way you do.

3. Much of the draft tries to make a distinction between “routable 
name prefixes” and “non routable name components”. No such 
distinction exists in either the semantics of CCNx or NDN names, or in 
their encoding. A deep feature of both designs is the notion that 
something is routable if it, or a prefix thereof, has a FIB entry. 
Period. The namespace designer does not have to pre-determine what part 
of the name is no longer useful for routing and specify it differently. 
Going with a QoS Marker approach with the marker part of the name 
introduces some pretty deep changes (at least as I see it) and will 
cause QoS-related things to permeate lots of parts of the design 
unrelated to QoS, such as name parsing, PIT structure, etc.

##Technical and Editorial Comments##
[I have snipped out parts of the draft I have no comments on, including 
the page break filigree and other artifacts of the .txt rendering]. My 
comments are all preceded by <do> and terminated with </do>.

        QoS Treatments in ICN using Disaggregated Name Components
                     draft-anilj-icnrg-dnc-qos-icn-01

Abstract

    Mechanisms for specifying and implementing end-to-end Quality of
    service (QoS) treatments in Information-Centric Networks (ICN) has
    not been explored so far.  There has been some work towards
    implementing QoS in ICN; however, the discussions are mainly 
centered
    around strategies used in efficient forwarding of Interest packets.
    Moreover, as ICN has been tested mainly as an IP overlay, it's QoS 
is

<do>s/it’s/its/ </do>

    still governed by the IP QoS framework and there is a need for
    implementing QoS in ICN natively.  This document describes 
mechanisms
    to classify and provide associated "name-based" extensions to
    identify QoS within the framework of ICN's core design principles.
    The name-based design provides a mechanism to carry QoS information
    and implement the treatments as ICN packets travel across different
    routers in the ICN network.  Detailed discussion is provided for QoS
    specific procedures in each of the ICN network elements i.e.
    consumer, producer and forwarder.

1.  Introduction

    Information Centric Networking (ICN) is rapidly emerging as an
    alternative networking mechanism for the TCP/IP based host-centric
    networking paradigm.  Use cases of video conferencing [MPVCICN] and
    real-time streaming [NDNRTC] signify the value of ICN architecture
    and have been studied in detail.  Also, a number of studies on
    routing of Interest and flow classification [ICNFLOW] have been
    published; however, relatively less work has been done on end-to-end
    quality of service (QoS) architecture for ICN.  QoS is important to
    deliver preferential service to a variety of traffic flows resulting
    into better user experience.  Evaluation and study of prior work 
done

<do> s/into/in a/ </do>

    in this area is provided in Section 3.  The current QoS
    implementation is based on either Layer-3 TOS or DiffServ, which is
    applicable only for ICN as an overlay.  The QoS mechanisms described
    in this draft are applicable to the native ICN architecture.  A
    detailed discussion related to the design of name-based QoS 
encoding,
    which is in line with ICN's fundamental design principles.

<do>This last sentence is not a sentence - no verb</do>

3.  Prior Work and Motivation

3.1.  Prioritization of Interest Packets

    Among the work related to the quality of service (QoS) requirements
    in ICN network, larger emphasis is given to an optimized and
    efficient routing of Interest packets towards its content.

    M.F.  Al-Naday et.al. in [NADAY] argues that information awareness 
of
    ICN network would help build scalable QoS model.  In the context of
    CCN/NDN network design, authors points to the possibility of using

<do> s/authors points/the authors point/ >/do>

    the QoS aware name prefixes, with potentially limiting the third
    parties (e.g. network operators) from defining an alternative QoS
    enforcement mechanisms.  Moreover, the QoS solution is developed
    around the PURSUIT architecture, which may not be applicable to CCN/
    NDN.

    Weibo Chu et.al. in [WEIBO] present a QoS model for ICN network 
based
    on the popularity ranking of the content and its placement/location
    in the network.  They present a classification of the content into
    three categories - locally cached, remotely cached, and uncached
    contents, hence the network delay is modeled as a function of the
    distance of the content from the requester.  Essentially, the QoS
    problem is seen in the perspective of faster routing of Interest
    request towards its content.

    In [XINGWEI] authors present a QoS mechanism, which is applicable to
    the routing of Interest requests in ICN network.  The basis of the
    proposal is to decide the suitability of the forwarding link to make
    the process more energy efficient.  They use the same Data 
forwarding
    algorithm specified in the original NDN design [JACOBSON].

    In [CHRISTOS] Christos et.al. argue about a need for a 
differentiated
    routing and forwarding mechanisms based on not only the name of the
    content but also specifying the nature of the traffic.  They further
    emphasize that this differentiation is better handled at the network
    level rather than leaving it for the upper layer.

    In all above use cases, the QoS related discussions are mainly
    focused on the forwarding of the Interest requests.  It assumes that
    Data packets are forwarded in downstream direction according to the
    pending Interest table (PIT).  There is little or no discussions
    provided about how preferential treatment can be implemented and
    enforced in the Data packet path.

<do>The hop-by-hop interest shaper paper (Wang, Yaogong and Rozhnova, 
Natalya and Narayanan, Ashok and Oran, David and Rhee, Injong: An 
Improved Hop-by-hop Interest Shaper for Congestion Control in Named Data 
Networking, ACM ICN 2013) in fact shows how by shaping interests, the 
returning data packets are controlled and shaped, hence there is work on 
how if you have preferential treatment of Interests, preferential 
treatment of the returning data is achieved automatically </do>

3.2.  Flow Classification in ICN

    Flow classification [ICNFLOW] describe the methods for 
classification
    of data flows in ICN.  The method called equivalence class component
    count (EC3), uses a predefined number of name components of the

<do> the count is not “predefined”. It is set by the producer when 
it creates the data packet containing the corresponding content object 
(not necessarily when the content object itself is created). It can also 
be included in an Interest and set by the consumer (or aggregated by a 
forwarder) which is by no means “predefined” either. </do>

    content name forming an equivalence class.  While approach has the
    flexibility in re-grouping of components in aggregating the flows, 
it
    suffers from an inconsistent interpretation due to implicit binding
    of the equivalence class into the content namespace.  In the second

<do> I don’t understand what you’re saying here. What is 
inconsistent about the interpretation? By your logic, **any** flow 
aggregation would introduce inconsistencies since the aggregation is 
done by an element that is not in charge of the namespace. </do>

    method called equivalence class name component type (ECNCT), the 
flow
    classification information is directly embedded in the hierarchical
    content name.  This paves the way to achieve implicit aggregation of

<do>It seems you I and don’t have the same definition of 
“aggregation”. By my definition, aggregation means mixing multiple, 
possibly unrelated, flows together when maintaining flow state (and when 
enqueuing/dequeing packets for transmission). Using my definition ECNT 
does not do any implicit aggregation. It may permit hierarchical flow 
decomposition, but that’s something entirely different. </do>

    data flows analogous to the prefix-based aggregation of content 
names
    in FIB table.  At the forwarder level, a flow table is implemented 
to
    store the equivalence classes and is used to perform the flow
    classification decisions.

<do>this part I mostly agree with, however there are may ways to realize 
a “flow table”, some of which use an independent data structure with 
per-flow entries, while others do not (e.g. SFQ) </do>

    While both the schemes provide data flow classification in ICN, they
    are not sufficient for implementing a preferential service treatment
    in the network.  Table 1 provide a summary of important differences
    between the flow classification and QoS treament implementation.

<do> s/treament/treatment/ </do>
<do> The above statement, while true, gives the impression that there is 
some deficiency in the flow classification design of EC3  or ECNT that 
needs to be remedied. That is not so, as if you adopt an architecture 
that cleanly separates flow classification from QoS treatment there is 
nothing missing. QoS treatment is handled completely separately. To 
elaborate on this, consider IntServ for IP. In Intserv, realized by 
RSVP, the machinery for specifying flows (the FLOWSPEC) is completely 
independent from the machinery for specifying QoS treatment (the TSPEC).

I’ll also point out that flow classification can be extremely useful 
even in the absence of any QoS machinery, just to get flow fairness in 
your congestion control. By having an independent way to specify 
equivalence classes, that information can be used as input to the 
hashing scheme that drives a stochastic fair queuing (SFQ) algorithm. 
</do>

    +---+-------------------------------+-------------------------------+
    | # | Flow Classifier               | QoS Marker                    
|
    +---+-------------------------------+-------------------------------+
    | 1 | Identify the type of data     | Identify the QoS treatment    
|
    |   | flow                          |                               
|
    | 2 | Set by the producer           | Set by the consumer           
|
    | 3 | Immutable for the lifetime of | Can be modified in the        
|
    |   | Interest                      | network                       
|
    | 4 | Part of the routable content  | Non-routable part of the      
|
    |   | name                          | content name                  
|
    +---+-------------------------------+-------------------------------+

            Table 1: Difference between Flow Class and QoS Marker

<do>I find this table very hard to swallow, since it tries to compare 
QoS marker with flow classifier, when architecturally flow classifier 
neither needs to (nor in my opinion should) have QoS treatment 
functions. Even line 1 has problems, since in your design the QoS marker 
both identifies the type of data flow *and* its treatment. In line 3, if 
you use EC3 as the classifier, it is **not** immutable for the lifetime 
of an interest and can in fact be aggregated/de-aggregated hop-by-hop. 
Line 4 is similarly misguided, both for the architectural reasons in my 
general comment (3) above and because you seem to be prohibiting 
QoS-sensitive routing by saying the QoS marker can only appear in a 
“non-routable” part of the content name. </do>

    By design, the data flow classification described in ECNCT is set by
    the producer at the time of registration of the content name and
    hence it is immutable for the life time of the content.  Also, flow
    classification is encoded as part of routable name and therefore it
    has direct effect on both, PIT and FIB matching.

<do> no, it has no effect on PIT matching in CCNx, since in CCNx uses 
only exact match. NDN used to only have prefix match, but now has 
switched primarily to exact match, with prefix match an option that can 
be requested by a consumer who needs it to invoke NDN’s content 
discovery mechanisms. </do>

   Since flow
    classification mechanisms are initiated or triggered at the 
producer,
    they lack to convey consumer's or application's context in deciding
    the traffic treatment in the network for individual data flows.

<do> yes, and this is by design </do>

4.  QoS Marker Encoding Choices

    In ICN protocols like CCNx, the ability to mutate the protocol
    metadata is directly controlled by whether it is placed inside the
    security envelop or outside.  Likewise, for the encoding of the
    protocol metadata, such as QoS marker, two design choices are
    possible.

    o  In first choice, encoding of QoS marker inside content name (in
       both Interest and content object) makes it immutable as it is
       inside an authentication signature and routers along a path 
cannot
       change it.

<do>not only is it immutable, but it prohibits different consumers from 
requesting different treatments for the same data. Each possible QoS 
treatment a consumer might want would require a separate hashed and 
signed copy of the data. This seems somewhere between a serious mistake 
and a total non-starter. </do>

    o  In the second choice, we define a mandatory hop-by-hop header to
       be able to change the QoS marker over time.

<do>why does it have to be mandatory? I can’t see any reason to do 
that whether you mean just mandatory to implement, or mandatory to 
include in the packet.</do>

    While QoS mutability can be a useful feature, it can potentially
    suffer from an inconsistent end-to-end QoS treatment handling as 
each
    router can potentially change the QoS marking based on per hop
    traffic conditions.

<do>some (me included) would say this is a feature, not something you 
suffer from. In fact, Diffserv supports both dropping and remarking. The 
problem with Diffserv here is that there is only **one** tiny QoS field 
in the packet, which means that if you remark you lose the original 
state. Your QoS marker scheme appears to suffer from the same 
deficiency. There is no reason at all to replicate this limitation for 
ICN. We could easily allow two QoS treatment TLVs, one for the original 
requested treatment and another mutated by routers to encode the active 
treatment, so that the original intent can be preserved end-to-end and 
also recovered by routers at boundaries between places that have 
remarked. Even better might be a QoS treatment stack where the remarking 
could be pushed/popped at boundaries that reshape/police packets. You in 
fact suggest this might be a good idea for further study later in the 
document. It would be better to either discuss it here, or not raise the 
bogie-man of “inconsistency” here. </do>

    On the other hand, the optional content name
    field in the content object makes it a nameless object.  As an
    example, the nameless content objects are used inside a Manifest.
    So, one could put a QoS marking in an Interest name (to be used
    inside manifest), but it would not be used in the content object.  
It
    is for further study to find an encoding scheme to put QoS marker in
    content name of a nameless content object.

<do> this last sentence seems to be self-contradictory. How can you put 
something in the name of a nameless object? </do>

    From routing and forwarding perspective, all name components are
    routable, in the sense that if they are in a FIB they will be

<do> s/are routable/are potentially routable/ </do>

    matched.  In case of name-based QoS marking, we can assume that
    router publishes only the name prefixes, exclusive of QoS markers.

<do>Why? See my comment above about QoS-sensitive routing. Also, it’s 
the producer that publishes name prefixes, not routers. Routers only 
propagate them using the routing protocol </do>

    That said, globally routable FIBs would likely only have general 
name
    components.

    In summary, we have following options to design the QoS marking
    scheme based on.

    o  Define a mandatory hop-by-hop header to be able to change the QoS
       over time i.e. hop-by-hop.

<do>again, why mandatory? </do>

    o  Encode QoS marker as a distinguished field inside a content
       object, but not part of the content name.

<do> covered by the hash and the signature, right? Just need to be clear 
here, since it means is a given piece of content needs different QoS 
markers for different purposes, you need multiple copies of the data. If 
outside, then perhaps you shouldn’t say it’s “inside” the 
content object </do>

    o  Find a way to make it work with nameless objects to be able to 
put
       it inside a name.

<do>this isn’t very satisfying… especially as the previous bullet 
would contradict it. </do>

    In rest of the document, we discuss the design name-based encoding 
of
    QoS marker.

<do> s/the design name-based/the design of name-based/ </do>

5.  Name based QoS Marking

    Producer decides the classification of the data flow packets;

<do> s/Producer/The producer/ <do>

    however, it is consumer's prerogative what QoS treatment is actually
    provided to them by the network.  Consumer application itself or the
    network on behalf of consumer, can perform the QoS marking in the
    Interest messages.

<do> this statement comports with my opinions of who is responsible for 
what, but seems to contradict what you say earlier about QoS markers 
being inside content objects, or part of their name. </do>

Following factors govern the type of QoS markers
    we may require.

    o  Consumer's subscription: The type of service user subscribes with
       the service provider e.g. subscription with high-speed data plan
       vs. low-speed data plan.

    o  Service type: The type of service or the application consumer is
       running.  As a reference, we may refer to service classes as
       described in [RFC4594] (see section 2.0).

<do>I think we need much finer granularity than this. Even a single 
application or service may need many different QoS treatments for 
different pieces of data. In fact, the two things cited may be better 
understood as *policies* that govern which QoS treatments are 
permissible under what circumstances, rather than the guiding source of 
what QoS treatments one needs to define. </do>

5.1.  QoS Marker in Content Name

    Supporting the ICN design principles, the QoS marking is encoded in
    the content name field.  Prior to their consumption, as both content
    and the content name are published by the content producer, it is
    important to make distinction between the content name and the QoS
    marker.  As shown in Figure 1, there is a logical separation between
    the content name and the QoS marker.

<do> this seems to be a physical and syntactical separation and not just 
a logical separation. </do>

   To support the consumer driven
    ICN design, QoS marker is encoded as non-routable part of the 
content
    name and hence is editable.  To support the content matching
    algorithm at PIT and prefix matching algorithm at FIB, QoS marker is
    added at the end of the content name.

<do> which means you can’t do exact match in the PIT anymore? How can 
you tell if the QoS marker is present or not so you can decide whether 
to strip it? You talk later about interest aggregation, which further 
complicates this whole approach. </do>

    +------------+------------+------------+-------------+-------------+
    |   Content  |   Content  |   Content  |     QoS     |     QoS     |
    | Name comp-1| Name comp-2| Name comp-3| Name comp-1 | Name comp-2 |
    |            |            |            |             |             |
    +------------+------------+------------+-------------+-------------+
    |<---------Routable name comp--------->|<--Non-routable name comp->|

           Figure 1: Disaggregate Content and QoS Name Components

    The non-routable name component design of QoS marker allows consumer
    to add the QoS marking to the Interest message.  The reasoning 
behind
    making it non-routable is that it does not affect the forwarder's
    name or prefix matching process directly; however, it can influence
    FIB's selection of forwarding faces the Interest packet is forwarded
    to.  This allows for an implementation of QoS-aware forwarding
    strategy for both Interest and Data packets.

<do>I certainly agree that having something that can be added by the 
consumer to an Interest and mutated by the forwarders is necessary, but 
why on earth put it in the name? </do>

    Finally, the idea of routable vs non-routable is that in general we
    believe that as QoS marker is the consumer-initiated activity and
    content producer (a.k.a. publisher) would publish and the routing
    protocol would advertise only the general name segments (i.e.
    content-name without any QoS marker in it) and thus be updated in a
    FIB entry.

<do>If not “published” by the producer, then how do you do flow 
classification? This appears to be inconsistent in what you say earlier 
in the document. </do>

5.2.  Name-based QoS Marker Scheme

    Figure 1 shows a reference model for name component-based QoS marker
    scheme.  The number of QoS name components required shall depend on

<do> s/shall depend/depends/ </do>

    the type of QoS encoding uses as well as the total number of markers
    required.  QoS marker design can either be hierarchical or based on 
a
    flat naming scheme.  The exact requirements and design specification
    of QoS marker is subject of further study.

    While exact specification of QoS marking are being studied, 
following
    are the potential mechanisms that can used for encoding of QoS
    marking into content name:

    o  Using the path parameters addition to HTTP URI [RFC3986] (see
       section 3.3).  We provide a summary of path parameter below from
       [RFC3986].

       The path component contains data organized in hierarchical form
       serves to identify a resource within the scope of the URI's 
scheme
       and its naming authority.  A path consists of a sequence of path
       segments separated by a slash ("/") character, which indicate
       hierarchy.  A path parameter, part of a path segment that occurs
       after its name, control the representations of resource.  A
       reserved characters often allowed in a path segment to delimit
       scheme-specific or dereference-handler-specific subcomponents.
       For example, the semicolon (";") and equals ("=") reserved
       characters are often used to delimit parameters and parameter
       values applicable to that segment.

       The semicolon ';' delimits the parameters i.e. anything in a path
       segment that comes after a ';' is treated as a new parameter.  
The
       '=' separate parameter names from their values i.e. anything that
       is specified after '=' sign is treated as parameter value.  A
       parameter may have multiple values separated by a ',' symbol.  
Few
       examples of path parameter are shown below.

       *  /content/name;param=val1 - Content name with single path param
          with single value

       *  /content/name;param1=val1;param2=val1 - Content name with two
          path params

       *  /content/name;param1=val1,val2 - Content name with single path
          param with two value

<do> are you saying a CCNx forwarder actually has to parse all this 
rather than just doing tokenized match on name components? Do you think 
this can be done fast in the Interest forwarding path? (I don’t). 
</do>

    o  Using the 'application payload name segments' approach defined in
       CCNX [CCNXMESSAGES] (see section-3.6.1.1).

    The exact form and structure of QoS marking are being developed and
    more details shall be provided in next revision.

<do>it may be worthwhile to have ICNRG give feedback on whether the 
group believes we should go down the path of putting QoS treatments in 
CCNx names before working on (and registering with IANA!) these 
components. I’ll also point out that if we go this way, they should 
not be *application* payload name segments, since they hopefully will 
not be application-specific, but rather globally agreed traffic 
treatments ala RFC4594 and registered as such in a separate registry 
with IANA. </do>

6.  Network Procedures

    An important takeaway of implementing QoS is effective management of
    network resources such as, link capacity, bandwidth, and memory.  
ICN

<do>well, this may seem pedantic, but you need effective management of 
network resources whether or not you provide QoS features. </do>

    follows a pull-based or a receiver driven design, which directly
    influences the load on to the network.  Network based policy

<do> **Any** scheme that transmits packets influences the load on the 
network. What are you trying to say here? What is special about the pull 
model? I indeed think its special, and I say how, but you don’t seem 
to reflect this in how QoS markers are intended to work. </do>

    configuration decides how the Interest and Data traffic is carried
    optimally, and producers, depending on where the content is, 
responds

<do>optimality is nice, but I suspect it can’t be achieved via network 
policy configuration (alone). </do>

    to the requests from the consumers.  With introduction of QoS 
marking
    in the Interest packet, important changes in the behavior of each of
    these network elements are discussed.

6.1.  Consumer Procedure

    Consumer sends out the Interest packet into the network adding the
    QoS marker as per its service subscription and/or quality
    requirements.  Consumer does QoS marking and adds it as non-routable
    name component, as shown in Figure 1.

    Consumer, the initiator of the Interest is the most appropriate
    network entity to perform the QoS marking for the following reasons:

    o  ICN fundamentally is a pull-based, consumer driven design and
       consumer directly influences the resource allocation in the
       network in terms of timing and rate of Interest traffic.

    o  Consumer is aware of the context of the application initiating 
the
       Interest.  For instance, an application starting a simple video
       download compared to initiating a video conferencing.

    o  Consumer at least partially in control of the traffic paths in
       both directions [MIRCC].

    As an alternative to consumer adding the QoS marker in the Interest,
    the network (i.e. forwarder) can be allowed to amend the content 
name
    with the QoS marker.  It should be possible since QoS marker is
    encoded as a non-routable component of the content name.

<do>It’s not possible unless you change the rules for exact match in 
the PIT. </do>

  The network
    shall add the QoS marker based on the information such as, user's

<do> s/shall add/adds/ </do>

    service or subscription authorization.  In this context, an 
immediate
    forwarder (a.k.a. default gateway) would be the appropriate network
    node to perform this marking.

<do> this is a small point, but in the ISP customer environment, the 
first hop router is rarely the one that would do the marking, since it 
isn’t controlled by the ISP (e.g. home router). It would usually be 
the PE router owned by the provider. If the consumer’s router (or the 
consumer for that matter) does mark, then it’s the PE router that’s 
the one likely to remark. </do>.

    Following aspects need more discussion:

    o  Should network be allowed to add QoS marker in non-routable
       component of content name or should it add as a separate field of
       the Interest packets.

<do>this is the very first time you bring up the possibility of *not* 
putting the QoS marker in the name. At this point it makes things pretty 
confusing as to what you are proposing in the design.   </do>

    o  Once QoS marker is added and Interest is admitted into the 
network
       should network be allowed to modify the QoS marker.

    o  Since QoS markings are explicit, should the QoS marker be aware 
of
       consumer's subscription and make the relationship between the two
       explicit.

<do>How would it be aware? This tangles up with client authentication, 
which seems orthogonal </do>

6.2.  Forwarder Procedure

    ICN forwarder, in addition to preserving the Interest state into PIT
    table (mapping between content name and the interface it receives 
the
    Interest on), now also preserves the state of QoS marker against the
    interface.  In a representative domain, the PIT structure is 
enhanced
    by adding a new column to save the state of QoS marker.  This forms 
a
    3-tuple information set comprising content name, interface, and QoS
    marker.

<do> we’ve talked about this a number of times already. If you make 
the marker a third column on the PIT state you get data duplication and 
possibly an explosion of PIT entries. It’s entirely adequate to 
associate the QoS marker/treatment only with the interface the interest 
was received on. </do>

Unlike PIT, there is no change in the structure of FIB
    table; however, forwarder forwards to the upstream ICN router both
    content name and QoS marker, as they are received from its
    predecessor.

<do>unless you do QoS routing, in which case QoS treatments might be 
associated with a FIB entry. </do>

    With enhanced QoS-aware content name design, forwarder performs the
    content store (CS) lookup only using routable name component.

<do>this seems wrong. what if there are name components that aren’t 
“routable” in the sense you use the word, but aren’t the QoS 
marker? In any event, some designs have a merged PIT and CS, so having 
different rules for lookups in the two would be highly problematic for 
them. </do>

  It is
    possible that a forwarder implementation may choose to understand
    name components types and do special things based on it.  
Conversely,
    the PIT aggregation logic uses both content name and QoS marker in
    PIT lookup, which makes it a QoS-aware Interest aggregation.
    Section 7.1 provide more details about new PIT design and related
    procedures.

    While there are no changes in the FIB table, the conventional name
    prefix based multipath forwarding path selection of forwarder can 
use
    the QoS marker to make the QoS-aware forwarding decision.  For
    example, QoS marking can be used to select a low latency interface
    over a high latency interface or it can be used to select a high
    bandwidth path over a low bandwidth path or vice versa.

<do>I don’t see how you do this if you don’t have some property in 
the FIB entry that tells you. </do>.

    The following aspects of QoS-aware forwarder design need more
    attention:

    o  How mapping is done between QoS marking and the forwarding queue
       after the forwarding interface is selected.

<do>while this is certainly true, this is one area where I don’t see 
*any* difference from IP or any other packet-oriented network 
architecture. You have a packet, you need to put it in a queue. Which 
queue does it go in and what happens to the other packets already in the 
queue? So, I suspect that the only part of this that belongs in an ICN 
QoS treatment design is what the treatments are and evidence to show 
that there are feasible queuing algorithms that can accomplish the 
intended semantics. </do>

    o  From the perspective of per-hop-behavior (PHB), it is important 
to
       understand if remarking of QoS is allowed and if one marker is
       sufficient for it.  One possibility is to preserve the original
       QoS marker added by consumer and have a running marker set by the
       intermediate forwarder in the network.

<do>yes, this is one possibility. Another design question is whether the 
consumer is in control of whether the forwarder drops or remarks, and 
whether that decision is baked into the specification of each particular 
QoS treatment, or independently signaled in the Interest packet. There 
are arguments for either approach. </do>

    o  In the context of QoS remarking by the network, it may also 
become
       important to let consumer know, what network is doing with their
       QoS marking.  From the network behavior perspective, following 
are
       the possibilities:

       *  QoS marking is preserved and obeyed in subsequent hop

<do> here it think you mean “current hop”, not subsequent hop, since 
a forwarder can’t tell if the “subsequent” hop in the next 
forwarder is doing something or not. </do>

       *  QoS marking is preserved but not obeyed

       *  QoS marking is remarked and obeyed

<do>I doubt any network operator would be willing to tell you if they 
actually obeyed your QoS treatment request. There’s also the problem 
of who gets to know which forwarder(s) ignored your request, or even 
what that means. For example, say you have a low-latency QoS treatment 
and you hit a forwarder not supporting priority queues, but whose 
interfaces are never more than 10% loaded. Is it obeying your QoS 
treatment or not? </do>

<do> as an aside, years ago there was a major effort by All Morton and a 
pretty capable group from various Telcos in ITU to come up with a 
specification for how one could attribute QoS degradation to particular 
network elements in particular autonomous systems along a path. They did 
some very good work and had what some considered to be a technically 
sound solution. Of course they got laughed out of the room when they 
went back to their organizations and proposed that the customers having 
multi-AS paths could come try to blame one of the AS-es on the path. 
</do>

6.2.1.  QoS and Multipath Forwarding

    QoS marking in the Interest packet does not change the multipath
    forwarding capability of ICN forwarders.  In Section 7.2, more
    details are provided about specific QoS behavior related to 
multipath
    forwarding.

6.3.  Producer Procedure

    Producer is aware of the disaggregation between routable name and 
the

<do> s/Producer/The producer/ </do>

    non-routable QoS marker.  It looks up the content in content store
    (CS) only using routable name component.  An intermediate router 
acts
    in a similar manner.

<do> see earlier comment about changing the exact match PIT and CS 
semantics </do>

    Depending on the what kind of QoS marking is done in the Interest
    packet, producer can have following response behaviors:

    o  Producer may respond in a different manner with the Data packet 
to
       the consumer.

<do> what does this mean, exactly? </do>

    o  One such aspect of QoS aware response could be to provide the
       consumer information about how much distance (e.g. number of 
hops)
       Interest has travelled into the network before it is satisfied.

<do> that’s not QoS-aware, that’s a basic forwarding feature. Also, 
since we do symmetric routing, you can do this just by re-purposing the 
hop-limit field to be a return hop count field in Data packets. I think 
I’ve proposed this in the past, and we could easily write a quick 
draft to add this to CCNx. </do>

    o  QoS-aware response does not change the original requested 
content.

7.  QoS-Aware Forwarder Design

    Towards supporting end-to-end QoS and handling of Interest and Data
    traffic throughout the network, important network procedures are
    discussed in Section 6.  There are some important design changes in
    the way PIT maintains the pending Interests and the way forwarding
    decisions are made.  This section discuss in detail each of the
    changes.

7.1.  Enhanced PIT Design

    The new PIT design has added a new column to maintain the QoS marker
    received in the Interest packet.  The enhancement in the PIT table
    and its behavior are applicable only specific to its Interest
    aggregation feature of multiple Interest packet received for the 
same
    content.

<do> see earlier comment about PIT explosion. Again, this state needs to 
be stored in the incoming interface information, and hence is only one 
extra field in a part of the PIT that is O(#of incoming interfaces an 
Interest for this matching name has been received from) </do>

         +----+----------------+--------------------+--------------+
         |    |                |                    |              |
         | #  | Interface Id   | Content Name       | QoS Marker   |
         +---------------------------------------------------------+
         | 1  | face-1         | /youtube/med/vid-1 | /qos-level-1 |
         +---------------------------------------------------------+
         | 2  | face-2         | /youtube/med/vid-1 | /qos-level-2 |
         +---------------------------------------------------------+
         | 3  | face-1         | /youtube/med/vid-1 | /qos-level-3 |
         +----+----------------+--------------------+--------------+

                Figure 2: Enhanced PIT Design with QoS Marker

<do> Why on earth would you store the name multiple times? </do>

    The scenarios emerging from the new QoS marking and new PIT design
    are discussed here.  Three PIT entries are shown in Figure 2 to
    explain the new PIT design and its behavior.  Notice that in the
    modified PIT design, content name always takes the higher precedence
    over the QoS marker in deciding the Interest aggregation.  Having
    said this, following scenarios of Interest arrival at forwarder are
    possible:

    a.  Two or more Interests with different content name, but with
        different QoS markers are received on two different interfaces.

    b.  Two or more Interests with different content name, but with same
        QoS markers are received on two different interfaces.

    c.  Two or more Interests with same content name and with same QoS
        markers are received on two different interfaces.

    d.  Two or more Interests with same content name, but with different
        QoS markers are received on two different interfaces.

    e.  Two or more Interests with same content name, but with different
        QoS markers are received on the same interface.

<do> I think I’ve already made the comment on the -00 draft that this 
taxonomy is overly complicated and confusing </do>

    Scenarios (a) and (b), since both Interests are received with
    different content name, no PIT aggregation is required and Interest
    are forwarded if content is not found in CS.  In case (c), since 
both
    content name and QoS marker are same, Interest is aggregated in PIT
    and second Interest is not forwarded.

    In scenarios (d) and (e), since Interests are received with same
    content name, the PIT aggregation decision is made based on the QoS
    marker.  These two scenarios lead to a potential PIT scaling issue,
    which is discussed in Section 7.2.

7.2.  Multiple Interest and PIT Scaling

    With two scenarios (d) and (e) in Section 7.1 forwarder forwards 
both
    the Interests to upstream router creating two PIT entries as shown 
in
    Figure 2 i.e. entries #1 and #3.  This behavior violates the
    conventional PIT behavior; however, is essential to support the
    different QoS treatment.

<do> no it isn’t, if you have a partial order defined over the space 
of QoS treatments. It’s a problem if all QoS treatments are considered 
unordered.  </do>

    In order to support QoS-aware forwarding, the conventional PIT
    aggregation needs to be loosened up proportional to the number of 
QoS
    markers; in other words, the QoS treatments.  Without this, upstream
    forwarder would not get an opportunity to obey each of the QoS
    treatments.  The theoretical upper bound on the PIT scaling for 
given
    content will be equal to number of QoS markers.

    The impact on the PIT scaling depends on and can be mitigated by the
    following mechanisms:

    o  By keeping the number of QoS markers limited

    o  By keeping the QoS marker unique and avoiding the hierarchy or
       order among them.

    o  In real-time case, the PIT may not hit the upper bound all the
       times i.e. not all QoS markers are utilized all the times on the
       same content name.

<do> this is an important point. If we can demonstrate that for any 
given equivalence class in a namespace, the number of different QoS 
treatments that might be active at any given time is tightly bounded, 
that might be sufficient </do>

    o  Using an optimization in multiple Interest handling, as discussed
       in Section 7.3

7.3.  Handling PIT Scaling

    The PIT scaling issue described in Section 7.2 can be handled with 
an
    optimization discussed here.

    The forwarder can avoid forwarding the second/duplicate Interest if
    it receives it with a lower QoS marking than the one already pending
    in PIT.  Thus, achieving the Interest aggregation based on the 
higher
    QoS marker for given content name.  Conversely if the received
    Interest is with a higher QoS marking, then forwarder forwards the
    Interest and updates the pending PIT entry with higher QoS marking.
    Also, note that forwarder updates the PIT entry irrespective of the
    interface the higher QoS marked Interest is received on.

<do> be careful here to have some mathematical rigor about what it means 
to be “higher”. </do>

    Notice that forwarding of Interest with higher QoS marker in spite 
of
    having an already pending with a lower QoS marker, is a breach of
    Interest aggregation at PIT in conventional terms; however, it is
    necessary to give an opportunity for upstream routers to provide

    appropriate QoS treatment to the higher priority Interest and the
    resulting Data traffic flow.

    These are the scenarios, which provide for a QoS-aware PIT design,
    Interest aggregation and forwarding in ICN router.

7.4.  Data Delivery at PIT

    With QoS-aware Interest aggregation at PIT, more than one Interest
    are flowing in the network for the same content.  With a higher
    probability of a priority treatment to the higher QoS marked 
Interest
    at each hop and with the possibility of multipath forwarding, it is
    highly likely that the Interest with higher QoS marking shall be
    satisfied faster than the Interest with lower QoS marking.

<do> sorry, I don’t follow this. Some paths may be more loaded with 
high QoS treatment than others, and hence if you choose to send a lower 
marked Interest on the lightly loaded path it could very well be 
satisfied sooner. It might also hit a closer cache. </do>

    The Data packet arrival may satisfy all the PIT entries for the 
given
    content name irrespective of the QoS markers in Data packet.  This 
is
    possible mainly because the content itself in Data packet does not
    change by different QoS marker in the Interest.  Depending on 
whether
    forwarder implements a PIT scaling optimization, two scenarios of
    Data forwarding are possible:

    o  Data packet to the downstream interface is forwarded with its
       original QoS marking recorded by the PIT.

<do> yes, however this also determines how the data packet is queued on 
the downstream interface.   </do>

    o  Data packet to the downstream interface is forwarded with the
       higher QoS marking recorded by the PIT by virtue of PIT
       optimization.

<do> no. I see no justification for this option. </do>

    With the PIT optimization described in Section 7.3, it is possible 
to
    satisfy a pending Interest with lower QoS marking with arrival of a
    Data packet having higher QoS marker.  As a result, a user with 
lower
    QoS subscription may experience a better response time from the
    network.  Note that this is a legitimate behavior, as ICN is
    fundamentally designed to optimize the network round-trip time
    providing better user experience.

<do> while this might seem superficially true, there are, at least for 
some scenarios like adaptive video streaming, good reasons to 
intentionally delay or otherwise give lower service to requests issued 
with particular QoS treatments. This, one the other hand, is a longer 
discussion/explanation than is appropriate for these comments. </do>

<do> **However**, we’ve now gotten to the end of the main content, and 
you haven’t talked about the interaction of QoS markers (or in my 
world, QoS treatments) with caching *at all*. Any spec that purports to 
show how QoS works for ICN needs to address not just link resources, but 
cache resources as well. In some circumstances (e.g. the constrained 
network scenarios discussed in 
https://datatracker.ietf.org/doc/draft-gundogan-icnrg-iotqos/) you need 
to specify the behavior of forwarder memory and forwarding CPU capacity 
as well. That’s a major missing piece for your work. </do>

8.  Security Considerations

    ICN being name-based networking opens up new security and privacy
    considerations which have to be studied in the context of name-based
    QoS framework.

    Depending on where the QoS marker is encoded in the Interest, 
certain
    security attack scenarios against QoS treatment are possible.  If 
the
    QoS marker located inside the security envelope, it can be read, but
    not changed.  Conversely, if the QoS marker is placed outside of the
    security envelope, it can be added as hop-by-hop message header and,
    therefore, can be modified by the transit ICN forwarders; however, 
it
    also opens it to various attack scenarios.

<do> and these are? Especially, are they new, amplifications of existing 
attacks, or inherent in the ability of any on-path element to most 
anything it likes? You also need to distinguish attacks mounted by 
consumers, those mounted by producers, those mounted by on-path 
forwarders, and those mounted by off-path elements. </do>

    Consumer procedure discussed in Section 6.1 and forwarder procedure
    discussed in Section 6.2 shall decide the security requirements
    related to implementing QoS treatments in ICN.

9.  Summary

    This draft provides an architecture to implement end-to-end QoS
    treatments in ICN network using a name-based disaggregation of
    routable content name and non-routable, mutable QoS markers.  The
    independence between content name and QoS marking makes their
    evolution easier and yet bounded to content name keeping with ICN
    principles.

    A new PIT design and a potential impact on PIT scaling is presented.
    An optimization to deal with the PIT scaling problem is discussed
    where a number of pending Interests requests in PIT for same content
    to be normalized around the highest QoS marking.

    Security requirements are dependent on whether QoS marker is encoded
    inside security envelop or outside of it.  Consumer and forwarder
    procedure requirements shall also govern the security features.

    A detailed analysis and evaluation shall be performed, through
    prototype using [VICN] and/or simulation [NDNSIM], of the impact on
    PIT aggregation and effect of optimization.  The details on design 
of
    a naming scheme for QoS marking in content name needs to be worked
    on.  It is also necessary to test and measure the effectiveness of
    the QoS framework by deploying it using a multimedia streaming
    application.

10.  Acknowledgements

    We thank all contributors, reviewers and the chairs for their
    valuable time in providing the comments and feedback, which has
    helped to improve this draft.  We would like to thank Mark Mosco who
<do> s/Mark Mosco/Marc Mosko/ </do>

    provided a useful feedback on proposed name-based encoding scheme 
and
    nameless content objects.

11.  IANA Considerations

    This draft includes no request to IANA.

<do> change this to “TBD” since once completed the document would 
undoubtedly have substantial actions needed by IANA </do>

**[End of comments]**

---

From: Akbar Rahman <[email-masked]>
Date: Thu, 19 Sep 2019 05:16:38 +0000

Hi Dave/Authors,

I have a few additional comments on top of Dave’s extremely thorough review:

  1.  Overall – A very useful draft that helps advance the issue of how to deal with E2E QoS (especially the data path) appropriately in ICN
  2.  General – I was expecting some reference/comparison/lessons-learned with Hybrid-ICN.  Do you think it would be worth adding such info?
  3.  Abstract – While it is true many ICN systems have been implemented as an IP overlay, there are also other key ICN systems that have been implemented as an Underlay (e.g. as per the Deployment Considerations draft).  This key point should be clarified.
  4.  Section 1: does TOS stand for Type of Service (in any case some reference should be given)?
  5.  Section 3.1: "authors points to" -> "the authors point to"
  6.  Section 3.1: Suggest adding a reference for "PURSUIT architecture"
  7.  Section 3.2: Suggest adding the full spelling for FIB (i.e. Forwarding Information Base)
  8.  Section 3.2: Table 1 discusses the difference between Flow Classification and QoS Marker, which is to be described in Section 4. Will it flow better to move Table 1 from Section 3.2 to Section 4?
  9.  Section 5.1: "...would publish and the routing protocol would advertise only..." - Suggest rephrasing this sentence
  10. Section 6: It may be helpful to give one or some examples of QoS Marker, especially when talking about Consumer Procedure and Forwarded Procedure.
  11. Section 6: Does QoS Marker apply to content packets only, or also for Interest packets? In other words, is there any QoS issues related to forwarding Interest packets?
  12. Section 8: Should also consider security requirements for Producer Procedure in Section 6.3?
  13. Section 9: "Interests requests" -> "Interest requests"
  14. Section 9: Appears that more investigation/experimentation is required before we can decide if applying QoS treatment to (1) both Interest and Data/Content packets is better than just applying QoS treatment to (2) only on the Data/Content path

Best Regards,

Akbar

From: icnrg <[email-masked]> On Behalf Of David R. Oran
Sent: Sunday, September 15, 2019 9:53 AM
To: [email-masked]
Cc: Suthar, Prakash <[email-masked]>; Jangam, Anil <[email-masked]>; Stolic, Milan <[email-masked]>
Subject: [icnrg] Comments on the recently posted draft-anilj-icnrg-dnc-qos-icn-01

Many thanks for the continued work on this. It helps a lot to inform our development of QoS capabilities for NDN and CCNx in ICNRG. I have three sorts of comments. First are general/architectural comments. These are directly below in the email. Second are detailed technical comments, some of which emerge directly from the architectural questions; others are about specifics of the proposed design. Third are some editorial comments, typos, and nits. The latter two sets are embedded in the copy of the draft itself, which I append to the message.

General/Architectural comments

I doubt you will be surprised by these comments, as I’ve made most of them multiple times on both the earlier version of this draft, and in general discussions at ICNRG. I do think they are important enough to keep repeating since they cut to the core of what a QoS design for NDN and CCNx should look like. They also relate closely to the material in the recent draft I posted: https://datatracker.ietf.org/doc/draft-oran-icnrg-qosarch/ so reading that first will help folks (not just the authors) understand better where I’m coming from.

Here you go:

  1.  The design couples flow classification and QoS treatment into a single mechanism, the QoS Marker. I think this is a quite serious mistake. I cannot find a single good reason do do this, because flow classification is a property of the named objects in the namespace and how they relate to each other, while QoS treatment is a property of a particular fetch operation by an individual consumer. The flow classification persists independent of who the consumers are and what they are doing. The QoS treatment for the same content object could differ substantially when fetched by different consumers. Combining these just seems wrong.

  1.  Placing the QoS marker (as currently defined) as part of the name is also a mistake in my view. It complicates name parsing and involves what I consider to be problematic changes to both PIT and CS data structures (and possibly the FIB as well). I give what I think are convincing arguments in the QoSArch draft for why QoS treatment should be entirely independent of names. For flow classification, the most important question is whether the flows are bound into the security envelope of the content object or not. The flow classification draft presents both approaches and arguments for and against each. Your draft doesn’t seem to capture these issues in a way that adds to the understanding or informs the tradeoffs. It could be that both mechanisms of flow classification might be needed. By being clear about what is bound securely to a name, what is hierarchically decomposable (if you need flows and sub flows) and what can be aggregated by routers seems a key architectural decision that you can’t defer to some later detailed design discussion by defining QoS markers the way you do.

  1.  Much of the draft tries to make a distinction between “routable name prefixes” and “non routable name components”. No such distinction exists in either the semantics of CCNx or NDN names, or in their encoding. A deep feature of both designs is the notion that something is routable if it, or a prefix thereof, has a FIB entry. Period. The namespace designer does not have to pre-determine what part of the name is no longer useful for routing and specify it differently. Going with a QoS Marker approach with the marker part of the name introduces some pretty deep changes (at least as I see it) and will cause QoS-related things to permeate lots of parts of the design unrelated to QoS, such as name parsing, PIT structure, etc.

Technical and Editorial Comments

[I have snipped out parts of the draft I have no comments on, including the page break filigree and other artifacts of the .txt rendering]. My comments are all preceded by <do> and terminated with </do>.

Abstract

Mechanisms for specifying and implementing end-to-end Quality of
service (QoS) treatments in Information-Centric Networks (ICN) has
not been explored so far. There has been some work towards
implementing QoS in ICN; however, the discussions are mainly centered
around strategies used in efficient forwarding of Interest packets.
Moreover, as ICN has been tested mainly as an IP overlay, it's QoS is

<do>s/it’s/its/ </do>

implementing QoS in ICN natively. This document describes mechanisms

  1.  Introduction

into better user experience. Evaluation and study of prior work done

<do> s/into/in a/ </do>

detailed discussion related to the design of name-based QoS encoding,

  1.  Prior Work and Motivation

M.F. Al-Naday et.al. in [NADAY] argues that information awareness of
ICN network would help build scalable QoS model. In the context of
CCN/NDN network design, authors points to the possibility of using

Weibo Chu et.al. in [WEIBO] present a QoS model for ICN network based

the process more energy efficient. They use the same Data forwarding

In [CHRISTOS] Christos et.al. argue about a need for a differentiated

<do>The hop-by-hop interest shaper paper (Wang, Yaogong and Rozhnova, Natalya and Narayanan, Ashok and Oran, David and Rhee, Injong: An Improved Hop-by-hop Interest Shaper for Congestion Control in Named Data Networking, ACM ICN 2013) in fact shows how by shaping interests, the returning data packets are controlled and shaped, hence there is work on how if you have preferential treatment of Interests, preferential treatment of the returning data is achieved automatically </do>

Flow classification [ICNFLOW] describe the methods for classification
of data flows in ICN. The method called equivalence class component
count (EC3), uses a predefined number of name components of the

<do> the count is not “predefined”. It is set by the producer when it creates the data packet containing the corresponding content object (not necessarily when the content object itself is created). It can also be included in an Interest and set by the consumer (or aggregated by a forwarder) which is by no means “predefined” either. </do>

content name forming an equivalence class. While approach has the
flexibility in re-grouping of components in aggregating the flows, it
suffers from an inconsistent interpretation due to implicit binding
of the equivalence class into the content namespace. In the second

<do> I don’t understand what you’re saying here. What is inconsistent about the interpretation? By your logic, any flow aggregation would introduce inconsistencies since the aggregation is done by an element that is not in charge of the namespace. </do>

method called equivalence class name component type (ECNCT), the flow
classification information is directly embedded in the hierarchical
content name. This paves the way to achieve implicit aggregation of

<do>It seems you I and don’t have the same definition of “aggregation”. By my definition, aggregation means mixing multiple, possibly unrelated, flows together when maintaining flow state (and when enqueuing/dequeing packets for transmission). Using my definition ECNT does not do any implicit aggregation. It may permit hierarchical flow decomposition, but that’s something entirely different. </do>

data flows analogous to the prefix-based aggregation of content names
in FIB table. At the forwarder level, a flow table is implemented to
store the equivalence classes and is used to perform the flow
classification decisions.

<do>this part I mostly agree with, however there are may ways to realize a “flow table”, some of which use an independent data structure with per-flow entries, while others do not (e.g. SFQ) </do>

<do> s/treament/treatment/ </do>
<do> The above statement, while true, gives the impression that there is some deficiency in the flow classification design of EC3 or ECNT that needs to be remedied. That is not so, as if you adopt an architecture that cleanly separates flow classification from QoS treatment there is nothing missing. QoS treatment is handled completely separately. To elaborate on this, consider IntServ for IP. In Intserv, realized by RSVP, the machinery for specifying flows (the FLOWSPEC) is completely independent from the machinery for specifying QoS treatment (the TSPEC).

I’ll also point out that flow classification can be extremely useful even in the absence of any QoS machinery, just to get flow fairness in your congestion control. By having an independent way to specify equivalence classes, that information can be used as input to the hashing scheme that drives a stochastic fair queuing (SFQ) algorithm. </do>

+---+-------------------------------+-------------------------------+
| # | Flow Classifier | QoS Marker |
+---+-------------------------------+-------------------------------+
| 1 | Identify the type of data | Identify the QoS treatment |
| | flow | |
| 2 | Set by the producer | Set by the consumer |
| 3 | Immutable for the lifetime of | Can be modified in the |
| | Interest | network |
| 4 | Part of the routable content | Non-routable part of the |
| | name | content name |
+---+-------------------------------+-------------------------------+

<do>I find this table very hard to swallow, since it tries to compare QoS marker with flow classifier, when architecturally flow classifier neither needs to (nor in my opinion should) have QoS treatment functions. Even line 1 has problems, since in your design the QoS marker both identifies the type of data flow and its treatment. In line 3, if you use EC3 as the classifier, it is not immutable for the lifetime of an interest and can in fact be aggregated/de-aggregated hop-by-hop. Line 4 is similarly misguided, both for the architectural reasons in my general comment (3) above and because you seem to be prohibiting QoS-sensitive routing by saying the QoS marker can only appear in a “non-routable” part of the content name. </do>

<do> no, it has no effect on PIT matching in CCNx, since in CCNx uses only exact match. NDN used to only have prefix match, but now has switched primarily to exact match, with prefix match an option that can be requested by a consumer who needs it to invoke NDN’s content discovery mechanisms. </do>

Since flow
classification mechanisms are initiated or triggered at the producer,
they lack to convey consumer's or application's context in deciding
the traffic treatment in the network for individual data flows.

  1.  QoS Marker Encoding Choices

o In first choice, encoding of QoS marker inside content name (in
both Interest and content object) makes it immutable as it is
inside an authentication signature and routers along a path cannot
change it.

<do>not only is it immutable, but it prohibits different consumers from requesting different treatments for the same data. Each possible QoS treatment a consumer might want would require a separate hashed and signed copy of the data. This seems somewhere between a serious mistake and a total non-starter. </do>

<do>why does it have to be mandatory? I can’t see any reason to do that whether you mean just mandatory to implement, or mandatory to include in the packet.</do>

While QoS mutability can be a useful feature, it can potentially
suffer from an inconsistent end-to-end QoS treatment handling as each
router can potentially change the QoS marking based on per hop
traffic conditions.

<do>some (me included) would say this is a feature, not something you suffer from. In fact, Diffserv supports both dropping and remarking. The problem with Diffserv here is that there is only one tiny QoS field in the packet, which means that if you remark you lose the original state. Your QoS marker scheme appears to suffer from the same deficiency. There is no reason at all to replicate this limitation for ICN. We could easily allow two QoS treatment TLVs, one for the original requested treatment and another mutated by routers to encode the active treatment, so that the original intent can be preserved end-to-end and also recovered by routers at boundaries between places that have remarked. Even better might be a QoS treatment stack where the remarking could be pushed/popped at boundaries that reshape/police packets. You in fact suggest this might be a good idea for further study later in the document. It would be better to either discuss it here, or not raise the bogie-man of “inconsistency” here. </do>

inside manifest), but it would not be used in the content object. It

<do> this last sentence seems to be self-contradictory. How can you put something in the name of a nameless object? </do>

<do>Why? See my comment above about QoS-sensitive routing. Also, it’s the producer that publishes name prefixes, not routers. Routers only propagate them using the routing protocol </do>

That said, globally routable FIBs would likely only have general name
components.

<do> covered by the hash and the signature, right? Just need to be clear here, since it means is a given piece of content needs different QoS markers for different purposes, you need multiple copies of the data. If outside, then perhaps you shouldn’t say it’s “inside” the content object </do>

o Find a way to make it work with nameless objects to be able to put
it inside a name.

<do>this isn’t very satisfying… especially as the previous bullet would contradict it. </do>

In rest of the document, we discuss the design name-based encoding of
QoS marker.

  1.  Name based QoS Marking

<do> this statement comports with my opinions of who is responsible for what, but seems to contradict what you say earlier about QoS markers being inside content objects, or part of their name. </do>

o Service type: The type of service or the application consumer is
running. As a reference, we may refer to service classes as
described in RFC4594<see%20section%202.0>.

<do>I think we need much finer granularity than this. Even a single application or service may need many different QoS treatments for different pieces of data. In fact, the two things cited may be better understood as policies that govern which QoS treatments are permissible under what circumstances, rather than the guiding source of what QoS treatments one needs to define. </do>

<do> this seems to be a physical and syntactical separation and not just a logical separation. </do>

ICN design, QoS marker is encoded as non-routable part of the content

<do> which means you can’t do exact match in the PIT anymore? How can you tell if the QoS marker is present or not so you can decide whether to strip it? You talk later about interest aggregation, which further complicates this whole approach. </do>

to add the QoS marking to the Interest message. The reasoning behind

<do>I certainly agree that having something that can be added by the consumer to an Interest and mutated by the forwarders is necessary, but why on earth put it in the name? </do>

<do>If not “published” by the producer, then how do you do flow classification? This appears to be inconsistent in what you say earlier in the document. </do>

the type of QoS encoding uses as well as the total number of markers
required. QoS marker design can either be hierarchical or based on a
flat naming scheme. The exact requirements and design specification
of QoS marker is subject of further study.

While exact specification of QoS marking are being studied, following
are the potential mechanisms that can used for encoding of QoS
marking into content name:

o Using the path parameters addition to HTTP URI RFC3986<see%0A%20%20%20%20%20%20section%203.3>. We provide a summary of path parameter below from
[RFC3986].

  serves to identify a resource within the scope of the URI's scheme

  segment that comes after a ';' is treated as a new parameter.  The

  parameter may have multiple values separated by a ',' symbol.  Few

     with single value

     path params

     param with two value

<do> are you saying a CCNx forwarder actually has to parse all this rather than just doing tokenized match on name components? Do you think this can be done fast in the Interest forwarding path? (I don’t). </do>

o Using the 'application payload name segments' approach defined in
CCNX CCNXMESSAGES<see%20section-3.6.1.1>.

<do>it may be worthwhile to have ICNRG give feedback on whether the group believes we should go down the path of putting QoS treatments in CCNx names before working on (and registering with IANA!) these components. I’ll also point out that if we go this way, they should not be application payload name segments, since they hopefully will not be application-specific, but rather globally agreed traffic treatments ala RFC4594 and registered as such in a separate registry with IANA. </do>

  1.  Network Procedures

An important takeaway of implementing QoS is effective management of
network resources such as, link capacity, bandwidth, and memory. ICN

<do>well, this may seem pedantic, but you need effective management of network resources whether or not you provide QoS features. </do>

<do> Any scheme that transmits packets influences the load on the network. What are you trying to say here? What is special about the pull model? I indeed think its special, and I say how, but you don’t seem to reflect this in how QoS markers are intended to work. </do>

configuration decides how the Interest and Data traffic is carried
optimally, and producers, depending on where the content is, responds

<do>optimality is nice, but I suspect it can’t be achieved via network policy configuration (alone). </do>

to the requests from the consumers. With introduction of QoS marking
in the Interest packet, important changes in the behavior of each of
these network elements are discussed.

6.1. Consumer Procedure

o Consumer is aware of the context of the application initiating the
Interest. For instance, an application starting a simple video
download compared to initiating a video conferencing.

As an alternative to consumer adding the QoS marker in the Interest,
the network (i.e. forwarder) can be allowed to amend the content name
with the QoS marker. It should be possible since QoS marker is
encoded as a non-routable component of the content name.

<do>It’s not possible unless you change the rules for exact match in the PIT. </do>

The network

service or subscription authorization. In this context, an immediate
forwarder (a.k.a. default gateway) would be the appropriate network
node to perform this marking.

<do> this is a small point, but in the ISP customer environment, the first hop router is rarely the one that would do the marking, since it isn’t controlled by the ISP (e.g. home router). It would usually be the PE router owned by the provider. If the consumer’s router (or the consumer for that matter) does mark, then it’s the PE router that’s the one likely to remark. </do>.

<do>this is the very first time you bring up the possibility of not putting the QoS marker in the name. At this point it makes things pretty confusing as to what you are proposing in the design. </do>

o Once QoS marker is added and Interest is admitted into the network
should network be allowed to modify the QoS marker.

o Since QoS markings are explicit, should the QoS marker be aware of
consumer's subscription and make the relationship between the two
explicit.

<do>How would it be aware? This tangles up with client authentication, which seems orthogonal </do>

6.2. Forwarder Procedure

ICN forwarder, in addition to preserving the Interest state into PIT
table (mapping between content name and the interface it receives the
Interest on), now also preserves the state of QoS marker against the
interface. In a representative domain, the PIT structure is enhanced
by adding a new column to save the state of QoS marker. This forms a
3-tuple information set comprising content name, interface, and QoS
marker.

<do> we’ve talked about this a number of times already. If you make the marker a third column on the PIT state you get data duplication and possibly an explosion of PIT entries. It’s entirely adequate to associate the QoS marker/treatment only with the interface the interest was received on. </do>

<do>unless you do QoS routing, in which case QoS treatments might be associated with a FIB entry. </do>

<do>this seems wrong. what if there are name components that aren’t “routable” in the sense you use the word, but aren’t the QoS marker? In any event, some designs have a merged PIT and CS, so having different rules for lookups in the two would be highly problematic for them. </do>

It is
name components types and do special things based on it. Conversely,

prefix based multipath forwarding path selection of forwarder can use

<do>I don’t see how you do this if you don’t have some property in the FIB entry that tells you. </do>.

<do>while this is certainly true, this is one area where I don’t see any difference from IP or any other packet-oriented network architecture. You have a packet, you need to put it in a queue. Which queue does it go in and what happens to the other packets already in the queue? So, I suspect that the only part of this that belongs in an ICN QoS treatment design is what the treatments are and evidence to show that there are feasible queuing algorithms that can accomplish the intended semantics. </do>

o From the perspective of per-hop-behavior (PHB), it is important to

<do>yes, this is one possibility. Another design question is whether the consumer is in control of whether the forwarder drops or remarks, and whether that decision is baked into the specification of each particular QoS treatment, or independently signaled in the Interest packet. There are arguments for either approach. </do>

o In the context of QoS remarking by the network, it may also become
important to let consumer know, what network is doing with their
QoS marking. From the network behavior perspective, following are
the possibilities:

<do> here it think you mean “current hop”, not subsequent hop, since a forwarder can’t tell if the “subsequent” hop in the next forwarder is doing something or not. </do>

<do>I doubt any network operator would be willing to tell you if they actually obeyed your QoS treatment request. There’s also the problem of who gets to know which forwarder(s) ignored your request, or even what that means. For example, say you have a low-latency QoS treatment and you hit a forwarder not supporting priority queues, but whose interfaces are never more than 10% loaded. Is it obeying your QoS treatment or not? </do>

<do> as an aside, years ago there was a major effort by All Morton and a pretty capable group from various Telcos in ITU to come up with a specification for how one could attribute QoS degradation to particular network elements in particular autonomous systems along a path. They did some very good work and had what some considered to be a technically sound solution. Of course they got laughed out of the room when they went back to their organizations and proposed that the customers having multi-AS paths could come try to blame one of the AS-es on the path. </do>

QoS marking in the Interest packet does not change the multipath
forwarding capability of ICN forwarders. In Section 7.2, more
details are provided about specific QoS behavior related to multipath
forwarding.

6.3. Producer Procedure

Producer is aware of the disaggregation between routable name and the

non-routable QoS marker. It looks up the content in content store
(CS) only using routable name component. An intermediate router acts
in a similar manner.

<do> see earlier comment about changing the exact match PIT and CS semantics </do>

o Producer may respond in a different manner with the Data packet to
the consumer.

o One such aspect of QoS aware response could be to provide the
consumer information about how much distance (e.g. number of hops)
Interest has travelled into the network before it is satisfied.

<do> that’s not QoS-aware, that’s a basic forwarding feature. Also, since we do symmetric routing, you can do this just by re-purposing the hop-limit field to be a return hop count field in Data packets. I think I’ve proposed this in the past, and we could easily write a quick draft to add this to CCNx. </do>

o QoS-aware response does not change the original requested content.

  1.  QoS-Aware Forwarder Design

7.1. Enhanced PIT Design

The new PIT design has added a new column to maintain the QoS marker
received in the Interest packet. The enhancement in the PIT table
and its behavior are applicable only specific to its Interest
aggregation feature of multiple Interest packet received for the same
content.

<do> see earlier comment about PIT explosion. Again, this state needs to be stored in the incoming interface information, and hence is only one extra field in a part of the PIT that is O(#of incoming interfaces an Interest for this matching name has been received from) </do>

    |    |                |                    |              |

<do> I think I’ve already made the comment on the -00 draft that this taxonomy is overly complicated and confusing </do>

are forwarded if content is not found in CS. In case (c), since both

With two scenarios (d) and (e) in Section 7.1 forwarder forwards both
the Interests to upstream router creating two PIT entries as shown in
Figure 2 i.e. entries #1 and #3. This behavior violates the
conventional PIT behavior; however, is essential to support the
different QoS treatment.

<do> no it isn’t, if you have a partial order defined over the space of QoS treatments. It’s a problem if all QoS treatments are considered unordered. </do>

In order to support QoS-aware forwarding, the conventional PIT
aggregation needs to be loosened up proportional to the number of QoS
markers; in other words, the QoS treatments. Without this, upstream
forwarder would not get an opportunity to obey each of the QoS
treatments. The theoretical upper bound on the PIT scaling for given
content will be equal to number of QoS markers.

<do> this is an important point. If we can demonstrate that for any given equivalence class in a namespace, the number of different QoS treatments that might be active at any given time is tightly bounded, that might be sufficient </do>

The PIT scaling issue described in Section 7.2 can be handled with an
optimization discussed here.

in PIT. Thus, achieving the Interest aggregation based on the higher

<do> be careful here to have some mathematical rigor about what it means to be “higher”. </do>

Notice that forwarding of Interest with higher QoS marker in spite of
having an already pending with a lower QoS marker, is a breach of
Interest aggregation at PIT in conventional terms; however, it is
necessary to give an opportunity for upstream routers to provide

probability of a priority treatment to the higher QoS marked Interest

<do> sorry, I don’t follow this. Some paths may be more loaded with high QoS treatment than others, and hence if you choose to send a lower marked Interest on the lightly loaded path it could very well be satisfied sooner. It might also hit a closer cache. </do>

The Data packet arrival may satisfy all the PIT entries for the given
content name irrespective of the QoS markers in Data packet. This is
possible mainly because the content itself in Data packet does not
change by different QoS marker in the Interest. Depending on whether
forwarder implements a PIT scaling optimization, two scenarios of
Data forwarding are possible:

<do> yes, however this also determines how the data packet is queued on the downstream interface. </do>

With the PIT optimization described in Section 7.3, it is possible to
satisfy a pending Interest with lower QoS marking with arrival of a
Data packet having higher QoS marker. As a result, a user with lower
QoS subscription may experience a better response time from the
network. Note that this is a legitimate behavior, as ICN is
fundamentally designed to optimize the network round-trip time
providing better user experience.

<do> while this might seem superficially true, there are, at least for some scenarios like adaptive video streaming, good reasons to intentionally delay or otherwise give lower service to requests issued with particular QoS treatments. This, one the other hand, is a longer discussion/explanation than is appropriate for these comments. </do>

<do> However, we’ve now gotten to the end of the main content, and you haven’t talked about the interaction of QoS markers (or in my world, QoS treatments) with caching at all. Any spec that purports to show how QoS works for ICN needs to address not just link resources, but cache resources as well. In some circumstances (e.g. the constrained network scenarios discussed in https://datatracker.ietf.org/doc/draft-gundogan-icnrg-iotqos/) you need to specify the behavior of forwarder memory and forwarding CPU capacity as well. That’s a major missing piece for your work. </do>

  1.  Security Considerations

Depending on where the QoS marker is encoded in the Interest, certain
security attack scenarios against QoS treatment are possible. If the
QoS marker located inside the security envelope, it can be read, but
not changed. Conversely, if the QoS marker is placed outside of the
security envelope, it can be added as hop-by-hop message header and,
therefore, can be modified by the transit ICN forwarders; however, it
also opens it to various attack scenarios.

<do> and these are? Especially, are they new, amplifications of existing attacks, or inherent in the ability of any on-path element to most anything it likes? You also need to distinguish attacks mounted by consumers, those mounted by producers, those mounted by on-path forwarders, and those mounted by off-path elements. </do>

  1.  Summary

PIT aggregation and effect of optimization. The details on design of

  1.  Acknowledgements

provided a useful feedback on proposed name-based encoding scheme and
nameless content objects.

  1.  IANA Considerations

<do> change this to “TBD” since once completed the document would undoubtedly have substantial actions needed by IANA </do>

[End of comments]

[Banner]

[Banner] [Banner]
This e-mail is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and/or otherwise protected from disclosure to anyone other than its intended recipient. Unintended transmission shall not constitute waiver of any privilege or confidentiality obligation. If you received this communication in error, please do not review, copy or distribute it, notify me immediately by email, and delete the original message and any attachments. Unless expressly stated in this e-mail, nothing in this message or any attachment should be construed as a digital or electronic signature.