Subject: [arch-d] Layer violations considered harmful?

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Jun 2019 11:37:48 +1200

Hi,

For something I'm currently working on, I was looking for a citation for "layer violations considered harmful" or something similar.

A little to my surprise, I didn't find it anywhere.

Zimmerman's 1978 paper on the OSI model describes the layered model, but doesn't discuss layer violation as far as I can see.

Dave Clark's 1988 "The design philosophy of the DARPA internet protocols" doesn't either.

RFC1958, which I edited, doesn't either, and that was an explicit choice. Yakov Rekhter wrote to me in 1996: 'In other words, "layer violation" is theory, but when making pragmatic decisions we coul violate the theory...' so we left it out.

So did anyone ever analyse and justify this shibboleth?

Regards
   Brian Carpenter

---

From: [email-masked]
Date: Wed, 5 Jun 2019 17:04:24 -0700

> So did anyone ever analyse and justify this shibboleth?

Isn’t that just the networking equivalent of “don’t violate the abstraction boundary” that is in every OO programming course today?

Tony

---

From: George Michaelson <[email-masked]>
Date: Thu, 6 Jun 2019 10:19:13 +1000

setsockopt() always felt like it was the unavoidable pain inherent in
reaching through your layer to talk to another layer.

MTU exemplifies what goes wrong when you depend on something not well
exposed or encapsulated

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Jun 2019 13:01:18 +1200

> Isn’t that just the networking equivalent of “don’t violate the abstraction boundary” that is in every OO programming course today?

Sure. But if you search the corpus of RFCs, pretty much every mention is a case of justified layer violation ;-). See Richard Li's message.

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Jun 2019 13:32:59 +1200

> reaching through your layer to talk to another layer.

Yes. See the current 6man discussion on anycast for a prime example.
 
> MTU exemplifies what goes wrong when you depend on something not well

Indeed. Layer 4 desparately needs to hear from Layer 3.

Thanks to an off-list hint, I find that RFC874 refers rather negatively
to "over-zealous application of the Principle of Layering". So even in
1982, we had our doubts. At that time, I recall that modular programming
and layered design was what passed for software engineering in general.
And the same author says in RFC872 that 'Figure 1 and Figure 2 might be
all that need be "said."  Their moral is meant to be that in ARPANET-style
layering, layers aren't monoliths.'

    Brian

---

From: Yaakov Stein <[email-masked]>
Date: Thu, 6 Jun 2019 06:02:05 +0000

Brian,

Even in the ITU documents that define layering, such as clause 5 of X.200 (the 7 layer model) and G.800 (the client-server model), 
there is intentionally no mention of layer "violations".

These documents simply state as a matter of fact that entities at layer N communicate with other entities at layer N
(and of course interface entities at layer N-1 and N+1).
It was considered too horrible to contemplate anything else, and thus these are never mentioned.
There is simply no way of expressing a layer violation in the framework of G.800.

In 2012, in response to a liaison from ITU SG15 complaining about the TC layer violation in PTP (IEEE 1588) ,
IEEE 802.1 asserted "Rigorous adherence to layering principles is fundamental to the continued growth of networking".

Y(J)S

-----Original Message-----
From: Architecture-discuss [mailto:[email-masked]] On Behalf Of Brian E Carpenter
Sent: יום ה 06 יוני 2019 04:33
To: George Michaelson <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: [arch-d] Layer violations considered harmful?

 

Thanks to an off-list hint, I find that RFC874 refers rather negatively to "over-zealous application of the Principle of Layering". So even in 1982, we had our doubts. At that time, I recall that modular programming and layered design was what passed for software engineering in general.
And the same author says in RFC872 that 'Figure 1 and Figure 2 might be all that need be "said."  Their moral is meant to be that in ARPANET-style layering, layers aren't monoliths.'

    Brian 

_______________________________________________
Architecture-discuss mailing list
[email-masked]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Farchitecture-discuss&amp;data=01%7C01%7Cyaakov_s%40rad.com%7Cc0366cad84ef409de5b208d6ea1ef493%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&amp;sdata=6YVS7jNjxo1VrTStfBnnedncIPy2X2mkOprUDB1PRQ4%3D&amp;reserved=0

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 6 Jun 2019 10:22:51 +0200

This probably depends on what you consider a "violation". I can see at
least three: "peeking", "tweaking" and "cheating", with the former
obtaining information from a lower layer, e.g., to optimize performance,
and the latter trying to override or alter lower-layer responsibilities.
Having TCP discover path MTU seems like the first. Routers messing with
layer-4+ seems like cheating.

Henning

On Thu, Jun 6, 2019 at 1:38 AM Brian E Carpenter <
[email-masked]> wrote:

> Hi,
>
> For something I'm currently working on, I was looking for a citation for
> "layer violations considered harmful" or something similar.
>
> Zimmerman's 1978 paper on the OSI model describes the layered model, but
> doesn't discuss layer violation as far as I can see.
>
> RFC1958, which I edited, doesn't either, and that was an explicit choice.
> Yakov Rekhter wrote to me in 1996: 'In other words, "layer violation" is
> theory, but when making pragmatic decisions we coul violate the theory...'
> so we left it out.
>
> https://www.ietf.org/mailman/listinfo/architecture-discuss
>

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Thu, 6 Jun 2019 11:30:37 +0200

Hi Brian,

> So did anyone ever analyse and justify this shibboleth?

RFC 3439 https://tools.ietf.org/html/rfc3439
contains a discussion on Layering, but rather in the
purport of "Layering Considered Harmful"
https://tools.ietf.org/html/rfc3439#section-3
discussing the problem of redundant and repeated
functionality in the layers as well as increased
complexity by increased layering.

However, Section 3.1 "Optimization Considered Harmful"
mentions that optimization across layers may cause
also problems (increased complexity, tighter coupling,
lower reliability).

RFC 1122 also discusses "strict layering" a bit:

1.3.1  Organization

         Protocol layering, which is generally used as an organizing
         principle in implementing network software, has also been used
         to organize this document.  In describing the rules, we assume
         that an implementation does strictly mirror the layering of the
         protocols.  Thus, the following three major sections specify
         the requirements for the link layer, the internet layer, and
         the transport layer, respectively.  A companion RFC [INTRO:1]
         covers application level software.  This layerist organization
         was chosen for simplicity and clarity.

         However, strict layering is an imperfect model, both for the
         protocol suite and for recommended implementation approaches.
         Protocols in different layers interact in complex and sometimes
         subtle ways, and particular functions often involve multiple
         layers.  There are many design choices in an implementation,
         many of which involve creative "breaking" of strict layering.
         Every implementor is urged to read references [INTRO:7] and
         [INTRO:8].

Even RFC 817 https://tools.ietf.org/html/rfc817 mentions:

The way to achieve this goal of packet sharing is to break down the
barrier between the layers of the protocols, in a  very  restrained  and
careful  manner, so that a limited amount of information can leak across
the barrier to enable one layer to optimize its behavior with respect to
the desires of the layers above and below it.

So this is probably a starting point.

Regards
 Roland

---

From: John Day <[email-masked]>
Date: Thu, 6 Jun 2019 06:56:04 -0400

Actually, this is not true.  

The first question answered by the OSI Commentaries was “Is the identification of an (N+1)-protocol in an (N)-protocol a layer violation?” The conclusion accepted doing it because at the time, they could see that it was wrong but not why it was happening or how to get around it. Not to mention it was politically incorrect to think otherwise.

Later it was clear that the purpose of these sorts of fields was to identify the syntax of the encapsulated protocol. This insight came when trying design recursive connection establishment for the application layer. It was realized that the so-called Presentation Layer performs the same function for the Application Layer. It was realized this was necessary in some lower layers as well and that these were ill-formed layers. Properly-formed layers did not require PCI (header) fields of this type. (Avoiding registration authorities is very important anyway.)

In addition, Watson showed that (N)-port-ids (the identifier for the binding between the (N)-entity and the (N+1)-entity) and (N)-connection-endpoint-ids ((N)-CEPI, the identifier for the instance of a protocol state machine for a single flow (whether the flow is connection or connectionless)) should be distinct. The (N)-port-ids should not be carried in an (N)-protocol. Put another way, no information (only data, which should be encrypted by the layer above if it hasn’t been already) shared with an adjacent layer should be carried in protocol.

This has significant (positive) implications for protocols and security.

Agreed that Deep Layer Inspection is a layer violation and a security violation.  However, in a proper internet architecture, i.e. where the internet layer overlays the network layer, MTU is a property of the layer. Of course, this is not what the Internet did.

Take care,
John Day

> On Jun 6, 2019, at 02:02, Yaakov Stein <[email-masked]> wrote:
> 
> Brian,
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 7 Jun 2019 08:45:06 +1200

Thanks to everybody who replied, on and off list. Maybe the discussion needs to move to the internet-history list at this point. At least in recent history, some degree of layer violation seems to have become quite respectable.

Regards
   Brian Carpenter

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Jun 2019 07:23:50 -0700

> On Jun 6, 2019, at 1:45 PM, Brian E Carpenter <[email-masked]> wrote:
> 
> At least in recent history, some degree of layer violation seems to have become quite respectable.

Perhaps tolerated with different levels of pain, but I doubt the bulk would call it “respectable”.

Most layer “violations” appear to failures of protocol designs in one of two ways:
1) poor design of the protocol layering in the first place (as John pointed out)
2) poor design of new features hat break those layerings

E.g., to the point of MTU, as John mentioned every layer has a transmission unit. In fact, every layer has (or should have) what every other layer has. OSI not only didn’t recognize that recursive structure (as John and I have repeatedly noted), it was designed on the very principle that the opposite was true.

OSI was designed based on the principle of separation of function, that each layer had a separate, independent role and that protocol capabilities were uniquely tied to a particular layer. The fact that this was wrong is itself what led to the need to violate those layers.

If every layer managed its own MTU, had its own *independent* identifiers, etc., the *need* for layer violation would cease. That’s not to say it would disappear - even well-designed, modular, layered systems have users who will rationalize a need to violate those layers, typically to achieve the precise behavior the layering was intended to inhibit.

Joe

---

From: John Day <[email-masked]>
Date: Mon, 10 Jun 2019 10:53:45 -0400

There is something wrong here. I pretty much agree with Joe!  ;-)

See below.

> On Jun 10, 2019, at 10:23, Joe Touch <[email-masked]> wrote:
> 
> 
> 
>> On Jun 6, 2019, at 1:45 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
>> 

Mostly what happened in the OSI lower layers that indicated something was wrong was that functions were repeated in layers.*  As I said, the main violation was ‘identifying an (N+1)-protocol in an (N)-layer.” Which at the time, we could see it was a violation, but couldn’t see what the solution was. We only figured that out later when a recursive application connection establishment protocol was designed and specified.

In the upper layers, it turned out there were layer violations, primarily due to the PTTs stealing the Session Layer for Videotex. Remember that!? In 1983, the upper 3 layers were effectively made into one by ‘adjusting’ the protocol specifications so they could be done as a single state machine. (There was another question on the Reference Model, that dealt with which layers had to create flows sequentially and which could be combined. The upper 3 could be combined.) The lower 4 were done sequentially but because there was multiplexing in them, not all at the same time, nor was there a one-to-one mapping between flows at (N) and at (N-1)-). Later still, the upper layers were re-organized so that Session Functional Units were ‘application modules’ that could be combined to create application protocols. Presentation became a function.  All of this combined with the recursive application layer structure made a very powerful structure.
> If every layer managed its own MTU, had its own *independent* identifiers, etc., the *need* for layer violation would cease. That’s not to say it would disappear - even well-designed, modular, laye…

This is true.  The original internet concept was between protocol translation and an overlay. (see Abbate) The ITU chose protocol conversion, which of course would have different MTUs in layers of the same rank and require fragmentation between layers of the same rank.  INWG, and later OSI, adopted the overlay approach, so that MTU was a property of the layer. However around 1980 or so, the Internet took the ITU approach. This was when the term ‘gateway’ was replaced by 'router.’

Take care,
John

*Actually CYCLADES first detected this problem, but tried to avoid it by defining in the overlay approach that the Transport Layer had ‘internet’ addresses and various networks had Network Layers with network addresses. (see Alex McKenzie’s article on INWG96. All of the *transport protocols* in that discussion had addresses, and operated over network layers with network addresses.) This was in the mid-70s before IP was separated from TCP. After IP was separated from TCP and since all of the ’networks’ DARPA was working with except the ARPANET were multiaccess media, i.e. layer 2 protocols. The IP became a network layer protocol and the Internet found itself in the ITU model, where it has been ever since. Notice the ubiquitous adoption today of the ISDN concepts of control and data planes.

---

From: Yaakov Stein <[email-masked]>
Date: Tue, 11 Jun 2019 04:40:34 +0000

At the risk of starting a debate that I really don’t want to start,
I have to state that part of the misunderstanding here concerns what we mean by a “layer”.

<diatribe>

I think most of you are talking only about the outdated OSI/X.200 layering,
which is only one model, was pretty much arbitrary, and is missing many of the features now needed.
Were we to be speaking of G.80x layering then the answer would be a clear – no layer violations are respectable.

The reason that we sanction layer violations in OSI is simply that it is outdated.
Whenever I hear people calling MPLS “layer 2.5”
or whenever I see a protocol stack containing IP over MPLS over Ethernet over PW over MPLS over Ethernet,
I ask why people are still holding on to this model.
Ditto whenever people try explaining why “session” and “presentation” are layers,
but security and OAM are not.

In a pure layering model every layer network (NOT network layer) is an independent network,
potentially operated by a different business entity,
and with its own instantiation of all the necessary features of a network – addressing, OAM, security, etc.
This is how service providers view their network services,
and a layer violation (if technically possible) is a criminal action (potentially stealing service or access to resources, subverting proper operation, etc.).

</diatribe>

When using an arbitrary layering model that has only historical (and not commercial) justification,
then layer violations may indeed be justified.

Y(J)S

From: John Day [mailto:[email-masked]]
Sent: יום ב 10 יוני 2019 17:54
To: Joe Touch <[email-masked]>
Cc: Brian E Carpenter <[email-masked]>; Yaakov Stein <[email-masked]>; [email-masked]
Subject: Re: [arch-d] Layer violations considered harmful?

See below.

On Jun 10, 2019, at 10:23, Joe Touch <[email-masked]<mailto:[email-masked]>> wrote:

On Jun 6, 2019, at 1:45 PM, Brian E Carpenter <[email-masked]<mailto:[email-masked]>> wrote:

Take care,
John

Joe

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 Jun 2019 07:36:33 +0200

On Jun 11, 2019, at 06:40, Yaakov Stein <[email-masked]> wrote:
> […]

+100

Actually, the numbered layers we talk about today (L1-4, L7) have quite different boundaries from those envisioned 41 years ago (in 1978) for OSI.  ISO 7498/X.200 (“the OSI model”) is utterly irrelevant today.  When people still talk about “OSI layers”, ask them whether they have read that thing.  Enough said…  
The now traditional layering is described in RFC 1122 (of 1989, almost 30 years ago).

BTW, layer violations can be fine, if you know that what you are doing there will have to evolve in parallel with *all* those layers (no separation of concern).  For instance, header compression is organized layer violation.

Grüße, Carsten

---

From: Guntur Wiseno Putra <[email-masked]>
Date: Tue, 11 Jun 2019 15:06:50 +0700

Dear Architecture-discuss,

It is supposedly a story on "layer"; and is perhaps helpful:

"A Survey of Protocols and Standards of Internet of Things" by Salman, Tara
& Raj Jain (Arxiv-Cornell University, February 2019 or Journal for Advance
Compouting & Communications, March 2017)

https://arxiv.org/abs/1903.11549

Among this story referred to the IETF.

Regard,
Guntur Wiseno Putra

Pada Selasa, 11 Juni 2019, Carsten Bormann <[email-masked]> menulis:

> Actually, the numbered layers we talk about today (L1-4, L7) have quite
> different boundaries from those envisioned 41 years ago (in 1978) for OSI.
> ISO 7498/X.200 (“the OSI model”) is utterly irrelevant today.  When people
> still talk about “OSI layers”, ask them whether they have read that thing.
> Enough said…
> BTW, layer violations can be fine, if you know that what you are doing
> there will have to evolve in parallel with *all* those layers (no
> separation of concern).  For instance, header compression is organized
> layer violation.
>
> Grüße, Carsten
>

---

From: Yaakov Stein <[email-masked]>
Date: Tue, 11 Jun 2019 08:35:45 +0000

Indeed this paper refers to the IETF (amongst other SDOs) and talks about the different layers of IoT.

What precisely is the relevance to the subject of this thread?

Y(J)S

From: Guntur Wiseno Putra [mailto:[email-masked]]
Sent: יום ג 11 יוני 2019 11:07
To: Carsten Bormann <[email-masked]>
Cc: Yaakov Stein <[email-masked]>; [email-masked]
Subject: Re: [arch-d] Layer violations considered harmful?

"A Survey of Protocols and Standards of Internet of Things" by Salman, Tara & Raj Jain (Arxiv-Cornell University, February 2019 or Journal for Advance Compouting & Communications, March 2017)

https://arxiv.org/abs/1903.11549<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1903.11549&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=MCRn1HEw4xqlWcfX%2BB37GUOlHRNe4P%2FefvsDzSA%2BRkE%3D&reserved=0>

Regard,
Guntur Wiseno Putra

Pada Selasa, 11 Juni 2019, Carsten Bormann <[email-masked]<mailto:[email-masked]>> menulis:

+100

Grüße, Carsten

_______________________________________________
Architecture-discuss mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/architecture-discuss<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Farchitecture-discuss&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=Knq1i3lfzD80fJAJpQs9GAxhStLvyurXi4pxwKX78Uc%3D&reserved=0>

---

From: Guntur Wiseno Putra <[email-masked]>
Date: Tue, 11 Jun 2019 18:53:09 +0700

The paper referred to the ITU and the IEEE beside the IETF.

The last 2nd paragraph of the introduction, as it told us about current IoT
standards and protocols being developed for different layers of network
stack, may add elements of context in talking about layers procedures...

Regard,
Guntur Wiseno Putra

Pada Selasa, 11 Juni 2019, Yaakov Stein <[email-masked]> menulis:

> *From:* Guntur Wiseno Putra [mailto:[email-masked]]
> *Sent:* יום ג 11 יוני 2019 11:07
> *To:* Carsten Bormann <[email-masked]>
> *Cc:* Yaakov Stein <[email-masked]>; [email-masked]
> *Subject:* Re: [arch-d] Layer violations considered harmful?
>
>
>
> "A Survey of Protocols and Standards of Internet of Things" by Salman,
> Tara & Raj Jain (Arxiv-Cornell University, February 2019 or Journal for
> Advance Compouting & Communications, March 2017)
>
>
>
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1903.11549&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=MCRn1HEw4xqlWcfX%2BB37GUOlHRNe4P%2FefvsDzSA%2BRkE%3D&reserved=0>
>
>
>
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Farchitecture-discuss&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=Knq1i3lfzD80fJAJpQs9GAxhStLvyurXi4pxwKX78Uc%3D&reserved=0>
>
>

---

From: Guntur Wiseno Putra <[email-masked]>
Date: Tue, 11 Jun 2019 18:57:47 +0700

The previous message of mine should be:

As it was said in the last 2nd paragraph of the introduction, it gave an
overview about current IoT standards and protocols being developed for
different layers of network stack: thus, it may add elements of context in
talking about layers procedures...

Regard,
Guntur Wiseno Putra

Pada Selasa, 11 Juni 2019, Guntur Wiseno Putra <[email-masked]> menulis:

> The last 2nd paragraph of the introduction, as it told us about current
> IoT standards and protocols being developed for different layers of network

---

From: John Day <[email-masked]>
Date: Tue, 11 Jun 2019 08:04:38 -0400

Thanks, I will read this with interest.  We did a similar survey (which we haven’t published yet) that concluded the primary purpose of different IoT standards (especially the data transfer aspects) was to create barriers to competition.

As for the other things, the paper seems to touch on as on-going issues, security, mobility, availability, etc. are either engineering issues or straightforward architecture issues.

Thanks again, I will enjoy reading it.

Take care,
John

> On Jun 11, 2019, at 07:53, Guntur Wiseno Putra <[email-masked]> wrote:
> 
> The last 2nd paragraph of the introduction, as it told us about current IoT standards and protocols being developed for different layers of network stack, may add elements of context in talking about layers procedures...
> 
> 
> Regard,
> Guntur Wiseno Putra
> 
> Pada Selasa, 11 Juni 2019, Yaakov Stein <[email-masked] <mailto:[email-masked]>> menulis:
> From: Guntur Wiseno Putra [mailto:[email-masked] <mailto:[email-masked]>] 
> To: Carsten Bormann <[email-masked] <mailto:[email-masked]>>
> Cc: Yaakov Stein <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>
> https://arxiv.org/abs/1903.11549 <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1903.11549&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=MCRn1HEw4xqlWcfX%2BB37GUOlHRNe4P%2FefvsDzSA%2BRkE%3D&reserved=0>
>  
> 
> Pada Selasa, 11 Juni 2019, Carsten Bormann <[email-masked] <mailto:[email-masked]>> menulis:
> 
> On Jun 11, 2019, at 06:40, Yaakov Stein <[email-masked] <mailto:[email-masked]>> wrote:
> > 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/architecture-discuss <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Farchitecture-discuss&data=01%7C01%7Cyaakov_s%40rad.com%7Cebf1f929452049ad655808d6ee43c350%7Cf9047108cc2c4e4897a343fad1b3bf9d%7C1&sdata=Knq1i3lfzD80fJAJpQs9GAxhStLvyurXi4pxwKX78Uc%3D&reserved=0>_______________________________________________