Subject: [tsvwg] Comments on draft-ietf-tsvwg-transport-encrypt-06

From: Thomas Fossati <[email-masked]>
Date: Mon, 17 Jun 2019 15:53:55 +0000

Hi Gorry, Colin,

I have read your fine draft and gathered a few comments for your
consideration.

I think the intro material (Sections 1 and 2) could be slimmed down
substantially:
- There's a bit of repetition (sometimes literally, e.g.: 2nd and 3rd
  para of the intro section carry the same exact information);
- Content-wise, with regards to setting the context, there seems to be
  good overlap with a few already published RFCs - for example both
  Path signals [RFC8558] and Wire image [RFC8546] could be used to
  factor out a bunch of text here?  Just a suggestion.

The rest is really good material with the right balance of breadth and
depth (IMHO).

However, one of the raison d'etre of the document, as I interpret it, is
to describe which tools & techniques are available to network operators
that can survive transports that mux and prioritise inside an encrypted
envelope.

And while the document does a very good job identifying what *doesn't*
work anymore, the "every time a transport field is encrypted, a passive
measurement tool gets his wings" story, while factually accurate, only
tells one half of the tale.

In that respect, I think the draft should also consider any emerging
techniques & tools.  In particular, ML solutions (especially RNN and
reinforcement learning architectures) look like one very plausible way
this is going to evolve.

Which is not to say these methods are in all cases mature enough for
production.  On the contrary, I think currently very few of them would
be ready for prime time.  But this is one line of work that has shown,
in published research at least, encouraging levels of precision &
accuracy (even with encrypted traffic), while dealing with all the
typical scenarios a network operator faces.  (I've seen ML applied to
fault detection and localisation, QoE/QoS measurements in both real-time
comms and adaptive video, anomalies / misuse / intrusion detection, the
whole lot.)

The thing is that if ML or like approaches are not good enough or not
mature enough, the risk that MiTM and forced fall-back to TCP become a
practical and effective tool in the hands of network operators increases
substantially, which would be pretty unfortunate.  (And yes, we could
have handled the transition more graciously, cf. SPUD/PLUS, but we
decided not to, so here we are.)  In this respect, I think it's critical
to create a shared understanding of what the challenges are and what
concrete actions can be taken to accelerate / incentivise the
development and adoption of ML in this venerable field.  To this end,
it'd be great if the document provided (even succinctly) a critical look
on ML that highlights the potential and actual successes, its
limitations (both in general, and specifically WRT encrypted transports)
as well as the current gaps.

For example, one thing that becomes quickly apparent once you start
surveying the existing literature is that comparing different
ML-for-network-traffic experiments is quite problematic.  The lack of
uniform evaluation metrics as well as standardised labelled datasets
(which is only aggravated by ML's sensitivity on training and validation
data) looks like a gap that urgently needs filling and one where the
IETF is ideally placed to help reducing the observed fragmentation.

OK, I'll stop here - I wanted this to be short and snappy but I got
carried away somehow.  Hopefully my points are still visible above the
noise line.

Thanks again for the great job gathering 20+ years of collective
knowledge and experience.

Cheers, t

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: Colin Perkins <[email-masked]>
Date: Tue, 18 Jun 2019 11:46:14 +0100

Hi Thomas,

Thanks for your feedback. 

I agree with your comments on the introductory sections; there’s certainly scope to tighten up that text.

As for the rest, I don’t disagree that machine learning will be important, but I’m not the right person to write about it. I’m also not clear where such material would fit in the document. It might be better to have a separate document, that this can maybe refer to, that starts to discuss solutions to the issues identified.

Cheers,
Colin

> On 17 Jun 2019, at 16:53, Thomas Fossati <[email-masked]> wrote:
> 
> Hi Gorry, Colin,
> 

-- 
Colin Perkins
https://csperkins.org/

---

From: Eliot Lear <[email-masked]>
Date: Tue, 18 Jun 2019 13:03:26 +0200

+1.  Of course pointers to real world ML solutions (and other solutions as well) should be included.  I like the draft.  Very well written.

Eliot

> On 18 Jun 2019, at 12:46, Colin Perkins <[email-masked]> wrote:
> 
> Hi Thomas,
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 18 Jun 2019 07:41:06 -0700

> IETF is ideally placed to help reducing the observed fragmentation.

Thomas,

ML requires massive amounts of data to be most effect effective, but strong
security mandates that we only expose the minimal amount of data we need
to. So this is the tussle. If someone were to say we need uniform metrics
exposed in transport protocols in the Internet, then the first question to
be asked is "what information about transport protocols is _required_ to be
exposed to the network?" Architecturally and from a position of security,
the answer to this question in the Internet is none. A better question to
ask might be what information would be useful and optional to expose and
how can we limit the exposure to only trusted parties (for instance, I
might trust my local network more than I trust the Internet, so I may be
willing to expose data to the local network if there are sufficient
mechanisms to scope it). That implies the need for a mechanism whereby
hosts can signal the network with ancilary scoped information attached to
packets. IMO the mechanism for this should be Hop-by-Hop options.

Tom

> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sender immediately and do not disclose the
> contents to any other person, use it for any purpose, or store or copy the
> information in any medium. Thank you.
>

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 18 Jun 2019 22:47:25 +0000

Hi Colin,

On 18/06/2019, 11:46, "Colin Perkins" <[email-masked]> wrote:
> As for the rest, I don’t disagree that machine learning will be
> important, but I’m not the right person to write about it. I’m also
> not clear where such material would fit in the document. It might be
> better to have a separate document, that this can maybe refer to, that
> starts to discuss solutions to the issues identified.

That might be a good plan.

Someone who could help with this is David Meyer, I guess.  He did a couple
of talks in Dallas (93) and Seoul (97) that give a very nice overview on
the topic.

Cheers, t

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 18 Jun 2019 22:52:21 +0000

Hi Tom,

> ML requires massive amounts of data to be most effect effective, but
> strong security mandates that we only expose the minimal amount of
> data we need to. So this is the tussle. If someone were to say we need
> uniform metrics exposed in transport protocols in the Internet, then
> the first question to be asked is "what information about transport
> protocols is _required_ to be exposed to the network?" Architecturally
> and from a position of security, the answer to this question in the
> Internet is none. A better question to ask might be what information
> would be useful and optional to expose and how can we limit the
> exposure to only trusted parties (for instance, I might trust my local
> network more than I trust the Internet, so I may be willing to expose
> data to the local network if there are sufficient mechanisms to scope
> it). That implies the need for a mechanism whereby hosts can signal
> the network with ancilary scoped information attached to packets. IMO
> the mechanism for this should be Hop-by-Hop options.

Sorry for the confusion. The thing I am suggesting is to standardise the
interface to the inference engine, not to force higher layers to leak
information.

The (naive?) model I have in mind is one where the ML trainer not only
has the ability to measure passively but also to inject ad hoc
application flows itself - ideally from multiple vantage points.  These
can be then correlated with their blind counterparts both in the
training and check/reinforcement phases.

In principle, the application endpoints don't need to reveal anything,
apart from their network headers and whatever the transport already
exposes (e.g., spin-bit, connection id).  The onus of generating the
otherwise hidden data is on the ML engine owner and whoever chooses to
partner with it.  For example, a mobile network operator could partner
with customers that opt to participate in a crowdsourced measurement
campaign.

To this end, the two log formats that are emerging around QUIC: qlog [1]
and QUIC trace [2] would act as the source of high fidelity
transport-specific information.

On top of those, there could be a bunch of standardised application
specific formats - e.g., time series of MOS scores for real-time A/V
apps, perceived video quality for HAS, $your_QoE_estimator of choice.

And maybe another level of tagging is a set of KPIs at the global scope,
provided by the network, (e.g., congestion levels, percent of rejected
flows) if the available samples are too sparse to reconstruct the big
picture.

This is effectively a hierarchy of standardised log data formats that
can be composed easily, for which pre-processing tools can be shared,
that would help with reproducibility plus all the other advantages that
come with a common, well defined format.

Of course, there's the inherent correlation/anonymity dichotomy that
needs addressing, but I think the IETF has the right expertise in house
to tackle this kind of stuff - the time we spent on making sure RFC7937
was shippable is a testament to that :-)

Cheers, t

[1] https://datatracker.ietf.org/meeting/104/materials/slides-104-tsvarea-logging-tooling-and-debugging-for-modern-network-protocols-robin-marx-00
[2] https://datatracker.ietf.org/meeting/104/materials/slides-104-tsvarea-quic-tracing-victor-vasiliev-00

---

From: Colin Perkins <[email-masked]>
Date: Wed, 19 Jun 2019 18:56:50 +0100

> On 18 Jun 2019, at 23:47, Thomas Fossati <[email-masked]> wrote:

Agree he’d be a good person, if he’s interested in contributing.

-- 
Colin Perkins
https://csperkins.org/

---

From: "Gorry (erg)" <[email-masked]>
Date: Wed, 19 Jun 2019 20:22:02 +0200

> Agree he’d be a good person, if he’s interested in contributing.

Yes, fine with me too.

-  I think we need to be sure this is experience not research.

There are plenty of papers, proposals for new approaches, ioam proposals, etc, that go beyond the current use, and I think these aspirations can’t be in this ID.

Gorry