Subject: [Coin] 答复: Draft on Transport Protocol Issues

From: "Hejianfei (Jeffrey)" <[email-masked]>
Date: Mon, 4 Nov 2019 08:57:26 +0000

Hi Ike,
(chair hat off)
Some (late) thoughts about your ideas on transport protocol issues(thank you for raising them).

1, Data plane abstraction: ICN or session style?
I see two potential alternatives for in-net compute. One is “ICN-style”, e.g. two examples below.
                http://www.named-function.net<http://www.named-function.net/>,
https://conferences.sigcomm.org/acm-icn/2017/proceedings/icn17-31.pdf
In that world, “end-to-end” is totally different with traditional TCP/IP network, so the questions for the transport are also different.
The other way is “session-style”, to explore if we can continue to use the traditional network notions such as session, address etc. The ConvergeCast in some textbooks is close to in-net compute, although I am not sure if they have successful implementations in real world. Reference: http://www.cs.yale.edu/homes/aspnes/pinewiki/ConvergeCast.html.
As I told my friends in ICN community, I am happy to see ICN coming true. But I am also interested if “session-style” can work for in-net compute. There are already some ad hoc implementations, however too early to say if we can generalize them.

2, Pain and gain
As you listed correctly and clearly, the traditional transport can’t fit well with in-net compute. We need to modify. At least two functions of transport are usually required in my view: reliability(dealing with the packet loss) and rate control( dealing with the congestion). Even these two basic requirements are challenging in the context of in-net compute. For example, the packet loss should be considered in two cases: before the compute and after. Further, as you pointed out, generally, for the network devices to compute on the packets, they should be able to identify the service instance( correlated packets from different sources), the demarcation of “operators” and “operands” inside the packets.
The paper (https://arxiv.org/abs/1903.06701) implements above UDP, while my team is trying to implement at “message” layer with RoCEv2 NIC.
I think it is probably feasible in principle, certainly it is not easy. The question is if the cost can be justified by enough gain.
In the LAN scenarios like DCN, the way to justify is clear based on my discussion with some colleagues working on distributed systems. For example, for the AI case, if the solution can increase the training speed by 20%, it is enough for them to adopt a new network solution, since the total cost of networks is less than 10% in the whole solution(high performance GPUs are very expensive) and lifecycle of switches are also becoming shorter inside DC.
Different scenarios may have different answers: DCN, industrial networks, carrier’s networks (WAN or metro net to connect the edge computing).

Just my two cents,
Jeffrey

发件人: Coin [mailto:[email-masked]] 代表 Ike Kunze
发送时间: 2019年10月2日 20:31
收件人: Dirk Kutscher <[email-masked]>
抄送: Jan Rüth <[email-masked]>; Klaus <[email-masked]>; [email-masked]
主题: Re: [Coin] Draft on Transport Protocol Issues

Hi Dirk,

thank you for your valuable feedback.
I agree with you in that we should not directly try to derive an overall COIN transport protocol/architecture, but as I said,
our goal is to collect and discuss general challenges which we face when deploying COIN elements.
I think that the building blocks that you mention could very well be a result of our discussions and could illustrate
how the challenges can be approached.

In this context, I think that categorizing different kinds of scenarios and interactions, as you propose, can help us to
identify which of the challenges might be solved using the same or similar building blocks.
However, I think that finding a suitable level of abstraction for the discussions will be challenging.

Whatever categorization we might choose here, dealing with authorizing and authenticating COIN elements
should be a focus of our discussions as this in fact is where the end-to-end principle that we always mention breaks.
We are very interested to hear about ideas how identifying the COIN elements might be possible.

Trying to solve a particular problem with experimental work is a nice suggestion.
Do you know of other work that could relate to or already tries to solve some of the problems here?

Best,
Ike

Am 01.10.2019 um 19:05 schrieb Dirk Kutscher <[email-masked]<mailto:[email-masked]>>:

Hi Ike,
Thanks for these thoughts -- these are certainly relevant points IMO.
Nobody is going to stop if you offer writing up more details, but here are some suggestions from my side:
·         if you talk about challenges, it would be most useful to describe the assumptions upfront. I.e., what is the COIN application, what is the environment?
·         what typically works well is starting with some experimental work that tries to solve a particular problem — typically yields specific challenges.
·         your list is really comprehensive (transport protocols, provenance verification etc.). What might work in the end is a “building blocks” approach (not a grand architecture) — just my thinking…
·         the “middlebox” notion is IMO fundamentally wrong (as an intercepting device that is transparent to other entities). We may not need to know where a function resides, but we should know its identity to authorize access to our data. (I know that you are not suggesting it here.)
·         if you want to spend some effort in analyzing challenges, a good first step could also be to categorize different kind of scenarios and interactions. For example, a distributed computing approach such as the one in our recent CFN paper could be different from pipelined processing system where every ADU has to pass every function in a sequence.
Happy to discuss more about this!
Dirk
On 30 Sep 2019, at 13:52, Ike Kunze wrote:
Hello list,

attention: long email incoming.
TL:DR: We would like to know if the transport protocol issues are of interest to the community
and we are looking for collaborators to work on a corresponding draft.

As suggested in Montreal, we at COMSYS would like to work on a draft to collect the implications of COIN
on transport protocols as well as challenges or obstacles that might hinder widespread deployment of COIN
like the often-mentioned end-to-end principle.
In the meeting, we had the impression that the problems that we see are not clear to everyone which is why
we would first like to briefly share our thoughts on the topic with the list to see if these considerations are
of interest to the community in which case we would then proceed and start writing the actual draft.

Today’s transport protocols offer a variety of functionality based on the notion that the network is to be treated
as an unreliable communication medium which does not alter the transmitted data
(which, in practice, is not really true due to middleboxes).
Some, like TCP, establish a reliable connection on top of the unreliable network while others,
like UDP, simply transmit datagrams without a connection and without guarantees into the network.

In this overall context, we see several issues and challenges that arise when we combine traditional transport protocols with COIN:

1. Addressing:
Traditionally, end-hosts directly address each other.
With COIN, one host might want to specify which kind of functionality should be applied to the transmitted data
on the path and where or by whom it should be applied.
Instead of direct addressing of the end-hosts, indirection mechanisms are needed to route a packet along
a pre-defined or dynamically chosen path which realizes the desired functionality.
Related concepts which might be of use are Segment and Source Routing as well as (Service/Network) Function Chaining/Composition.

2. Flow granularity:
Assuming that there is a suitable addressing scheme which allows to define which kinds of functionality
should be applied to the transmitted data, a next question that arises is how the transmitted data
is to be treated by the devices implementing the functionality.
A TCP stream, for example, where the transmitted data is dynamically distributed across segments
should perhaps be treated differently than UDP datagrams where the application defines how the data
is distributed across the distinct datagrams.
Underlying this simple example lies the general question of how the data should be interpreted by the COIN elements:
should every packet be treated (1) individually, (2) as part of a message where the content of surrounding packets might be relevant
or (3) as part of a byte stream where all previous packets need to be taken into consideration?

3. Security and Authentication:
Assuming that there is a suitable addressing scheme and the COIN elements know how to treat the packets.
In this case, data transmitted from host A to host B will now be altered on the way.
This, of course, opens the door for foul play as every middlebox, both (misbehaving) COIN elements and
‘traditional’ middleboxes, could simply start altering parts of the original data.
What is needed is a mechanism so that the receiving host can verify (a) how and (b) by whom the data
has been altered on the way, i.e., some form of authentication.
Thinking a step further and considering developments such as QUIC where almost all transmitted data is encrypted,
there is a need for concepts which allow to include security mechanisms like encryption into COIN frameworks.

4. Advanced Transport Features:
Except for the crypto aspect, all of the mentioned challenges are relevant for every transport protocol
as they define the interaction between the transport and COIN.
This completely leaves out the fact that there is a significant difference in functionality between different transport protocols.
UDP, for example, is a connectionless protocol which does not offer any reliability while TCP first sets up
a dedicated connection and then ensures the successful transmission of all data.
In addition to that it adds flow and congestion control to avoid overloading the receiving end-host and the network.
When selecting a transport protocol for COIN, such advanced features have to be considered as well.
For illustration, consider the simple retransmission mechanism of TCP.
If we transfer this principle to a communication setting with COIN elements in between, it has to be defined how loss can
be recovered if packets are, e.g., lost at the last stage of a chain of COIN elements.
This question has been already worked on in the context of multicast transport protocols.
Another angle on this aspect is that different COIN devices have different computational and storage capacities which could,
for example, become interesting if they are supposed to embed into TCP for which they might be forced to hold some form
of TCP’s state which might not be possible on each type of COIN element.
The choice of devices could hence affect the types of transport protocols that can be operated on the COIN networks which
might make it necessary to define different classes of COIN-ready transport protocols.

In the draft, we would like to present these challenges and more in more detail and also give our impression
on how well they could be solved using existing transport protocols.
Depending on our progress, we would then like to start working out design requirements for an “ideal” COIN transport protocol.

Please let us know if this topic is of general interest to the community in which case we would start working on the draft
and please let us also know if you would like to join our efforts.

Best wishes,

Ike Kunze
--
Ike Kunze, M.Sc.
Researcher

COMSYS - Communication and Distributed Systems
RWTH Aachen University
Ahornstr. 55
52074 Aachen, Germany
Tel: +49 241 80-21422
[email-masked]<mailto:[email-masked]>
http://www.comsys.rwth-aachen.de/team/ike-kunze/

--
Coin mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/coin

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Mon, 4 Nov 2019 03:15:36 -0800

Ike:

I gather you or a team member will want to present there transport issues
at the meeting? Should we add you to the agenda?

Jeffrey: is this also something you would like to present?

mjm

Marie-José Montpetit, Ph.D.
Research Affiliate, MIT Media Laboratory
[email-masked]
[email-masked]

On November 4, 2019 at 3:57:59 AM, Hejianfei (Jeffrey) (
[email-masked]) wrote:

Hi Ike,

(chair hat off)

Some (late) thoughts about your ideas on transport protocol issues(thank
you for raising them).

I see two potential alternatives for in-net compute. One is “ICN-style”,
e.g. two examples below.

                http://www.named-function.net,

In that world, “end-to-end” is totally different with traditional TCP/IP
network, so the questions for the transport are also different.

The other way is “session-style”, to explore if we can continue to use the
traditional network notions such as session, address etc. The ConvergeCast
in some textbooks is close to in-net compute, although I am not sure if
they have successful implementations in real world. Reference:
http://www.cs.yale.edu/homes/aspnes/pinewiki/ConvergeCast.html.

As I told my friends in ICN community, I am happy to see ICN coming true.
But I am also interested if “session-style” can work for in-net compute.
There are already some ad hoc implementations, however too early to say if
we can generalize them.

2, Pain and gain

As you listed correctly and clearly, the traditional transport can’t fit
well with in-net compute. We need to modify. At least two functions of
transport are usually required in my view: reliability(dealing with the
packet loss) and rate control( dealing with the congestion). Even these two
basic requirements are challenging in the context of in-net compute. For
example, the packet loss should be considered in two cases: before the
compute and after. Further, as you pointed out, generally, for the network
devices to compute on the packets, they should be able to identify the
service instance( correlated packets from different sources), the
demarcation of “operators” and “operands” inside the packets.

The paper (https://arxiv.org/abs/1903.06701) implements above UDP, while my
team is trying to implement at “message” layer with RoCEv2 NIC.

I think it is probably feasible in principle, certainly it is not easy. The
question is if the cost can be justified by enough gain.

In the LAN scenarios like DCN, the way to justify is clear based on my
discussion with some colleagues working on distributed systems. For
example, for the AI case, if the solution can increase the training speed
by 20%, it is enough for them to adopt a new network solution, since the
total cost of networks is less than 10% in the whole solution(high
performance GPUs are very expensive) and lifecycle of switches are also
becoming shorter inside DC.

Different scenarios may have different answers: DCN, industrial networks,
carrier’s networks (WAN or metro net to connect the edge computing).

Just my two cents,

Jeffrey

*发件人**:* Coin [mailto:[email-masked]] *代表 *Ike Kunze
*发送时间:* 2019年10月2日 20:31
*收件人:* Dirk Kutscher <[email-masked]>
*抄送:* Jan Rüth <[email-masked]>; Klaus <
[email-masked]>; [email-masked]
*主题:* Re: [Coin] Draft on Transport Protocol Issues

Hi Dirk,

I agree with you in that we should not directly try to derive an overall
COIN transport protocol/architecture, but as I said,

our goal is to collect and discuss general challenges which we face when
deploying COIN elements.

I think that the building blocks that you mention could very well be a
result of our discussions and could illustrate

In this context, I think that categorizing different kinds of scenarios and
interactions, as you propose, can help us to

identify which of the challenges might be solved using the same or similar
building blocks.

However, I think that finding a suitable level of abstraction for the
discussions will be challenging.

Whatever categorization we might choose here, dealing with authorizing and
authenticating COIN elements

should be a focus of our discussions as this in fact is where the
end-to-end principle that we always mention breaks.

We are very interested to hear about ideas how identifying the COIN
elements might be possible.

Trying to solve a particular problem with experimental work is a nice
suggestion.

Do you know of other work that could relate to or already tries to solve
some of the problems here?

Best,

Ike

Am 01.10.2019 um 19:05 schrieb Dirk Kutscher <[email-masked]>:

Hi Ike,

Nobody is going to stop if you offer writing up more details, but here are
some suggestions from my side:

·         if you talk about challenges, it would be most useful to describe
the assumptions upfront. I.e., what is the COIN application, what is the
environment?

·         what typically works well is starting with some experimental work
that tries to solve a particular problem — typically yields specific
challenges.

·         your list is really comprehensive (transport protocols,
provenance verification etc.). What might work in the end is a “building
blocks” approach (not a grand architecture) — just my thinking…

·         the “middlebox” notion is IMO fundamentally wrong (as an
intercepting device that is transparent to other entities). We may not need
to know where a function resides, but we should know its identity to
authorize access to our data. (I know that you are not suggesting it here.)

·         if you want to spend some effort in analyzing challenges, a good
first step could also be to categorize different kind of scenarios and
interactions. For example, a distributed computing approach such as the one
in our recent CFN paper could be different from pipelined processing system
where every ADU has to pass every function in a sequence.

Dirk

Hello list,

attention: long email incoming.
TL:DR: We would like to know if the transport protocol issues are of
interest to the community
and we are looking for collaborators to work on a corresponding draft.

As suggested in Montreal, we at COMSYS would like to work on a draft to
collect the implications of COIN
on transport protocols as well as challenges or obstacles that might hinder
widespread deployment of COIN
like the often-mentioned end-to-end principle.
In the meeting, we had the impression that the problems that we see are not
clear to everyone which is why
we would first like to briefly share our thoughts on the topic with the
list to see if these considerations are
of interest to the community in which case we would then proceed and start
writing the actual draft.

Today’s transport protocols offer a variety of functionality based on the
notion that the network is to be treated
as an unreliable communication medium which does not alter the transmitted
data
(which, in practice, is not really true due to middleboxes).
Some, like TCP, establish a reliable connection on top of the unreliable
network while others,
like UDP, simply transmit datagrams without a connection and without
guarantees into the network.

In this overall context, we see several issues and challenges that arise
when we combine traditional transport protocols with COIN:

1. Addressing:
Traditionally, end-hosts directly address each other.
With COIN, one host might want to specify which kind of functionality
should be applied to the transmitted data
on the path and where or by whom it should be applied.
Instead of direct addressing of the end-hosts, indirection mechanisms are
needed to route a packet along
a pre-defined or dynamically chosen path which realizes the desired
functionality.
Related concepts which might be of use are Segment and Source Routing as
well as (Service/Network) Function Chaining/Composition.

2. Flow granularity:
Assuming that there is a suitable addressing scheme which allows to define
which kinds of functionality
should be applied to the transmitted data, a next question that arises is
how the transmitted data
is to be treated by the devices implementing the functionality.
A TCP stream, for example, where the transmitted data is dynamically
distributed across segments
should perhaps be treated differently than UDP datagrams where the
application defines how the data
is distributed across the distinct datagrams.
Underlying this simple example lies the general question of how the data
should be interpreted by the COIN elements:
should every packet be treated (1) individually, (2) as part of a message
where the content of surrounding packets might be relevant
or (3) as part of a byte stream where all previous packets need to be taken
into consideration?

3. Security and Authentication:
Assuming that there is a suitable addressing scheme and the COIN elements
know how to treat the packets.
In this case, data transmitted from host A to host B will now be altered on
the way.
This, of course, opens the door for foul play as every middlebox, both
(misbehaving) COIN elements and
‘traditional’ middleboxes, could simply start altering parts of the
original data.
What is needed is a mechanism so that the receiving host can verify (a) how
and (b) by whom the data
has been altered on the way, i.e., some form of authentication.
Thinking a step further and considering developments such as QUIC where
almost all transmitted data is encrypted,
there is a need for concepts which allow to include security mechanisms
like encryption into COIN frameworks.

4. Advanced Transport Features:
Except for the crypto aspect, all of the mentioned challenges are relevant
for every transport protocol
as they define the interaction between the transport and COIN.
This completely leaves out the fact that there is a significant difference
in functionality between different transport protocols.
UDP, for example, is a connectionless protocol which does not offer any
reliability while TCP first sets up
a dedicated connection and then ensures the successful transmission of all
data.
In addition to that it adds flow and congestion control to avoid
overloading the receiving end-host and the network.
When selecting a transport protocol for COIN, such advanced features have
to be considered as well.
For illustration, consider the simple retransmission mechanism of TCP.
If we transfer this principle to a communication setting with COIN elements
in between, it has to be defined how loss can
be recovered if packets are, e.g., lost at the last stage of a chain of
COIN elements.
This question has been already worked on in the context of multicast
transport protocols.
Another angle on this aspect is that different COIN devices have different
computational and storage capacities which could,
for example, become interesting if they are supposed to embed into TCP for
which they might be forced to hold some form
of TCP’s state which might not be possible on each type of COIN element.
The choice of devices could hence affect the types of transport protocols
that can be operated on the COIN networks which
might make it necessary to define different classes of COIN-ready transport
protocols.

In the draft, we would like to present these challenges and more in more
detail and also give our impression
on how well they could be solved using existing transport protocols.
Depending on our progress, we would then like to start working out design
requirements for an “ideal” COIN transport protocol.

Please let us know if this topic is of general interest to the community in
which case we would start working on the draft
and please let us also know if you would like to join our efforts.

Best wishes,

Ike Kunze
-- 
Ike Kunze, M.Sc.
Researcher

COMSYS - Communication and Distributed Systems
RWTH Aachen University
Ahornstr. 55
52074 Aachen, Germany
Tel: +49 241 80-21422
[email-masked]
http://www.comsys.rwth-aachen.de/team/ike-kunze/

-- 
Coin mailing list
[email-masked]

-- 
Coin mailing list
[email-masked]

---

From: Ike Kunze <[email-masked]>
Date: Mon, 4 Nov 2019 18:05:22 +0000

Hi Marie-José,

thanks for the offer.
We actually wanted to have offline discussions first, but if you think that discussing the transport issues
in a larger forum during the COIN meeting makes sense, Klaus could do a short presentation as he will be in Singapore.
Have a look at the draft [1] first, before deciding if it makes sense to add it to the agenda.

Best,
Ike

[1] https://www.ietf.org/id/draft-kunze-coinrg-transport-issues-00.txt
--
Ike Kunze, M.Sc.
Researcher, Ph.D. Student

Am 04.11.2019 um 12:15 schrieb Marie-Jose Montpetit <[email-masked]<mailto:[email-masked]>>:

Ike:

I gather you or a team member will want to present there transport issues at the meeting? Should we add you to the agenda?

mjm

Marie-José Montpetit, Ph.D.
Research Affiliate, MIT Media Laboratory
[email-masked]<mailto:[email-masked]>
[email-masked]<mailto:[email-masked]>

On November 4, 2019 at 3:57:59 AM, Hejianfei (Jeffrey) ([email-masked]<mailto:[email-masked]>) wrote:
Hi Ike,
(chair hat off)
Some (late) thoughts about your ideas on transport protocol issues(thank you for raising them).

1, Data plane abstraction: ICN or session style?
I see two potential alternatives for in-net compute. One is “ICN-style”, e.g. two examples below..
                http://www.named-function.net<http://www.named-function.net/>,
https://conferences.sigcomm.org/acm-icn/2017/proceedings/icn17-31.pdf
In that world, “end-to-end” is totally different with traditional TCP/IP network, so the questions for the transport are also different.
The other way is “session-style”, to explore if we can continue to use the traditional network notions such as session, address etc. The ConvergeCast in some textbooks is close to in-net compute, although I am not sure if they have successful implementations in real world. Reference:http://www.cs.yale.edu/homes/aspnes/pinewiki/ConvergeCast.html.
As I told my friends in ICN community, I am happy to see ICN coming true. But I am also interested if “session-style” can work for in-net compute. There are already some ad hoc implementations, however too early to say if we can generalize them.

2, Pain and gain

Just my two cents,
Jeffrey

发件人: Coin [mailto:[email-masked]<mailto:[email-masked]>] 代表 Ike Kunze
发送时间: 2019年10月2日 20:31
收件人: Dirk Kutscher <[email-masked]<mailto:[email-masked]>>
抄送: Jan Rüth <[email-masked]<mailto:[email-masked]>>; Klaus <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
主题: Re: [Coin] Draft on Transport Protocol Issues

Hi Dirk,

Best,
Ike

Hi Ike,
Thanks for these thoughts -- these are certainly relevant points IMO.
Nobody is going to stop if you offer writing up more details, but here are some suggestions from my side:
•         if you talk about challenges, it would be most useful to describe the assumptions upfront. I.e., what is the COIN application, what is the environment?
•         what typically works well is starting with some experimental work that tries to solve a particular problem — typically yields specific challenges.
•         your list is really comprehensive (transport protocols, provenance verification etc.). What might work in the end is a “building blocks” approach (not a grand architecture) — just my thinking…
•         the “middlebox” notion is IMO fundamentally wrong (as an intercepting device that is transparent to other entities).. We may not need to know where a function resides, but we should know its identity to authorize access to our data. (I know that you are not suggesting it here.)
•         if you want to spend some effort in analyzing challenges, a good first step could also be to categorize different kind of scenarios and interactions. For example, a distributed computing approach such as the one in our recent CFN paper could be different from pipelined processing system where every ADU has to pass every function in a sequence.
Happy to discuss more about this!
Dirk
On 30 Sep 2019, at 13:52, Ike Kunze wrote:
Hello list,

1. Addressing:

2. Flow granularity:

Best wishes,

Ike Kunze
--
Ike Kunze, M.Sc.
Researcher

--
Coin mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf..org/mailman/listinfo/coin<https://www.irtf.org/mailman/listinfo/coin>
--
Coin mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/coin

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Mon, 4 Nov 2019 13:19:00 -0500

I had a quick look at the draft. I could have detailed comments later but
it is a topic that we had flagged from the beginning and I think raising it
is good.

So I would recommend a short presentation.

mjm

On November 4, 2019 at 1:05:24 PM, Ike Kunze (
[email-masked]) wrote:

Hi Marie-José,

thanks for the offer.
We actually wanted to have offline discussions first, but if you think that
discussing the transport issues
in a larger forum during the COIN meeting makes sense, Klaus could do a
short presentation as he will be in Singapore.
Have a look at the draft [1] first, before deciding if it makes sense to
add it to the agenda.

Best,
Ike

Am 04.11.2019 um 12:15 schrieb Marie-Jose Montpetit <[email-masked]>:

Ike:

mjm

Hi Ike,

(chair hat off)

I see two potential alternatives for in-net compute. One is “ICN-style”,
e.g. two examples below..

2, Pain and gain

Just my two cents,

Jeffrey

Hi Dirk,

Best,

Ike

Hi Ike,

·         the “middlebox” notion is IMO fundamentally wrong (as an
intercepting device that is transparent to other entities).. We may not
need to know where a function resides, but we should know its identity to
authorize access to our data. (I know that you are not suggesting it here.)

Dirk

Hello list,

1. Addressing:

2. Flow granularity:

Best wishes,

Ike Kunze
-- 
Ike Kunze, M.Sc.
Researcher

--
Coin mailing list
[email-masked]
https://www.irtf..org/mailman/listinfo/coin
<https://www.irtf.org/mailman/listinfo/coin>

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 5 Nov 2019 09:02:05 -0500

Thanks

mjm

On November 5, 2019 at 8:47:44 AM, Ike Kunze (
[email-masked]) wrote:

Hi Marie-José,

okay, great.
Then we will prepare a short presentation and you can add us to the agenda.

Best,
Ike

Am 04.11.2019 um 19:19 schrieb Marie-Jose Montpetit <[email-masked]>:

mjm

Hi Marie-José,

thanks for the offer.

Best,
Ike

Ike:

mjm

Hi Ike,

(chair hat off)

2, Pain and gain

Just my two cents,

Jeffrey

Hi Dirk,

Best,

Ike

Hi Ike,

Dirk

Hello list,

1. Addressing:

2. Flow granularity:

Best wishes,

Ike Kunze
-- 
Ike Kunze, M.Sc.
Researcher

---

From: Ike Kunze <[email-masked]>
Date: Tue, 5 Nov 2019 13:47:42 +0000

Hi Marie-José,

okay, great.

Best,
Ike

Am 04.11.2019 um 19:19 schrieb Marie-Jose Montpetit <[email-masked]<mailto:[email-masked]>>:

I had a quick look at the draft. I could have detailed comments later but it is a topic that we had flagged from the beginning and I think raising it is good.

mjm

On November 4, 2019 at 1:05:24 PM, Ike Kunze ([email-masked]<mailto:[email-masked]>) wrote:

Hi Marie-José,

thanks for the offer.

Best,
Ike

Ike:

mjm

2, Pain and gain

Just my two cents,
Jeffrey

Hi Dirk,

Best,
Ike

Hi Ike,

1. Addressing:

2. Flow granularity:

Best wishes,

Ike Kunze
--
Ike Kunze, M.Sc.
Researcher