Subject: Re: [Cfrg] Streaming AEAD

From: Peter Gutmann <[email-masked]>
Date: Tue, 18 Feb 2014 09:13:57 +0000

"Manger, James" <[email-masked]> writes:

>What is needed is a way to use an AEAD algorithm to secure a message in
>chunks. Streaming-AEAD would allow a recipient to know they have received an
>authentic prefix of a message. It would allow crypto APIs to support
>streaming modes while never returning unauthentic plaintext.

I think the means of doing this is going to be protocol-specific, since it
depends on the PDU format being used.  For things like SSL/TLS and SSH the
problem is already solved since data amounts are quantised to the protocol
data packet size, for store-and-forward protocols like PGP and S/MIME you'd
need to add a mechanism for it in a PGP/SMIME-specific manner, but then you'd
also need to figure out how to communicate it to the other side... and then
you're back to relying on ad-hoc mechanisms.

When I had to do this years ago I used something like:

len1 || data1 || h1 = HMAC( len1 || data1 ), 
len2 || data2 || h2 = HMAC( h1 || len2 || data2 ),
len3 || data3 || h3 = HMAC( h2 || len3 || data3 ),
...

A length of 0 indicated end-of-contents.

Peter.

---

From: David McGrew <[email-masked]>
Date: Tue, 18 Feb 2014 08:23:19 -0500

Hi Peter,

On 02/18/2014 04:13 AM, Peter Gutmann wrote:
> depends on the PDU format being used.

I think that's right.   It still makes sense to consider what should be 
going on above the AEAD layer, in a generic sense.

> For things like SSL/TLS and SSH the
> data packet size,

I hope that's right.   I am not sure that SSL/TLS implementers would be 
comfortable if they could never use the decryptUpdate function though.

> for store-and-forward protocols like PGP and S/MIME you'd

Interesting, is there a writeup?

thanks,

David

>
> Peter.
> _______________________________________________
> Cfrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/cfrg
>

---

From: "Manger, James" <[email-masked]>
Date: Wed, 19 Feb 2014 14:59:22 +1100

> > depends on the PDU format being used.

Not necessarily.
I was thinking we could define a new alg identifier such as
"AEAD_AES_128_OCB_TAGLEN96/10K", meaning OCB on 10 KB chunks.
Once the 10228th byte (10KB - 96 bits) of plaintext is streamed
to the encrypt function, it finishes the first AEAD operation and
inserts a 96-bit tag into the ciphertext. A decryption function
caches the ciphertext it is streamed until it reaches a 10KB
boundary then it returns 10228 bytes of plaintext.

An app sees a different alg id, but doesn't need to notice that
the ciphertext PDU is actually ct tag ct tag ct tag...
[It might need to know which nonces have been used internally though!]

> Interesting, is there a writeup?

So for packetizing plaintext for ~AEAD chunks:
* TLS puts a sequence number in the AAD;
* Peter (sort of) puts a tag for the previous prefix in the AAD;
* I suggested using sequential nonces and a start/middle/end flag;
* AERO hides the details in state;
* Pipelineable On-Line Encryption randomizes unauthentic plaintext.

My only concern with putting state (eg sequence number, byte count, previous hash) into AAD is that you need something at a higher layer to ensure that AEAD(X||Y,P) is always interpreted as "chunk mode with state=X, user-provided-AAD=Y", and never as "non-chunk mode with user-provided-AAD=X||Y". You probably cannot safely use 1 key in chunk and non-chunk modes.

--
James Manger

---

From: Mridul Nandi <[email-masked]>
Date: Wed, 19 Feb 2014 10:38:53 +0530

Dear ALL,

When we have limited buffer or we need to release the computation of
message and ciphertext in a regular interval, we can go for AEAD which
supports intermediate tag.  You may have a look on section 6 of our paper
http://eprint.iacr.org/2013/767.pdf
which talks about intermediate tag. Computation of intermediate tag in our
construction is very less.

Thanks and regards,
Mridul Nandi
 Assistant Professor
Indian Statistical Institute
Kolkata

On Wed, Feb 19, 2014 at 9:29 AM, Manger, James <
[email-masked]> wrote:

> My only concern with putting state (eg sequence number, byte count,
> previous hash) into AAD is that you need something at a higher layer to
> ensure that AEAD(X||Y,P) is always interpreted as "chunk mode with state=X,
> user-provided-AAD=Y", and never as "non-chunk mode with
> user-provided-AAD=X||Y". You probably cannot safely use 1 key in chunk and
> non-chunk modes.
>
> --
> James Manger

---

From: David McGrew <[email-masked]>
Date: Sun, 23 Feb 2014 09:15:47 -0500

Hi James,

> [It might need to know which nonces have been used internally though!]

I take your point.  If there was a streaming-AEAD algorithm (using an 
API similar to RFC 5116 but extended a bit), then it could be used in a 
protocol regardless of that protocol's message formats and sizes.

It would be somewhat awkward that the sender needs to know the right 
chunk size that the receiver can handle, so the chunk size is something 
that one would want to "negotiate", or at least have the receiver advise 
the sender on.

If a streaming-AEAD gets defined, it would be attractive to define a way 
in which any of the existing AEAD schemes could be used in chunk-mode.

> My only concern with putting state (eg sequence number, byte count, previous hash) into AAD is that you need something at a higher layer to ensure that AEAD(X||Y,P) is always interpreted as "chunk…

Agreed; chunk mode would need to be well defined, understood by both 
sender and receiver, and never share keys with non-chunk modes.

David

---

From: David McGrew <[email-masked]>
Date: Tue, 25 Feb 2014 10:52:18 -0500

Hi James,

>> [It might need to know which nonces have been used internally though!]

A thought occurred to me.   Currently, many systems use an 
init/update/final interface for AEAD decryption.  The streaming AEAD 
algorithm that you propose would not enable them to continue using this 
interface as they currently are.   The problem is that the decryptUpdate 
function is required to provide plaintext when it returns; at least this 
is the case in some libraries.  If decryptUpdate is not required to 
provide plaintext when it returns, then the implementation could just 
buffer the (pre-auth-verification) plaintext until the auth verification 
is done.

David

> API similar to RFC 5116 but extended a bit), then it could be used in 
> a protocol regardless of that protocol's message formats and sizes.
>
> chunk size that the receiver can handle, so the chunk size is 
> something that one would want to "negotiate", or at least have the 
> receiver advise the sender on.
>
> If a streaming-AEAD gets defined, it would be attractive to define a 
> way in which any of the existing AEAD schemes could be used in 
> chunk-mode.
>
>>
>>
>> previous hash) into AAD is that you need something at a higher layer 
>> to ensure that AEAD(X||Y,P) is always interpreted as "chunk mode with 
>> state=X, user-provided-AAD=Y", and never as "non-chunk mode with 
>> user-provided-AAD=X||Y". You probably cannot safely use 1 key in 
>> chunk and non-chunk modes.
>
> .

---

From: David Jacobson <[email-masked]>
Date: Tue, 25 Feb 2014 08:36:08 -0800

On 2/25/14 7:52 AM, David McGrew wrote:
> algorithm that you propose would not enable them to continue using 
> this interface as they currently are.   The problem is that the 
> decryptUpdate function is required to provide plaintext when it 
> returns; at least this is the case in some libraries.  If 
> decryptUpdate is not required to provide plaintext when it returns, 
> then the implementation could just buffer the (pre-auth-verification) 
> plaintext until the auth verification is done.
>
> David
[snip]

It is simpler than that.  The decryptUpdate just blocks until it has 
gotten to a verification tag.  If the length parameter to the 
decryptUpdate is less than the plaintext up to the auth tag, it delivers 
the requested amount of plaintext, and buffers the remainder, which can, 
of course, be immediately returned on the next call.

Now like many protocols with chunking, there is the possibility of 
deadlock if such protocols are used interactively. That is something to 
think about.

    --David Jacobson