Subject: [TLS] Regarding the length of encrypted premaster secret.

From: nilesh <[email-masked]>
Date: Tue, 11 Oct 2011 16:47:07 +0530

Hi,

I have setup an apache server with TLS/SSL support. And I could capture 
some https packets.
Also I have written a dummy code to decrypt using AES algorithm.

However, in the current trace file, the size of encrypted premaster sent 
from client is 130bytes.
I tried first getting a premaster from the encrypted premaster and it 
failed (my program is correct as
I verified it on different trace, also the server private key is correct 
as wireshark could decrypt using this key).
If I remove initial 2bytes of encrypted premaster secret (make it 
128bytes) - then I am able to decrypt it without failure.
But the master_secret and key block generated are not correct then (the 
application data get decrypted to junk -
again, I verified my code to generate master_secret and key block with 
different trace).

<snip>
rsa = RSA_new();
fp = fopen("server.key", "rb"); // error handling is done.
PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
RSA_check_key(rsa);
// Below API fails with '-1' rc - when 130bytes encrypted_premaster is used.
RSA_private_decrypt(pre_master_len, encrypted_premaster, pre_master, 
rsa, RSA_PKCS1_PADDING);
</snip>

Could someone please advise if I missed anything? Should the encrypted 
premaster be always 128bytes?
I have attached the trace file and key file with the email.

-- 
Thanks,
Nilesh

---

From: Juho Vähä-Herttua <[email-masked]>
Date: Tue, 11 Oct 2011 16:23:12 +0300

On 11.10.2011, at 14.17, nilesh <[email-masked]> wrote:
> If I remove initial 2bytes of encrypted premaster secret (make it 128bytes) - then I am able to decrypt it without failure.

In TLS 1.0 first two bytes is the length of encrypted premaster secret in bytes, in SSL 3.0 they are not used.

> But the master_secret and key block generated are not correct then (the application data get decrypted to junk -
> again, I verified my code to generate master_secret and key block with different trace).

I would recommend verifying again. If the encrypted premaster decrypts to 48 bytes of data it is most likely correct and your problem is elsewhere. So check that first.

> Could someone please advise if I missed anything? Should the encrypted premaster be always 128bytes?

The length depends on RSA key size, with 1024-bit keys it is 128 bytes.

I'm not looking at the trace file since I'm writing on my phone. However Wireshark has a very good SSL debug log feature that logs the premaster secret and master secret and other useful information, use it for debugging.

I don't think this list is the right place to debug own implementation though. Test your program against other widely used implementations, if it fails it is probably broken.

Juho