Subject: [netconf] base64encodedvalue== is an invalid base64 value

From: Kent Watsen <[email-masked]>
Date: Fri, 29 Jan 2021 18:19:54 +0000

I’m unsure if this issue warrants a fix, but be aware that using "base64encodedvalue==“ in code doesn't roundtrip correctly:

$ echo "base64encodedvalue==" | base64 -d | base64
base64encodedvaluQ==

Specifically, there is no binary input that results in “ue==“.

$ echo "ue==" | base64 -d | base64
uQ==

The reason is a more detailed than this message warrants, but effectively, the second sextet is quantized and, worse, none of three different implementations I tested against raise a warning, much less an error...which is reasonable, IMO.

Options:

1) stick with "base64encodedvalue==“ for the examples.
2) use a different value that is a valid base64 string.

Thoughts?

With regards to option #1:
  - still seems okay, as the value never occurs in practice

With regards to option #2:

  - if it is desired to end with “==“:
      - the last character must be one of [A, Q, g, w]
      - options: b64tag==, BASE64DATA==

  - if it is desired to end with “=“:
      - the last character must be one of [A, E, I, M, Q, U, Y, c, g, k, o, s, w, 0, 4, 8]
      - options: BASE64VALUE=

  - if it is desisired to end with no padding:
      - any alphanumeric string with length%4==0 works
      - options: base64encodedval, anybase64val, b64value, base64string

K.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 29 Jan 2021 20:11:42 +0100

Who cares whether this value roundtrips in an examples??

/js

> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Kent Watsen <[email-masked]>
Date: Fri, 29 Jan 2021 20:05:06 +0000

> Who cares whether this value roundtrips in examples??

That is why I asked  ;)

It may matter to implementers of a client or a server writing unit tests that don’t require valid base64 values, other than that they’ll roundtrip correctly.  If they assume this value will work, they’re in for a surprise.  I don’t care anymore, as I’ve adjusted all my test cases already.

K.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 29 Jan 2021 23:15:17 +0100

> Who cares whether this value roundtrips in an examples??

I’d expect at least some decent implementations will check for RFC 4648’s mandate "These pad bits MUST be set to
   zero by conforming encoders”, just like some check for the right number of equals signs.
Examples that fail in decent implementations are suboptimal.
(They might even be used in arguments to claim that this decency needs to be removed.)

Grüße, Carsten

---

From: Kent Watsen <[email-masked]>
Date: Fri, 29 Jan 2021 23:26:45 +0000

> (They might even be used in arguments to claim that this decency needs to be removed.)

`base64` doesn’t produce a warning:

`openssl` doesn’t produce a warning:

    $ echo "base64encodedvalue==" | openssl enc -d -base64 -A | openssl enc -base64 -A
    base64encodedvaluQ==

Python doesn’t produce a warning:

    $ printf "import binascii\nprint(binascii.b2a_base64(binascii.a2b_base64('base64encodedvalue=='), newline=False).decode('ascii'))" |  python -
    base64encodedvaluQ==

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 30 Jan 2021 02:47:59 +0100

>    $ printf "import binascii\nprint(binascii.b2a_base64(binascii.a2b_base64('base64encodedvalue=='), newline=False).decode('ascii'))" |  python -

Thanks for the data points; adding 1½ more:

$ ruby -rbase64 -e 'puts Base64.encode64(Base64.decode64("base64encodedvalue=="))' 
base64encodedvaluQ==
$ ruby -rbase64 -e 'puts Base64.strict_encode64(Base64.strict_decode64("base64encodedvalue=="))'
<internal:pack>:281:in `unpack1': invalid base64 (ArgumentError)
	from /opt/homebrew/Cellar/ruby/3.0.0_1/lib/ruby/3.0.0/base64.rb:74:in `strict_decode64'
	from -e:1:in `<main>'

Checking conformance is a MAY(*) for a decoder, so I’m not surprised that many don’t do that.
(Or check only on specific request, as in Ruby.)

I would strongly expect any decoder that is used in a security context to actually check, so I *am* surprised by the openssl example above.
(Well, not that much surprised; it is, well, openssl.)

I wouldn’t have reacted to this thread if I hadn’t years ago, when writing a random test case generator, been surprised by a base64 implementation that did check.
I don’t recollect the details of that, though…

Grüße, Carsten

(*) https://tools.ietf.org/html/rfc4648#section-3.5 <https://tools.ietf.org/html/rfc4648#section-3.5>
For those not familiar with RFC 2119 interoperability keywords:
MAY means that an implementation is free to do so.
Which raises the interoperability requirement that any peer implementation MUST be able to interoperate with an implementation that does.
This would be unnecessary to state in this case because that MUST is in the paragraph before.
So the intent here apparently was stressing the optionality of checking in a decoder.