FIPS mode crypto:strong_rand_bytes(1) fails

[reuvenstr]

Describe the bug
Using ubuntu 22 with enabled FIPS mode I tried to enforce fips in rabbitmq using env variable(RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-crypto fips_mode true"), and it failed to start with the error:

initial call: credentials_obfuscation_svc:init/1
pid: <0.129.0>
registered_name: []
exception error: undefined function crypto:strong_rand_bytes/1
in function credentials_obfuscation_svc:check/3 (src/credentials_obfuscation_svc.erl, line 197)
in call from credentials_obfuscation_svc:init_state/0 (src/credentials_obfuscation_svc.erl, line 169)
in call from gen_server:init_it/2 (gen_server.erl, line 980)
in call from gen_server:init_it/6 (gen_server.erl, line 935)
ancestors: [credentials_obfuscation_sup,<0.127.0>]
message_queue_len: 0

So I thought that maybe rabbitmq requires erlang that compiled with fips in order to enforce it, but its failed with different error, that can be reproduced in simpler way
To Reproduce
Compile erlang/otp of tag OTP-26.2.1 with --enable-fips flag and run simple commands:
Enter erlang shell, load crypto app, and run the rand command:

erl -crypto fips_mode true
application:load(crypto).
crypto:strong_rand_bytes(128).

I get

** exception error: low_entropy
in function crypto:strong_rand_bytes/1 (crypto.erl, line 1155)

Expected behavior
Possibility to run rabbitmq with enforced fips

Affected versions
OTP-26.2.1+ , didn't see any changes in the changelog related to this issue

[reuvenstr]

With debugger mode i got this error

5> crypto:strong_rand_bytes(1).
** exception error: {nif_not_loaded,module,crypto,line,1158}
in function erlang:nif_error/1
called as erlang:nif_error({nif_not_loaded,module,crypto,line,1158})
in call from crypto:nif_stub_error/1 (crypto.erl, line 493)
in call from crypto:strong_rand_bytes/1 (crypto.erl, line 1154)

[sverker]

What do crypto:info() and crypto:info_fips() return?

What exactly do you mean with "debugger mode"?

[reuvenstr]

I ran this one:

ii(crypto).

but now I see that it's breaks everything, sorry for misleading

Upon your request

1> application:load(crypto).
ok
2> crypto:info().
#{otp_crypto_version => "5.4",compile_type => normal,
link_type => dynamic,
cryptolib_version_compiled => "OpenSSL 3.0.2 15 Mar 2022",
cryptolib_version_linked => "OpenSSL 3.0.2 15 Mar 2022",
fips_provider_available => true}
3> crypto:info_fips().
enabled

[sverker]

low_entropy would mean that the underlying OpenSSL and in the end the OS does not provide enough real unpredictable randomness (entropy).
To be sure, it would be interesting to know what RAND_bytes() returns in lib/crypto/c_src/rand.c. It looks like it treats all errors as low_entropy. Here's a diff to show that:

diff --git a/lib/crypto/c_src/rand.c b/lib/crypto/c_src/rand.c
index cc34f0afc7..931b7da217 100644
--- a/lib/crypto/c_src/rand.c
+++ b/lib/crypto/c_src/rand.c
@@ -36,9 +36,19 @@ ERL_NIF_TERM strong_rand_bytes_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM
 
     if ((data = enif_make_new_binary(env, bytes, &ret)) == NULL)
         goto err;
-    if (RAND_bytes(data, (int)bytes) != 1)
-        goto err;
-
+    {
+        int r = RAND_bytes(data, (int)bytes);
+        if (r != 1) {
+            char errstr[256];
+            long err;
+            enif_fprintf(stderr, "ERROR RAND_bytes return: %d\n", r);
+            while ((err = ERR_get_error())) {
+                ERR_error_string(err, errstr);
+                enif_fprintf(stderr, "ERROR RAND_bytes err=%ld, %s\n", err, errstr);
+            }
+            goto err;
+        }
+    }
     ERL_VALGRIND_MAKE_MEM_DEFINED(data, bytes);
     return ret;

[reuvenstr]

Ok, now it more informative 😃 Not understandable for me, maybe you know what does it mean?

ERROR RAND_bytes return: 0
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=134217857, error:08000081:elliptic curve routines::unknown group
ERROR RAND_bytes err=50856204, error:0308010C:digital envelope routines::unsupported
ERROR RAND_bytes err=301990032, error:12000090:random number generator::unable to fetch drbg
** exception error: low_entropy
in function crypto:strong_rand_bytes/1 (crypto.erl, line 1155)

[sverker]

The last one "random number generator::unable to fetch drbg" is the interesting one. It fails to fetch the random number generator provider (EVP_RAND_fetch). Seems something is wrong with the OpenSSL/OS configuration.

Does it work if you don't enable fips_mode?

[reuvenstr]

Yeap, without enabling FIPS it's ok

[sverker]

I cannot reproduce this. I tried on ubuntu 22 with OTP 26.2.1 and OpenSSL 3.0.2 built with fips support.
It's the same OTP code in crypto that runs with and without fips; strong_rand_bytes_nif() in lib/crypto/c_src/rand.c calling RAND_bytes() in OpenSSL.

Is it just crypto:strong_rand_bytes or do other functions in crypto also fail in fips_mode?

[reuvenstr]

I'm not sure if there is something else that fails, give me names and I can check
You built openssl by yourself, and also otp was pointing to it?
I took ubuntu 22.04 enabled pro and activated fips mode from it, so only otp was built by me

otp_build configure --enable-fips --with-ssl
make
make install

openssl list -providers
Providers:
  base
    name: OpenSSL Base Provider
    version: 3.0.2
    status: active
  fips
    name: Ubuntu 22.04 OpenSSL Cryptographic Module
    version: 3.0.5-0ubuntu0.1+Fips2.1
    status: active

[sverker]

Yes, I built OpenSSL myself from source. What did you do to activate fips on installed OpenSSL on ubuntu 22?

[reuvenstr]

I ran this commands

apt install ubuntu-advantage-tools
pro attach YOUR-PRO-LICENSE
pro enable fips # or pro enable fips-updates
reboot

cat /proc/sys/crypto/fips_enabled # returns 1

[reuvenstr]

For debug purposes there is small C code that is passes too

#include <stdio.h>
#include <openssl/rand.h>
#include <openssl/crypto.h>
int main() {
    unsigned char buffer[16];  // Buffer to hold 16 random bytes
    int ret = 0;
    long err;
    OPENSSL_init();
    ret = EVP_default_properties_enable_fips(NULL, 1);
    if(ret != 1)
    {
	fprintf(stderr, "Error enabling FIPS");
        return 1;
    }
    ret = EVP_default_properties_is_fips_enabled(NULL);
    printf("Fips status is %s\n", ret ? "true" : "false");
    if (RAND_bytes(buffer, 1) != 1) {
        fprintf(stderr, "Error generating random bytes\n");
        return 1;
    }

    // Print the random bytes in hexadecimal format
    printf("Generated random bytes: ");
    for (int i = 0; i < sizeof(buffer); i++) {
        printf("%02x", buffer[i]);
    }
    printf("\n");

    return 0;
}

[sverker]

Let's try to better mimic what OTP crypto does by adding
#include <openssl/provider.h>

and this directly after OPENSSL_init();

 if (!OSSL_PROVIDER_load(NULL, "fips")) {
        printf("Failed to load fips provider\n");
 }
 if (!OSSL_PROVIDER_load(NULL, "default")) {
      fprintf(stderr, "Failed to load default provider\n");
      return 1;
 }
 if (!OSSL_PROVIDER_load(NULL, "base")) {
      fprintf(stderr, "Failed to load base provider\n");
      return 1;
 }
 if (!OSSL_PROVIDER_load(NULL, "legacy")) {
        printf("Failed to load legacy provider\n");
 }

and see what happens then...

[reuvenstr]

That's what i got

#include <stdio.h>
#include <openssl/rand.h>
#include <openssl/crypto.h>
#include <openssl/provider.h>
int main() {
    unsigned char buffer[16];  // Buffer to hold 16 random bytes
    int ret = 0;
    long err;
    OPENSSL_init();
     if (!OSSL_PROVIDER_load(NULL, "fips")) {
        printf("Failed to load fips provider\n");
 }
 if (!OSSL_PROVIDER_load(NULL, "default")) {
      fprintf(stderr, "Failed to load default provider\n");
      return 1;
 }
 if (!OSSL_PROVIDER_load(NULL, "base")) {
      fprintf(stderr, "Failed to load base provider\n");
      return 1;
 }
 if (!OSSL_PROVIDER_load(NULL, "legacy")) {
        printf("Failed to load legacy provider\n");
 }
    ret = EVP_default_properties_enable_fips(NULL, 1);
    if(ret != 1)
    {
        fprintf(stderr, "Error enabling FIPS");
        return 1;
    }
    ret = EVP_default_properties_is_fips_enabled(NULL);
    printf("Fips status is %s\n", ret ? "true" : "false");
    // Generate random bytes
    if (RAND_bytes(buffer, 1) != 1) {
        fprintf(stderr, "Error generating random bytes\n");
        return 1;
    }

    // Print the random bytes in hexadecimal format
    printf("Generated random bytes: ");
    for (int i = 0; i < sizeof(buffer); i++) {
        printf("%02x", buffer[i]);
    }
    printf("\n");

    return 0;
}

And this is the output

Fips status is true
Generated random bytes: 62000000000000000000000000000000

[sverker]

A nice random hex byte 62, how disappointing :-)

What if you try some other crypto function, for example:

> erl -crypto fips_mode true
1> application:load(crypto).
2> crypto:hash(sha, <<"hello">>).

Regardless, I think I have to get myself an Ubuntu pro license and try to reproduce myself...

[reuvenstr]

A nice random hex byte 62, how disappointing :-)

😏

What if you try some other crypto function, for example:

> erl -crypto fips_mode true
1> application:load(crypto).
2> crypto:hash(sha, <<"hello">>).

It also works, returns some object

3> crypto:hash(sha, <<"hello">>).
<<170,244,198,29,220,197,232,162,218,190,222,15,59,72,44,
  217,174,169,67,77>>

Regardless, I think I have to get myself an Ubuntu pro license and try to reproduce myself...

At least they are free for personal use 😃

[reuvenstr]

Hey @sverker

Do you have any update about it?

[sverker]

Here's a preliminary fix you can try:

diff --git a/lib/crypto/c_src/algorithms.c b/lib/crypto/c_src/algorithms.c
index c7e1fdea04..7305aa8bb8 100644
--- a/lib/crypto/c_src/algorithms.c
+++ b/lib/crypto/c_src/algorithms.c
@@ -56,7 +56,7 @@ void init_algorithms_types(ErlNifEnv* env)
     init_hash_types(env);
 #endif
     init_pubkey_types(env);
-    init_curve_types(env);
+    //init_curve_types(env);
     init_rsa_opts_types(env);
     /* ciphers and macs are initiated statically */
 }

[reuvenstr]

Here's a preliminary fix you can try:

diff --git a/lib/crypto/c_src/algorithms.c b/lib/crypto/c_src/algorithms.c
index c7e1fdea04..7305aa8bb8 100644
--- a/lib/crypto/c_src/algorithms.c
+++ b/lib/crypto/c_src/algorithms.c
@@ -56,7 +56,7 @@ void init_algorithms_types(ErlNifEnv* env)
     init_hash_types(env);
 #endif
     init_pubkey_types(env);
-    init_curve_types(env);
+    //init_curve_types(env);
     init_rsa_opts_types(env);
     /* ciphers and macs are initiated statically */
 }

Thanks

It looks like it's working now, can you please explain what is happening?

[sverker]

Something goes wrong when switching FIPS mode on/off (with EVP_default_properties_enable_fips). The initialization code did that to figure out which elliptic curves are supported with and without FIPS. However, that's not needed as crypto:supports(curves) will run the same checks anyway the first time it's called.

This seems to me like a bug in OpenSSL.

[sverker]

Fix in OTP-26.2.5.4.