`EVP_DigestSignInit: unsupported (OpenSSL::HMACError)` and `Digest initialization failed: initialization error (OpenSSL::Digest::DigestError)`

[yahonda]

This issue is reported initially at rails/rails#48483 and here is Ruby script to reproduce this error.

The original issue has two exceptions. EVP_DigestSignInit: unsupported (OpenSSL::HMACError) and Digest initialization failed: initialization error (OpenSSL::Digest::DigestError)

EVP_DigestSignInit: unsupported (OpenSSL::HMACError)

Steps to reproduce

• Run these scripts

require "openssl"
OpenSSL::VERSION

digest = "MD4"
secret = "saltsaltsaltsaltsaltsaltsaltsalt"
data = "cDcvTnJ3TDRqbmFpQWN4OWVJWjRKU1E4YnNsWWpuZ1NUbFF5OTZRQW90QT0tLTNGZkVOYVJhY3l5dkZOa3RjMklCeFE9PQ=="
OpenSSL::HMAC.hexdigest(digest, secret, data)

Expected behavior

It should run as Ruby 3.0.5 does.

$ ruby -v
ruby 3.0.5p211 (2022-11-24 revision ba5cf0f7c5) [x86_64-linux]
$ irb
irb(main):001:0>
irb(main):002:0> require "openssl"
=> true
irb(main):003:0> OpenSSL::VERSION
=> "2.2.2"
irb(main):004:0>
irb(main):005:0> digest = "MD4"
=> "MD4"
irb(main):006:0> secret = "saltsaltsaltsaltsaltsaltsaltsalt"
=> "saltsaltsaltsaltsaltsaltsaltsalt"
irb(main):007:0> data = "cDcvTnJ3TDRqbmFpQWN4OWVJWjRKU1E4YnNsWWpuZ1NUbFF5OTZRQW90QT0tLTNGZkVOYVJhY3l5dkZOa3RjMklCeFE9PQ=="
irb(main):008:0> OpenSSL::HMAC.hexdigest(digest, secret, data)
=> "464c3e47de854fe82cc7514f1fc16a8f"
irb(main):009:0>

Actual behavior

It gets "EVP_DigestSignInit: unsupported (OpenSSL::HMACError)" with Ruby 3.2.2.

$ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
$ irb
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> digest = "MD4"
=> "MD4"
irb(main):003:0> secret = "saltsaltsaltsaltsaltsaltsaltsalt"
=> "saltsaltsaltsaltsaltsaltsaltsalt"
irb(main):004:0> data = "cDcvTnJ3TDRqbmFpQWN4OWVJWjRKU1E4YnNsWWpuZ1NUbFF5OTZRQW90QT0tLTNGZkVOYVJhY3l5dkZOa3RjMklCeFE9PQ=="
irb(main):005:0> OpenSSL::HMAC.hexdigest(digest, secret, data)
/home/yahonda/.rbenv/versions/3.2.2/lib/ruby/3.2.0/openssl/hmac.rb:55:in `initialize': EVP_DigestSignInit: unsupported (OpenSSL::HMACError)
	from /home/yahonda/.rbenv/versions/3.2.2/lib/ruby/3.2.0/openssl/hmac.rb:55:in `new'
	from /home/yahonda/.rbenv/versions/3.2.2/lib/ruby/3.2.0/openssl/hmac.rb:55:in `hexdigest'
	from (irb):5:in `<main>'
	from /home/yahonda/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.7.0/exe/irb:9:in `<top (required)>'
	from /home/yahonda/.rbenv/versions/3.2.2/bin/irb:25:in `load'
	from /home/yahonda/.rbenv/versions/3.2.2/bin/irb:25:in `<main>'
irb(main):006:0>

Digest initialization failed: initialization error (OpenSSL::Digest::DigestError)

Steps to reproduce

• Run these scripts

require "openssl"
OpenSSL::VERSION
digest = "MD4"
OpenSSL::Digest.new(digest)

Expected behavior

It should run as Ruby 3.0.5 does.

$ ruby -v
ruby 3.0.5p211 (2022-11-24 revision ba5cf0f7c5) [x86_64-linux]
$ irb
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> OpenSSL::VERSION
=> "2.2.2"
irb(main):003:0> digest = "MD4"
irb(main):004:0> OpenSSL::Digest.new(digest)
=> #<OpenSSL::Digest: 31d6cfe0d16ae931b73c59d7e0c089c0>
irb(main):005:0>

Actual behavior

$ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
$ irb
irb(main):001:0> require "openssl"
=> true
irb(main):002:0> digest = "MD4"
irb(main):003:0> OpenSSL::Digest.new(digest)
(irb):3:in `initialize': Digest initialization failed: initialization error (OpenSSL::Digest::DigestError)
	from (irb):3:in `new'
	from (irb):3:in `<main>'
	from /home/yahonda/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.7.0/exe/irb:9:in `<top (required)>'
	from /home/yahonda/.rbenv/versions/3.2.2/bin/irb:25:in `load'
	from /home/yahonda/.rbenv/versions/3.2.2/bin/irb:25:in `<main>'
irb(main):004:0>

[yahonda]

These errors looks started since Ruby 3.1 and 3.2 base image changed from "bullseye" to "bookworm" and these errors have been workarounds by using Debian "bullseye" based Docker image at Rails CI. rails/buildkite-config#47

[yahonda]

Looks like MD4 is regognized as legacy in OpenSSL 3.

https://wiki.openssl.org/index.php/OpenSSL_3.0

The legacy provider. This is a collection of legacy algorithms that are either no longer in common use or strongly discouraged from use. However some applications may need to use these algorithms for backwards compatibility reasons. This provider is NOT loaded by default. This may mean that some applications upgrading from earlier versions of OpenSSL may find that some algorithms are no longer available unless they load the legacy provider explicitly. Algorithms in the legacy provider include MD2, MD4, MDC2, RMD160, CAST5, BF (Blowfish), IDEA, SEED, RC2, RC4, RC5 and DES (but not 3DES).

[rhenium]

Duplicate of #500.

Yes - the implementation of MD4 belongs to the legacy provider in OpenSSL >= 3.0 which is not enabled by default. #635 will add a method to load OpenSSL providers, or you can currently do it through an OpenSSL configuration file.

Since MD4 is obsolete in every sense and is apparently only used in the test suite (https://github.com/rails/rails/blob/87ab32f72a494ea18a4e2335becaebb66f5ce673/activesupport/test/rotation_coordinator_tests.rb#L100), I'd suggest simply replacing it with a modern digest algorithm.

[yahonda]

Thanks for the update. rails/rails#48493 will replace MD4 with SHA256 in the test case.