When a TLS-enabled listener fails to bind to a port, the entire TLS config (potentially including sensitive values) is logged together with the exception

[vlcht]

Describe the bug

RabbitMQ logs sensitive data as the password of certificate private key in rabbit@'machineName'.log:

Text from log file:

2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> Failed to start Ranch listener {acceptor,{0,0,0,0},5671} in ranch_ssl:listen(#{num_acceptors => 10,handshake_timeout => 5000,socket_opts => [{ip,{0,0,0,0}},{port,5671},inet,{backlog,128},{nodelay,true},{linger,{true,0}},{exit_on_close,false},{versions,['tlsv1.3']},{hibernate_after,6000},{keyfile,"c:\\certificates\\private.key"},{certfile,"c:\\certificates\\public.crt"},{cacertfile,"c:\\certificates\\ca.crt"},{fail_if_no_peer_cert,true},{verify,verify_peer},{password,'...'}],connection_type => supervisor,num_conns_sups => 1,max_connections => infinity}) for reason eaddrinuse (address already in use)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0>
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> crasher:
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> initial call: supervisor:ranch_acceptors_sup/1
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> pid: <0.624.0>
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> registered_name: []
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> exception exit: {listen_error,{acceptor,{0,0,0,0},5671},eaddrinuse}
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in function ranch_acceptors_sup:listen_error/5 (src/ranch_acceptors_sup.erl:94)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in call from ranch_acceptors_sup:start_listen_sockets/5 (src/ranch_acceptors_sup.erl:54)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in call from ranch_acceptors_sup:init/1 (src/ranch_acceptors_sup.erl:34)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in call from supervisor:init/1 (supervisor.erl:912)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in call from gen_server:init_it/2 (gen_server.erl:2276)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> in call from gen_server:init_it/6 (gen_server.erl:2236)
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> ancestors: [<0.621.0>,<0.619.0>,<0.618.0>,rabbit_sup,<0.211.0>]
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> message_queue_len: 0
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> messages: []
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> links: [<0.621.0>]
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> dictionary: []
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> trap_exit: true
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> status: running
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> heap_size: 6772
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> stack_size: 29
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> reductions: 22026
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0> neighbours:
2025-05-26 16:28:42.027000+03:00 [error] <0.624.0>
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> supervisor: {<0.621.0>,ranch_listener_sup}
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> errorContext: start_error
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> reason: {listen_error,{acceptor,{0,0,0,0},5671},eaddrinuse}
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> offender: [{pid,undefined},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {id,ranch_acceptors_sup},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {mfargs,
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {ranch_acceptors_sup,start_link,
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> [{acceptor,{0,0,0,0},5671},ranch_ssl,logger]}},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {restart_type,permanent},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {significant,false},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {shutdown,infinity},
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0> {child_type,supervisor}]
2025-05-26 16:28:42.043000+03:00 [error] <0.621.0>
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> supervisor: {<0.619.0>,ranch_embedded_sup}
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> errorContext: start_error
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> reason: {shutdown,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {failed_to_start_child,ranch_acceptors_sup,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {listen_error,{acceptor,{0,0,0,0},5671},eaddrinuse}}}
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> offender: [{pid,undefined},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {id,{ranch_listener_sup,{acceptor,{0,0,0,0},5671}}},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {mfargs,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {ranch_listener_sup,start_link,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> [{acceptor,{0,0,0,0},5671},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> ranch_ssl,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> #{num_acceptors => 10,handshake_timeout => 5000,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> socket_opts =>
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> [{ip,{0,0,0,0}},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {port,5671},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> inet,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {backlog,128},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {nodelay,true},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {linger,{true,0}},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {exit_on_close,false},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {versions,['tlsv1.3']},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {hibernate_after,6000},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {keyfile,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> "c:\\certificates\\private.key"},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {certfile,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> "c:\\certificates\\public.crt"},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {cacertfile,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> "c:\\certificates\\ca.crt"},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {fail_if_no_peer_cert,true},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {verify,verify_peer},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {password,<<"123">>}],
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> connection_type => supervisor,num_conns_sups => 1,
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> max_connections => infinity},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> rabbit_connection_sup,[]]}},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {restart_type,permanent},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {significant,false},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {shutdown,infinity},
2025-05-26 16:28:42.043000+03:00 [error] <0.619.0> {child_type,supervisor}]

Reproduction steps

1. RabbitMQ port is already used by another program.
2. Start (restart) RabbitMQ.
   ...

Expected behavior

The password of certificate private key file not logged or logged in a masked form.

Additional context

No response

[michaelklishin]

The answer is Configuration Value Encryption.

The docs explain how to use such values with advanced.config.

rabbitmq.conf

These days they are even supported in rabbitmq.conf as

key = encrypted:Value

where the value will be parsed as {encrypted, Value}. The Value in this example must be parsed to an Erlang binary, which will be true for some keys but not others.

According to the rabbitmq.conf schema in the core, this should be the case for ssl_options.password:

{mapping, "ssl_options.password", "rabbit.ssl_options.password",
    [{datatype, [tagged_binary, binary]}]}.

In other words, compute the encrypted value as explained in the docs and then

ssl_options.password = encrypted:a501a2c7958cb22cf1c665e7de2b2b3c032ed615

(I am using an example value)

[lhoguin]

The first log message is from Ranch and the value is filtered. The value is not filtered in the supervisor report, over which Ranch has no control.

[ansd]

@vlcht why are you using Erlang/OTP 28.0?
We don't officially support 28.0 yet: https://www.rabbitmq.com/docs/which-erlang#compatibility-matrix

[michaelklishin]

@vlcht a fix might appear when someone contributes it. You can be that someone. This is open source software after all.

[vlcht]

@vlcht why are you using Erlang/OTP 28.0? We don't officially support 28.0 yet: https://www.rabbitmq.com/docs/which-erlang#compatibility-matrix

@ansd thanks, i'm using 27.3, tried with 28

[vlcht]

Similar situation if RabbitMQ configured incorrectly (RabbitMQ 4.0.7, Erlang 27.3):


Is it the same bug?

[michaelklishin]

The runtime logs all exceptions with a lot of detail by design. Our team has gone some lengths before to make sure that the sensitive connection, session, channel values are stored internally as encrypted, and decrypted on access.

However, this is a very different case.

We have a few options here:

1. See if catching the exception would be enough in this specific context (rabbit_networking:start_listener0/6?)
2. Try filtering out these messages in the supervisor in question. Every time I have tried it before, it turned out to be a pain
3. Try to integrate credentials_obfuscation into the relevant code paths. I will be complex and likely never accepted into Ranch (and I'd agree with that decision)

@vlcht given that such events are not particularly common — we do not recommend running RabbitMQ next to other data services that could use the same port, or run multiple nodes on the same host in production — it's hard to justify spending time on this for our small team.

So we need someone to volunteer to look into this.

[michaelklishin]

@lhoguin has found out a much easier solution: the password now can be passed in as a function, so that's what we need to do in the RabbitMQ code.

We need, however, to verify that Erlang 26 supports this option, or make it specific to Erlang 27+.

[michaelklishin]

Erlang/OTP 26 has ssl version 11.1, 27 has 11.2 and 28 has 11.3.

[michaelklishin]

And the password option can be passed in as fun(() -> iodata()) starting with erlang/otp#5843.