mod_ssl uses the function ssl_log_ssl_error (ssl_engine_log.c:86) to log library errors. This function does call ap_log_error. 
My proposal would be to write a method ssl_log_ssl_cerror, which calls ap_log_cerror. This function would be called whenever a conn_rec is available instead of ssl_log_ssl_error. The advantage is that like this, we can see a client ip address in the logs, while with ap_log_error we can't. Its ok to use ap_log_error for configuration and initialization stuff, but for all connection related errors we should go for ap_log_cerror.

for example this two log lines are from one failed connection attempt, but the second line doesn't show an ip address:

   Thu May 28 11:55:04 2020 [client({c}a)=fc00::cafe:39298] [client(a)=fc00::cafe:39298] [msg="AH01996: SSL handshake failed: HTTP spoken on HTTPS port; trying to send HTML error page"] 


   Thu May 28 11:55:04 2020 [client({c}a)=-] [client(a)=-] [msg="SSL Library Error: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request -- speaking HTTP to HTTPS port!?"]

I think this would bring a lot of advantages (the client ip is otfen used to correlate requests, for example by an external program that parses apache logs) for quite small effort..