JdbcAppender using DataSourceConnectionSource could not append log event in embedded WAS

[minseo300]

Description

I'm trying to configure JdbcAppender using DataSourceConnectionSource(jndi) in embedded Tomcat(I create application with Springboot)

I defined data source in application.yml, and using jndi-name in log4j2.xml.

[log4j2.xml]

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="error">
    <Appenders>
        <JDBC name="databaseAppender" tableName="logging_event">
            <DataSource jndiName="jndi/mysql" />
            <Column name="log_event_date" pattern="%d" />
            <Column name="logger_name" pattern="%c" />
            <Column name="log_level" pattern="%p"/>
            <Column name="thread_name" pattern="%t"/>
            <Column name="message" pattern="%message" isClob="true"/>
        </JDBC>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="databaseAppender"/>
        </Root>
    </Loggers>
</Configuration>

[jndi configuration class]

@Configuration
public class JndiResource {
    @Bean
    TomcatServletWebServerFactory tomcatFactory() {
        return new TomcatServletWebServerFactory() {
            @Override
            protected TomcatWebServer getTomcatWebServer(Tomcat tomcat) {
                tomcat.enableNaming();
                return super.getTomcatWebServer(tomcat);
            }

            @Override
            protected void postProcessContext(Context context) {
                context.getNamingResources().addResource(getResource());
            }
        };
    }

    public ContextResource getResource() {
        ContextResource resource = new ContextResource();
        resource.setName("jndi/mysql");
        resource.setType("javax.sql.DataSource");
        resource.setProperty("driverClassName", "com.mysql.cj.jdbc.Driver");
        resource.setProperty("url", "jdbc:mysql://localhost:3306/logging_test");
        resource.setProperty("username", "root");
        resource.setProperty("password", "1234");
        return resource;
    }
}

As I analyzed JdbcAppender class, ConnectionSource in JdbcAppender is required, so if ConnectionSource is null, the exception is thrown while configuring log4j2 configuration.

In Spring boot application, log4j2 configuration is configured before embedded Tomcat loading jndi resource, so JdbcAppender isn't created and unable to locate appender for logger config.

Is there any plan to support log4j2 JdbcAppender with DataSourceConnectionSource with embedded WAS?

Configuration

**Version: 2.17.1

**Operating system: any

**JDK: 1.8

Logs

2024-10-28 15:17:16,545 main ERROR No ConnectionSource provided: connectionSource
2024-10-28 15:17:16,546 main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.db.jdbc.JdbcAppender for element JDBC org.apache.logging.log4j.core.config.ConfigurationException: Arguments given for element JDBC are invalid: field 'connectionSource' has invalid value 'null'
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.injectFields(PluginBuilder.java:208)
	at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:121)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1002)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:942)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:934)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:552)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:241)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:288)
	at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:618)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:691)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:708)
	at org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.reinitialize(Log4J2LoggingSystem.java:207)
	at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:73)
	at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:60)
	at org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.initialize(Log4J2LoggingSystem.java:163)
	at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:312)
	at org.springframework.boot.context.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:281)
	at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:239)
	at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:216)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
	at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:82)
	at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:63)
	at java.util.ArrayList.forEach(ArrayList.java:1259)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:117)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:111)
	at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:62)
	at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:375)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:333)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1340)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329)
	at org.example.MainApplication.main(MainApplication.java:9)

2024-10-28 15:17:16,546 main ERROR Null object returned for JDBC in Appenders.
2024-10-28 15:17:16,547 main ERROR Unable to locate appender "databaseAppender" for logger config "root"

Reproduction

[An isolated test reproducing the test.
JUnit tests similar to the ones in the code base are extremely appreciated.]

[ppkarwasz]

Hi @minseo300,

As I analyzed JdbcAppender class, ConnectionSource in JdbcAppender is required, so if ConnectionSource is null, the exception is thrown while configuring log4j2 configuration.

In Spring boot application, log4j2 configuration is configured before embedded Tomcat loading jndi resource, so JdbcAppender isn't created and unable to locate appender for logger config.

Is there any plan to support log4j2 JdbcAppender with DataSourceConnectionSource with embedded WAS?

We discussed the usage of JDBC connection pools provided by Spring Boot in #2807.
The overall sentiment I get from that discussion is that:

• Logging should use a dedicated connection pool that is used only for logging.
• Spring Boot can be used to share secrets (in application.yml for example).

Therefore I would recommend you to use a different kind of connection source. The choice depends on the kind of connection pool you are using.

Apache Commons DBCP

If your application uses the Apache Commons DBCP connection pool, add log4j-jdbc-dbcp2 to your dependencies and use the PoolingDriver connection source:

<PoolingDriver poolName="logging"
               driverClassName="${spring:spring.datasource.driver-class-name}"
               connectionString="${spring:spring.datasource.url}"
               userName="${spring:spring.datasource.username}"
               password="${spring:spring.datasource.password}"/>

HikariCP

If your application uses the default HikariCP connection pool you can use the ConnectionFactory connection source and a custom factory method. For example:

public class LoggingDataSource {

    public static DataSource newInstance() {
        PropertiesUtil environment = PropertiesUtil.getProperties();
        HikariDataSource dataSource = new HikariDataSource();
        dataSource.setDriverClassName(environment.getStringProperty("spring.datasource.driver-class-name"));
        dataSource.setJdbcUrl(environment.getStringProperty("spring.datasource.url"));
        dataSource.setUsername(environment.getStringProperty("spring.datasource.username"));
        dataSource.setPassword(environment.getStringProperty("spring.datasource.password"));
        // Only one connection at a time is needed
        dataSource.setMinimumIdle(0);
        dataSource.setMaximumPoolSize(1);
        return dataSource;
    }
}

Then you can use a connection source like:

<ConnectionFactory class="eu.copernik.spring.boot.LoggingDataSource"
                   method="newInstance"/>

Warning

The above example is not complete, since you need to ensure that the DataSource created by newInstance will be closed, when the application closes or a reconfiguration event occurs.

[minseo300]

@ppkarwasz
Thanks for replying.

In #2807, it seems you guys conclude there are more cons than pros to provide DataSourceConnectionSource in embedded WAS.
And that conclusion is based on the concept that separating logging configuration and application configuration makes application architecture simplifier.

DataSourceConnectionSource is supported to bind DataSource(defined at external source) using full JNDI path.

In Spring boot application, we can define JNDI resource and using it with the JNDI path. And I think there are no differences between using JNDI in embedded WAS and external WAS.

If we want to use external resource, define it and just bind it to where we want to use the resource. That's all we have to do.
If we want to use connection pool from external resource(defined at external source) for logging, then define DataSource at external source and bind it with JDNI path in log4j2.xml.

So there's a point. There are no differences the way to use between using JNDI in embedded WAS and external WAS.
I recognize that initializing JNDI resource time is different in embedded WAS and external WAS.

The reason why I can't use DataSourceConnectionSource in Spring boot application is when DataSourceConnectionSource try to look up JNDI resource, the JNDI resource is not initialized yet, so ConnectionSource is null but JdbcAppender requires that ConnectionSource is not null when initialize log4j2 logger context.

In my opinion,

If JdbcAppender allows that ConnectionSource to be null so log4j2 logger context initialize by log4j2.xml successfully(it means JdbcAppender doesn't care JNDI resource is initialized or not) and look up DataSource when JdbcAppender try to append log event to database, then we can use JNDI in Spring boot application.

But in this case, if user(who use log4j2) writes wrong JNDI path, then look up will be failed when JdbcAppender try to append log event to database. It means user will recognize they wrote wrong JNDI path after log4j2 configuration is initialized successfully. It could be Spring boot application is started successfully.

If you guys think that user recognize their mistake in log4j2.xml(or everything about configuring log4j2 configuration) after application is started successfully is a big problem, I can accept that opinion.

Just this issue is from there's no guide in log4j2 JdbcAppender documentation about using DataSourceConnectionSource in Spring boot application.