Asynchronous logging with log4j

posted 25-Nov-2009 | 20 comments | apache, java, java logging, log4j, logging, memory, Performance

In case you are not doing it already, using asynchronous logging is generally a good idea. You don’t want your application to slow down if the server IO is a little behind flushing all that logging to the filesystem. By making it asynchronous your application can continue running without having to wait for the log lines to be written to their final destination.

My personal choice for Java logging is log4j, there are a lot of different frameworks (including Suns own logging API), but log4j works great and is extremely flexible.

Log4j provides a built-in appender that provides asynchronous event logging, this appender wraps around the appender you actually want to use (file, console, event log, etc.) and provides the required de-coupling.

To use it you need to configure log4j programatically or using XML, you cannot configure the asynchronous appender using a properties file. A very simple XML configuration file to use asynchronous logging writing the events to the console would be:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
	<appender name="stdout" class="org.apache.log4j.ConsoleAppender">
		<layout class="org.apache.log4j.PatternLayout">
			<param name="ConversionPattern" value="%d{yyyy/MM/dd HH:mm:ss,SSS} [%t] %-5p %c %x - %m%n"/>
		</layout>
	</appender>
        <appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
            <param name="BufferSize" value="500"/>
            <appender-ref ref="stdout"/>
        </appender>
	<root>
		<priority value="error"></priority>
		<appender-ref ref="ASYNC"/>
	</root>
</log4j:configuration>

The root appender references the AsyncAppender, and the AsyncAppender references the ConsoleAppender in turn. If you want to define your own logger you just have to point it to the ASYNC appender:

<logger name="com.spartanjava" additivity="false">
    <level value="info" />
    <appender-ref ref="ASYNC" />
</logger>

A few tips on using AsyncAppender and log4j in general:

• If you see duplicated entries on your log, make sure your logger has the additivity attribute set to false.
• The BufferSize parameter of the AsyncAppender defines the number of entries that will be stored in memory to let the actual appender catch up flushing them to their final destination. If this buffer fills up, the logging will switch to a synchronous mode until buffer space becomes available, potentially becoming a performance issue. Make sure this buffer is big enough, taking into account the IO speed and number of log entries generated by your application.
• You can add the Blocking parameter to the AsyncAppender definition in your config file, if you set it to false (the default value is true) when the buffer fills up, the appender will discard log events until new buffer space becomes available. This can be unacceptable in applications where logging information is critical, but on the other hand can be very desirable where performance is a critical factor.