Apache Log4j

Logo of Log4j. Source: Apache Log4j 2021h.
Logo of Log4j. Source: Apache Log4j 2021h.

Apache Log4j is a logging framework for Java. A Java application can use Log4j to log important events during the course of its lifetime. The logs are later used for tracing application behaviour or performing audits. Unlike debugging an application using debuggers, log generation requires no user intervention apart from the initial calls to the logger API. Logs generated from production deployments give lot of useful context. Logs are a permanent record of what happened unlike debugging sessions that are transient.

Log4j is an open-source project managed by the Apache Software Foundation. Log4j 1.x is incompatible with Log4j 2. The latter has better design and richer features.

GUI-based tools are available to view and analyze Log4j logs. Log4j has inspired logging solutions in other languages.

Discussion

  • What's the architecture of Log4j?
    Class diagram of Log4j. Source: Apache Log4j 2021i.
    Class diagram of Log4j. Source: Apache Log4j 2021i.

    Logger class is the starting point. Applications use the LogManager to return a logger object with a specific LoggingContext. If such a logger doesn't exist, it's created.

    Every logger has a LoggerConfig. These config objects are initialized from configuration files. LoggerConfig has a class hierarchy, that is, one config class can inherit from another.

    A LoggerConfig object is associated with one or more Appender objects that actually deliver log events. For example, a logger can log to both the console and to a file. Among the appender types are ConsoleAppender, FileAppender, RollingFileAppender, AsyncAppender, SocketAppender, SMTPAppender, and many more. Because config objects have a hierarchy, events will be sent to appenders of parent classes as well.

    Each appender can be configured to log messages in a specific format. Such formatting is the job of the Layout class.

    A LoggerConfig object is associated with a log level. Levels enable automatic filtering of events. In addition, Filter objects can be applied to LoggerConfig objects and Appender objects. Filters give more flexibility at runtime. Only those events that pass the filters are send to appenders.

  • What are the essential features of Log4j?

    Log4j separates its API from implementations. Hence, applications can swap implementations while the code consistently calls Log4j API.

    An application typically has many packages, classes, and methods. Log4j allows us to configure logging for each component separately. However, to save work on managing many configurations, configurations can be inherited and hence reused. Thus, X.Y.Z inherits and overrides some configuration from X.Y.

    Log4j has log levels to control the scope of logging. Developers can also define custom levels. Likewise, custom Message types, Layouts, Filters and Lookups can be created. Because of a plugin architecture, Log4j is easily extensible.

    Log4j has built-in support for JMX. Many parts of the Log4j system can be locally or remotely monitored and controlled.

    For better performance, Log4j supports asynchronous logging. It does garbage-free or low garbage logging so that pressure on garbage collector is relieved. For better concurrency, it locks resources at the lowest possible level. Log4j won't lose events when appenders are reconfigured. Hence, it can be also be used for audit logging.

  • What log levels are available in Log4j?
    Mapping of event levels to built-in log levels. Source: Apache Log4j 2021i.
    Mapping of event levels to built-in log levels. Source: Apache Log4j 2021i.

    A log level is assigned to a LoggerConfig object. If not, the object inherits the level from one of its ancestors along the class hierarchy. Log4j defines the following built-in levels: ALL (Integer.MAX_VALUE), TRACE (600), DEBUG (500), INFO (400), WARN (300), ERROR (200), FATAL (100), and OFF (0). Applications can also define custom levels. Levels OFF and ALL are not generally used in API calls.

    Every event is logged with an indication of its level. For example, if the application sees an unexpected scenario it may log it as WARN; log it as ERROR if the transaction can't be completed; log it as FATAL if application can't proceed further without a reboot or reinitialization.

    We can control how much to log without changing the code. For example, with log level ERROR only errors or more series events will be logged. So the call logger.error() will be logged but not logger.info(). During debugging, log level can be changed to DEBUG to log more messages.

    Log level can be dynamically changed at runtime: Configurator.setLevel("com.example.Foo", Level.DEBUG);

  • How can I format Log4j messages?
    Log4j conversion patterns and their performance. Source: Narayanaswamy 2016.
    Log4j conversion patterns and their performance. Source: Narayanaswamy 2016.

    Log4j supports a number of logging formats, what are formally called as Layouts: CSV, GELF, JSON (pretty or compact), JSON Template, Pattern, RFC5424 (enhanced Syslog), Serialized (deprecated), Syslog (BSD Syslog and used in Log4j 1.2), XML (pretty or compact), and YAML.

    If location information is logged (filename, line number) or location-specific patterns are used (%C or %class, %F or %file, %l or %location, %L or %line, %M or %method), there's a performance impact, especially for asynchronous loggers. Default configuration excludes location information.

    For the pattern layout, message formatting relies on the conversion pattern, which is similar to printf formatting in C. For example, the conversion pattern "[%-5p] %d{yyyy/MM/dd HH:mm:ss,SSS} %t %c: %m%n" logs right padded five-character width logging level (%-5p), datetime in a specific format (%d{pattern}), thread name (%t), logger name (%c), log message (%m) and newline (%n). The "-5" in "%-5p" is a format modifier. Modifier "20.-30" implies left pad to 20 characters or truncate from the end to 30 characters.

  • What are Log4j API essentials that a developer should know?

    The main package defining the Log4j API is org.apache.logging.log4j. Public message types are in package org.apache.logging.log4j.message. An implementation is available in package org.apache.logging.log4j.simple.

    Essentially, if you're developing a library you should use the API, not the concrete implementation. The application that uses your library will determine whether to use org.apache.logging.log4j.simple or another implementation of the Log4j API.

    Log4j API has four main interfaces:

    • LogBuilder: To construct log events before logging them.
    • Logger: Main interface of the log4j package. Class org.apache.logging.log4j.simple.SimpleLogger implements this interface.
    • Marker: Adds filterable information to log messages. Markers are hierarchical. For example, "Error" marker can have children "SystemError" and "ApplicationError".
    • ThreadContext.ContextStack: ThreadContext Stack interface.

    Among the classes are EventLogger, Level, LogManager, MarkerManager, ThreadContext, and more. LogManager is the anchor point for the Log4j system. Methods of this class include getContext(), getLogger(), getRootLogger(), and more.

    LoggingException is thrown when logging error occurs.

    For detailed information, consult the complete Log4j API documentation.

  • What's SLF4J and is it better than Log4j 2?
    Java logging packages and adapters. Source: Adapted from Apache Log4j 2021h.
    Java logging packages and adapters. Source: Adapted from Apache Log4j 2021h.

    SLF4J is a "simple logging facade", which is really a logging interface that applications can call. SLF4J forwards API calls to a concrete implementation used by the application. Such an implementation could be java.util.logging, Log4j, Logback, etc. A logging implementation can conform to SLF4J API and thereby enable developers to easily switch implementations. We may say that SLF4J does for logging what ORMs have done for database interfacing.

    Both Log4j 2 and SLF4J separate the API from the implementation. This gives developers flexibility. To use Log4j, application code can call SLF4J API that are then routed to Log4j's specific implementation. An alternative is to make Log4j calls but use the log4j-to-slf4j adapter to route calls to any SLF4J-compliant library. Hence, calling Log4j API directly in code doesn't tie you down to only Log4j.

    In fact, Log4j has some advantages over SLF4J. SLF4J can log only strings but Log4j can also log objects (implementations do the necessary conversions). Log4j supports Java 8 lambda expressions. Log4j has better support for garbage-free logging.

  • What are some best practices when using Log4j?
    An example of a logging message. Source: Adapted from Kuć 2020.
    An example of a logging message. Source: Adapted from Kuć 2020.

    Logging gives us insights into application use and helps in troubleshooting. Too much logging can affect application's performance. Too little logging can be ineffective. Logs should include descriptive messages along with sufficient context to make them useful.

    Don't log sensitive information such as passwords, credit card numbers or access tokens.

    Use suitable log levels for your messages and use them consistently.

    When an exception in logged, include the exception object for more context, such as try {...} catch (IOException ioe) { LOGGER.error("Error while executing main thread", ioe); }. This will include the stack trace, which can be very useful.

    Logging in JSON can be better for storing logs in a log management system. JSON is also better for multiline messages such as stack traces. An appender can be configured for logging in JSON.

    Where performance is critical, consider logging asynchronously. Logging directly over a network may not be ideal since network errors can cause some log entries to be lost. Inside containers, log files are not permanent. Hence, log to the standard output or over a network to a centralized system.

  • What other projects are inspired from or associated with Log4j?
    Screenshot of Apache Chainsaw. Source: Apache Chainsaw 2021.
    Screenshot of Apache Chainsaw. Source: Apache Chainsaw 2021.

    Apache Log4j comes under an umbrella project named The Apache Logging Services Project. Apart from Log4j, this includes Apache Log4j Kotlin and Apache Log4j Scala. Both these are APIs in their respective languages for Log4j. There's also Apache log4cxx and Apache Log4Net that are ports of Log4j for C++ and .NET respectively. Apache Log4j Audit is meant for audit logging.

    Other ports include log4c (C); log4js, JSNLog (JavaScript); log4perl (Perl); Apache log4php (PHP); and log4r (Ruby). Log4j can also be used in languages Clojure and Groovy.

    Apache Chainsaw is a GUI-based log viewer. It simplifies the analysis of logs via colour coding, tooltips, filtering, search function, navigation, etc. In fact, there are many other log viewers out there: OtrosLogViewer, Lilith, Eclipse LogViewer Version, Splunk, LogSaw, LogMX, and more.

    Complete log management is possible with SolarWinds Log Analyzer, Sematext Logs, Datadog, Fluentd, LogDNA, Splunk, Elastic Stack, and more.

  • As a beginner, how do I get started with Log4j?
    Showing configuration file, dependencies and generated log file in VS Code IDE. Source: Devopedia 2022.
    Showing configuration file, dependencies and generated log file in VS Code IDE. Source: Devopedia 2022.

    The official Apache Log4j 2 project site is an ideal place to get started. It has links to the user's guide, articles, tutorials and FAQ. It also hosts the project's Javadoc API documentation.

    Installing and configuring Log4j 2 is also described. You need to update project dependencies in file pom.xml used by Maven builder. Log4j 2 configuration can be in XML, JSON, YAML or properties formats. This file would include the appenders, their pattern layouts, the loggers, and the mapping between loggers and appenders.

    Documentation informs how to configure Log4j for other builders including Maven, Ivy, Gradle, and SBT. Project dependencies are also listed.

    Migration guides are available for those who wish to migrate from Log4j 1.x to 2.x.

Milestones

1991

James Gosling and others start work on a new language for interactive television. They name it Oak. In 1995, this is renamed to Java. The first public release of the language happens in 1996.

1996

The E.U. SEMPER project creates its own tracing API for Java, which later evolves to Log4j. In these early years of Java, there's no universal logging framework or library. Every application includes its own logging or tracing API.

Jan
1999

The Apache Software Foundation adopts Log4j under its The Apache Logging Services Project. This project "creates and maintains open-source software related to the logging of application behavior." First versions of Log4j appears in October.

Jan
2001

Apache Log4j v1.0 is released. Package hierarchy starts at org.apache.log4j. In subsequent years, Log4j proves to be the first of its kind to gain traction in the Java world for logging. It's also credited for introducing hierarchical logging, a concept that other loggers go on to adopt.

Feb
2002

Sun Microsystems releases Java 1.4 that includes java.util.logging (available since mid-2001 in its beta release). Its API appears to be similar to Log4j. Open source implementations of this API for Java 1.2 and 1.3 become available.

May
2002

Apache Log4j v1.2 is released. This release is maintained till May 2012 when v1.2.17 is released. In August 2015, Apache announces end of life for Log4j 1.x.

Jul
2012
An example of how Log4j 2 simplifies the syntax compared to Log4j 1.x. Source: Raible 2014.
An example of how Log4j 2 simplifies the syntax compared to Log4j 1.x. Source: Raible 2014.

Work starts on Apache Log4j 2. It's a complete rewrite of Log4j and it's incompatible with Log4j 1.x. It's inspired by Log4j 1.x and java.util.logging. Log4j v2.0 released in July 2014. Package hierarchy changes in Log4j 2 to org.apache.logging.log4j.

Sep
2017

Java 9 is released. This breaks compatibility for Log4j 1.x due to a feature called Mapped Diagnostic Context (MDC). MDC's equivalent in Log4j 2 is Thread Context Map. Applications using Log4j 1.x but without MDC may still work. But given Log4j 2's many features, developers will find it worthwhile to migrate to it.

Nov
2021
Log4j JNDI attack explained. Source: GovCERT.ch 2021.
Log4j JNDI attack explained. Source: GovCERT.ch 2021.

A member of the Alibaba Cloud Security Team finds a vulnerability in Log4j 2. The team informs Apache. Called Log4Shell, the vulnerability is made public in December. Given the widespread use of Java and hence Log4j 2, this is considered a series vulnerability affecting cloud servers, enterprise applications, IoT devices, routers, firewalls, printers, etc. Apache fixes the issue in releases 2.15.0, 2.16.0 and 2.17.0 (Dec 2021). Meanwhile, websites and businesses list affected software and mitigation strategies.

Sample Code

  • // Source: https://sematext.com/blog/log4j2-tutorial/
    // Accessed 2021-12-28
    import org.apache.logging.log4j.LogManager;
    import org.apache.logging.log4j.Logger;
     
    public class MyApp {
        private static final Logger LOGGER = LogManager.getLogger(MyApp.class);
     
        public static void main(String[] args) {
            LOGGER.info("This is an INFO level log message!");
            LOGGER.error("This is an ERROR level log message!");
        }
    }
     

References

  1. Apache Chainsaw. 2021. "Apache Chainsaw: Homepage." Version 2.2.0-SNAPSHOT, The Apache Software Foundation, June 13. Accessed 2021-12-28.
  2. Apache Log4j. 2012a. "Changes Report." Apache log4j 1.2, The Apache Software Foundation, May 6. Accessed 2021-12-28.
  3. Apache Log4j. 2012b. "Apache log4j 1.2." Apache log4j 1.2, v1.2.17, The Apache Software Foundation, May 13. Updated 2015-08-05. Accessed 2021-12-28.
  4. Apache Log4j. 2021a. "Changes." Apache Log4j, v2.17.0, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  5. Apache Log4j. 2021b. "Log4j 2 Compatibility with Log4j 1." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  6. Apache Log4j. 2021c. "Migrating from Log4j 1.x." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  7. Apache Log4j. 2021d. "Configuration." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  8. Apache Log4j. 2021e. "Log4j Runtime Dependencies." Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  9. Apache Log4j. 2021f. "Project Dependencies." Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  10. Apache Log4j. 2021g. "Maven, Ivy, Gradle, and SBT Artifacts." Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  11. Apache Log4j. 2021h. "Frequently Asked Questions." Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  12. Apache Log4j. 2021i. "Architecture." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  13. Apache Log4j. 2021j. "JMX." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-30.
  14. Apache Log4j. 2021k. "Introduction." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-28.
  15. Apache Log4j. 2021l. "Custom Log Levels." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2021-12-28.
  16. Apache Log4j. 2021m. "Layouts." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  17. Apache Log4j. 2021n. "Layouts." Manual, Apache Log4j 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  18. Baeldung. 2017. "Intro to Log4j2 - Appenders, Layouts and Filters." Baeldung, February 28. Updated 2020-02-12. Accessed 2021-12-28.
  19. Bradley, Susan. 2021. "What’s all the fuss with Log4j2?" Blog, The Microsoft Patch Lady, Computerworld, IDG Communications Inc., December 20. Accessed 2021-12-28.
  20. Gilstrap, Brian. 2001. "The Java Logging API." Object Computing, June. Accessed 2022-01-04.
  21. GovCERT.ch. 2021. "Zero-Day Exploit Targeting Popular Java Library Log4j." Blog, Swiss Government Computer Emergency Response Team, December 12. Updated 2021-12-18. Accessed 2021-12-28.
  22. Groovy Docs. 2021. "Annotation Type Log4j2." Documentation, Groovy, v3.0.9, The Apache Software Foundation, September 2. Accessed 2021-12-29.
  23. IBM. 2021. "An update on the Apache Log4j 2.x vulnerabilities." IBM PSIRT Blog, IBM, December 11. Updated 2021-12-27. Accessed 2021-12-28.
  24. Khudairi, Sally. 2015. "Apache™ Logging Services™ Project Announces Log4j™ 1 End-Of-Life; Recommends Upgrade to Log4j 2." Apache Blog, The Apache Software Foundation Blog, August 6. Accessed 2021-12-28.
  25. Kuć, Rafal. 2020. "Java Logging Best Practices: 10+ Tips You Should Know to Get the Most Out of Your Logs." Blog, Sematext, August 3. Accessed 2021-12-29.
  26. Kuć, Rafal. 2021. "Log4j 2 Configuration Example: Tutorial on How to Use It for Efficient Java Logging." Blog, Sematext, March 11. Accessed 2021-12-28.
  27. Lambda Island. 2020. "Logging in Clojure: Making Sense of the Mess." Blog, Lambda Island, June 12. Accessed 2021-12-29.
  28. Log4j API Docs. 2021a. "Package org.apache.logging.log4j." Apache Log4j API 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  29. Log4j API Docs. 2021b. ".Interface Marker" Apache Log4j API 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  30. Log4j API Docs. 2021c. "Class SimpleLogger." Apache Log4j API 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  31. Log4j API Docs. 2021d. "Class LogManager." Apache Log4j API 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  32. Log4j API Docs. 2021e. "Overview." Apache Log4j API 2.17.1, The Apache Software Foundation, December 28. Accessed 2022-01-04.
  33. Microsoft Security. 2021. "Guidance for preventing, detecting, and hunting for exploitation of the Log4j 2 vulnerability." Blog, Microsoft Security, December 11. Updated 2021-12-27. Accessed 2021-12-28.
  34. Murphy, Kieron. 1996. "So why did they decide to call it Java?" InfoWorld, IDG Communications Inc., October 4. Accessed 2021-12-29.
  35. NCSC-NL. 2021. "Overview of software (un)affected by Log4j." NCSC-NL/log4shell, on GitHub, December 27. Accessed 2021-12-28.
  36. Narayanaswamy. 2016. "Log4j common conversion patterns." Tutorial, Narayana Tutorial, February 8. Updated 2019-02-26. Accessed 2021-12-28.
  37. Popma, Remko. 2016. "Moving on to Log4j 2: Log4j 1.2 is broken on Java 9." Apache Blog, Apache Logging Services, July 17. Accessed 2021-12-28.
  38. Raible, Matt. 2014. "Apache Log4j 2.0 - Worth the Upgrade?" InfoQ, July 31. Accessed 2021-12-28.
  39. Software Testing Help. 2019. "Top 8 BEST Log Management Software | Log Analysis Tool Review 2021." Software Testing Help, September 16. Updated 2021-11-30. Accessed 2022-01-04.
  40. Syrjälä, Juha. 2010. "Is there a log file analyzer for log4j files?" StackOverflow, April 4. Updated 2020-03-16. Accessed 2021-12-28.
  41. The Apache Software Foundation. 2021a. "Apache logging services." The Apache Software Foundation. Accessed 2021-12-28.
  42. The Apache Software Foundation. 2021b. "Apache Log4j 2: Project Data File." The Apache Software Foundation. Accessed 2021-12-29.
  43. Wikipedia. 2021a. "Log4j." Wikipedia, December 25. Accessed 2021-12-28.
  44. Wikipedia. 2021b. "Java (programming language)." Wikipedia, December 30. Accessed 2021-12-30.

Further Reading

  1. Kuć, Rafal. 2021. "Log4j 2 Configuration Example: Tutorial on How to Use It for Efficient Java Logging." Blog, Sematext, March 11. Accessed 2021-12-28.
  2. Baeldung. 2017. "Intro to Log4j2 - Appenders, Layouts and Filters." Baeldung, February 28. Updated 2020-02-12. Accessed 2021-12-28.
  3. The Apache Software Foundation. 2021. "Apache Log4j 2: User's Guide." v2.17.0, The Apache Software Foundation, December 17. Accessed 2022-01-04.
  4. Goers, Ralph. 2019. "Why was Log4j 2 created?" Blog, December 15. Accessed 2021-12-28.
  5. Mkyong. 2019. "Apache Log4j 2 Tutorials." March 29. Accessed 2021-12-28.
  6. Janssen, Thorben. 2018. "Java Logging Frameworks: log4j vs logback vs log4j2." Stackify, October 30. Accessed 2021-12-28.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
7
0
1685
2155
Words
1
Likes
3476
Hits

Cite As

Devopedia. 2022. "Apache Log4j." Version 7, January 4. Accessed 2023-11-13. https://devopedia.org/apache-log4j
Contributed by
1 author


Last updated on
2022-01-04 08:05:21
  • Log4j Configuration
  • Asynchronous Logging with Log4j
  • Log4j 1.x to 2.x Migration
  • SLF4J
  • Log4Shell
  • Log Analytics