Skip to content

Releases: odygrd/quill

v4.1.0

25 May 12:55
2ec2003
Compare
Choose a tag to compare
  • Following the transition from a compiled to a header-only library, the target_compile_options previously applied to
    the compiled library were mistakenly propagated to all programs linking against the header-only library.
    This issue is now fixed by removing those flags and explicitly adding them to tests and examples. As a result,
    executable targets no longer inherit flags from the library.
  • Removed unnecessary template specializations and merged their logic into the primary template
    for ArgSizeCalculator, Encoder, and Decoder using if constexpr.
  • Eliminated <functional> header dependency in the frontend
  • Replaced %(structured_keys) with %(named_args) in the PatternFormatter. This change now appends the
    entire key-value pair of named args to the message, not just the names.
  • Relocated certain classes to the detail namespace
  • Replaced sprintf with snprintf to fix macOS warning.
  • Reviewed and removed gcc cold attribute from a few functions.
  • Minor backend thread optimisations when logging c style strings or char arrays
  • Improved backend thread variable and function names and fixed a bug for an edge case when the transit event hard limit is reached

v4.0.0

22 May 11:57
06a0c04
Compare
Choose a tag to compare

This version represents a major revamp of the library, aiming to simplify and modernize it, resulting in the removal
of a few features. Please read through the changes carefully before upgrading, as it is not backwards compatible with
previous versions and some effort will be required to migrate.

I understand that these changes may inconvenience some existing users. However, they have been made with good
intentions, aiming to improve and refine the logging library. This involved significant effort and dedication.

Bug fixes and releases for v3 will continue to be supported under the v3.x.x branch.

Comparison

  • This version significantly improves compile times. Taking a look at some compiler profiling for a Release build with
    clang 15, we can see the difference. Below are the two compiler flamegraphs for building the recommended_usage
    example from the new version and the wrapper_lib example from the previous version.

The below flamegraph shows the difference in included headers between the two versions

Version Compiler FlameGraph
v4.0.0 quill_v4_compiler_profile.speedscope.png
v3.8.0 quill_v3_compiler_profile.speedscope.png

A new compiler benchmark has been introduced. A Python script generates 2000 distinct log statements with various
arguments. You can find the benchmark here.
Compilation now takes only about 30 seconds, whereas the previous version required over 4 minutes.

Version Compiler FlameGraph
v4.0.0 quill_v4_compiler_bench.speedscope.png
v3.8.0 quill_v3_compiler_bench.speedscope.png
  • Minor increase in backend thread throughput compared to the previous version.
Version Backend Throughput
v4.0.0 4.56 million msgs/sec average, total time elapsed: 876 ms for 4000000 log messages
v3.8.0 4.39 million msgs/sec average, total time elapsed: 910 ms for 4000000 log messages
  • Significant boost in hot path latency when logging complex types such as std::vector.
    The performance remains consistent when logging only primitive types or strings in both versions. Refer
    here for updated and detailed benchmarks.

Changes

  • Improved compile times

The library has been restructured to minimize the number of required headers. Refactoring efforts have focused on
decoupling the frontend from the backend, resulting in reduced dependencies. Accessing the frontend logging functions
now does not demand inclusion of any backend logic components.

 "quill/Backend.h" - It can be included once to start the backend logging thread, typically in main.cpp 
                     or in a wrapper library.
 
 "quill/Frontend.h"` - Used to create or obtain a `Logger*` or a `Sink`. It can be included in limited 
                       files, since an obtained `Logger*` has pointer stability and can be passed around.
 
 "quill/Logger.h", "quill/LogMacros.h" - These two files are the only ones needed for logging and will have 
                                         to be included in every file that requires logging functionality.
  • Backend formatting for user-defined and standard library types

One of the significant changes lies in the support for formatting both user-defined and standard library types.
Previously, the backend thread handled the formatting of these types sent by the frontend. It involved making a copy for
any object passed to the LOG_ macros as an argument using the copy constructor of a complex type instead of directly
serializing the data to the SPSC queue. While this method facilitated logging copy-constructible user-defined types with
ease, it also posed numerous challenges for asynchronous logging:

  • Error-Prone Asynchronous Logging: Copying and formatting user-defined types on the backend thread in an
    asynchronous logging setup could lead to errors. Previous versions attempted to address this issue with type
    trait checks, which incurred additional template instantiations and compile times.
  • Uncertainty in Type Verification: It is challenging to confidently verify types, as some trivially copiable
    types, such as struct A { int* m; }, could still lead to issues due to potential modifications by the user
    before formatting.
  • Hidden Performance Penalties: Logging non-trivially copiable types could introduce hidden cache coherence
    performance penalties due to memory allocations and deallocations across threads. For instance,
    consider std::vector<int> passed as a log argument. The vector is emplaced into the SPSC queue by the frontend,
    invoking the copy constructor dynamically allocating memory as the only members copied to SPSC queue
    are size, capacity, and data*. The backend thread reads the object, formats it, and then invokes the destructor,
    which in turn synchronizes the
    freed memory back to the frontend.

Additionally, after years of professional use and based on experience, it has been observed that user-defined types
are often logged during program initialization, with fewer occurrences on the hot path where mostly built-in types are
logged. In such scenarios, the overhead of string formatting on the frontend during initialization is not an issue.

In this new version, the use of the copy constructor for emplacing objects in the queue has been abandoned. Only POD
types are copied, ensuring that only raw, tangible data is handled without any underlying pointers pointing to other
memory locations. The only exception to this are the pointers to Metadata, LoggerBase and DecodeFunction
that are passed internally for each log message. Log arguments sent from the frontend must undergo
serialization beforehand. While this approach resolves the above issues, it does introduce more complexity when
dealing with user-defined or standard library types.

Built-in types and strings are logged by default, with the formatting being offloaded to the backend. Additionally,
there is built-in support for most standard library types, which can also be directly passed to the logger by
including the relevant header from quill/std.

The recommendation for user-defined types is to format them into strings before passing them to the LOG_ macros using
your preferred method. You can find an example of this here.

It's also possible to extend the library by providing template specializations to serialize the user-defined types
and offload their formatting to the backend. However, this approach should only be pursued if you cannot tolerate the
formatting overhead in that part of your program. For further guidance, refer to this example.

  • Header-Only library

The library is now header-only. This change simplifies exporting the library as a C++ module in the future. See
here on how to build a wrapper static library which includes the backend and will minimise the compile times.

  • Preprocessor flags moved to template parameters

Most preprocessor flags have been moved to template parameters, with only a few remaining as CMake options. This
change simplifies exporting the library as a C++ module in the future.

  • Renamed Handlers to Sinks

To enhance clarity, handlers have been renamed to sinks.

  • PatternFormatter moved to Logger

The PatternFormatter has been relocated from Sink to Logger, enabling a logger object to log in a specific
format. This allows for different formats within the same output file, a feature not previously possible.

  • Split Configuration

The configuration settings have been divided into FrontendOptions and BackendOptions.

  • Refactoring of backend classes

MacroMetadata and many backend classes have undergone refactoring, resulting in reduced memory requirements.

  • Improved wide strings handling on Windows

The library now offers significant performance enhancements for handling wide strings on Windows platforms.
It's important to note that only wide strings containing ASCII characters are supported. Previously, wide strings were
converted to narrow strings at the frontend, impacting the critical path of the application.
With this update, the underlying wide char buffer is copied and the conversion to UTF-8 encoding is deferred to
the backend logging thread. Additionally, this update adds support for logging STL containers consisting of
wide strings

  • Default logger removal

The default logger, along with the configuration inheritance feature during logger creation, has been removed. Now, when
creating a new logger instance, configurations such as the Sink and log ...

Read more

v3.9.0

18 May 02:11
Compare
Choose a tag to compare
  • Fix bug in ConsoleHandler when dynamic log level is used (#421)
  • Fix bug in TransitEvent when dynamic log level is used (#427)
  • Fix build error for Intel compiler classic (#414)
  • Added JsonConsoleHandler (#413)
  • Fix fold expression argument evaluation sequence. This bug could occur when logging c style strings

v3.8.0

24 Mar 13:57
Compare
Choose a tag to compare
  • Refactored MacroMetadata class to reduce its size.
  • Renamed some attributes in the PatternFormatter class for clarity. If you are using a custom format pattern, update the attribute names in your code to match the new names.
  • Improved accuracy of log statement timestamps. Previously, the timestamp was taken after checking if the queue had enough space to push the message, which could make it less accurate. Additionally, in the case of a blocking queue, the timestamp could be later in time. Now, the timestamp is taken and stored right after the log statement is issued, before checking for the queue size.
  • Reduced template instantiations during logging operations on the hot path. Fold expressions are now used for encoding/decoding arguments, minimizing template recursion overhead.
  • Removed compile-time format checks due to their significant impact on template instantiations, especially considering that only a few cases are invalid. For instance, while fmt::format("{}", 1, 2) is considered valid, fmt::format("{} {}", 1) is deemed invalid. In cases where an invalid format string is detected, the backend worker thread catches the generated exception and logs an error.
  • The throughput of the backend worker thread has been improved by approximately 5%.
  • Detect tmux as colour terminal. (#410)

v3.7.0

18 Mar 13:10
Compare
Choose a tag to compare
  • Fixed crash triggered by insufficient space in the queue upon invocation of flush(). (#398)
  • Fixed windows clang-cl build error. (#400)
  • Fixed compilation errors encountered on FreeBSD and extended get_thread_id() support to various other BSD operating systems. (#401)
  • Fix open_file() in the FileHandler to also create the parent path before opening the file. (#395)
  • Enhance logic for backend thread's flush() invocation; it now triggers only if the handler has previously written data. (#395)
  • Address an uncaught exception in the backend thread that could occur when a user manually removes the log file from the terminal while the logger is running. (#395)
  • Ensure that after a logger is removed, there are no subsequent calls to the Handler's flush() or run_loop(), provided the Handler is not shared. (#395)
  • Ignore the virtual destructor missing warning for the CustomTags class. (#402)
  • Update bundled libfmt to v10.2.1

v3.6.0

18 Dec 10:57
be7c583
Compare
Choose a tag to compare
  • Fixed QUILL_LOGGER_CALL_NOFN_LIMIT macros. (#381)
  • Resolved a bug that caused reading destructed arguments when structured logging format was used.
  • Modified member access from private to protected in ConsoleHandler for potential inheritance purposes.
  • Eliminated redundant whitespaces within JsonFileHandler.
  • Fixed JsonFileHandler to notify the file event notifier before log message writes.
  • Implemented a new attribute called %(structured_keys) within the PatternFormatter to facilitate the inclusion
    of keys in messages when using structured log formatting. This addition is useful for instances where logging occurs
    in both JSON and regular log formats, enabling the display of keys within the regular log-formatted messages.
    See updated example_json_structured_log.cpp

v3.5.1

08 Dec 22:44
Compare
Choose a tag to compare
  • Resolved issue with accessing the name() method within the Logger class. (#378)
  • Fixed a compilation error in SignalHandler specific to Windows when QUILL_DISABLE_NON_PREFIXED_MACROS is defined. (#380)

v3.5.0

27 Nov 17:14
b356340
Compare
Choose a tag to compare
  • Fixed LOG_TRACE_CFORMAT macros.
  • Added support for compile-time custom tags in quill::MacroMetadata to enhance message filtering and incorporate
    static information. New log macros suffixed with _WITH_TAGS introduced for this feature.
    Additionally, %(custom_tags) parameter added to PatternFormatter. (#349)
    See example_custom_tags.cpp
  • Some code improvements aiming to reduce total compilation time

v3.4.1

20 Nov 14:16
98438f5
Compare
Choose a tag to compare
  • Reduce backend worker unnecessary allocation. (#368)
  • Adjusted handling for empty std::string_view instances, addressing an issue where logging empty strings triggered an unintended memcpy with zero size and a nullptr, leading to address sanitizer warnings.
  • Fix clang build error when using -DQUILL_NO_EXCEPTIONS:BOOL=ON. (#357)

v3.4.0

05 Nov 13:46
cf109a1
Compare
Choose a tag to compare
  • Resolved bad_variant_access error occurring when using Quill as a pre-compiled library with a distinct queue
    type. (#276)

  • Resolved a bug in RotatingFileHandler associated with logfiles located outside the working directory,
    specifically when used with open_mode a. (#340)

  • Added a name() method to the Logger class which provides the logger name. (#345)

  • Fixed library and include paths in the pkg-config configuration. (#352)

  • Move get_root_logger() definition from cpp to the header file (#348)

  • Introduced support for logging character arrays. You can now log character arrays, even when they don't contain a null-terminating character. Additionally, character arrays with null characters in the middle are supported, and the logger will capture the content until the null character is encountered. (#353)

    For example

        union
      {
        char no_0[2];
        char mid_0[6]{'1', '2', '3', '4', '\0', 6};
      } char_arrays;
    
      // only output "12" even if there's no '\0' at the end
      LOG_INFO(logger, R"(This is a log info example for char array without '\0': {})", char_arrays.no_0);
    
      // output "1234" until the '\0'
      LOG_INFO(logger, R"(This is a log info example for char array with '\0' in middle: {})",
               char_arrays.mid_0);
  • Minor improvements in BoundedQueue and UnboundedQueue . Throughput is slightly improved.
    Previous: 2.21 million msgs/sec average, total time elapsed: 1809 ms for 4000000 log messages.
    New: 2.24 million msgs/sec average, total time elapsed: 1787 ms for 4000000 log messages.

  • Disable fmt::join(data, "") at compile time. (#356)

  • Fix compile error in Apple Clang 12. (#360)

  • Add guards for redefined preprocessor variables.

  • Fix uint64_t to time_t implicit conversion error in Clang 18.

  • Update bundled libfmt to v10.1.1