Releases: odygrd/quill
v4.1.0
- Following the transition from a compiled to a header-only library, the
target_compile_options
previously applied to
the compiled library were mistakenly propagated to all programs linking against the header-only library.
This issue is now fixed by removing those flags and explicitly adding them to tests and examples. As a result,
executable targets no longer inherit flags from the library. - Removed unnecessary template specializations and merged their logic into the primary template
forArgSizeCalculator
,Encoder
, andDecoder
using if constexpr. - Eliminated
<functional>
header dependency in the frontend - Replaced
%(structured_keys)
with%(named_args)
in thePatternFormatter
. This change now appends the
entire key-value pair of named args to the message, not just the names. - Relocated certain classes to the
detail
namespace - Replaced
sprintf
withsnprintf
to fix macOS warning. - Reviewed and removed gcc cold attribute from a few functions.
- Minor backend thread optimisations when logging c style strings or char arrays
- Improved backend thread variable and function names and fixed a bug for an edge case when the transit event hard limit is reached
v4.0.0
This version represents a major revamp of the library, aiming to simplify and modernize it, resulting in the removal
of a few features. Please read through the changes carefully before upgrading, as it is not backwards compatible with
previous versions and some effort will be required to migrate.
I understand that these changes may inconvenience some existing users. However, they have been made with good
intentions, aiming to improve and refine the logging library. This involved significant effort and dedication.
Bug fixes and releases for v3
will continue to be supported under the v3.x.x
branch.
Comparison
- This version significantly improves compile times. Taking a look at some compiler profiling for a
Release
build with
clang 15, we can see the difference. Below are the two compiler flamegraphs for building therecommended_usage
example from the new version and thewrapper_lib
example from the previous version.
The below flamegraph shows the difference in included headers between the two versions
Version | Compiler FlameGraph |
---|---|
v4.0.0 | |
v3.8.0 |
A new compiler benchmark has been introduced. A Python script generates 2000 distinct log statements with various
arguments. You can find the benchmark here.
Compilation now takes only about 30 seconds, whereas the previous version required over 4 minutes.
Version | Compiler FlameGraph |
---|---|
v4.0.0 | |
v3.8.0 |
- Minor increase in backend thread throughput compared to the previous version.
Version | Backend Throughput |
---|---|
v4.0.0 | 4.56 million msgs/sec average, total time elapsed: 876 ms for 4000000 log messages |
v3.8.0 | 4.39 million msgs/sec average, total time elapsed: 910 ms for 4000000 log messages |
- Significant boost in hot path latency when logging complex types such as
std::vector
.
The performance remains consistent when logging only primitive types or strings in both versions. Refer
here for updated and detailed benchmarks.
Changes
- Improved compile times
The library has been restructured to minimize the number of required headers. Refactoring efforts have focused on
decoupling the frontend from the backend, resulting in reduced dependencies. Accessing the frontend logging functions
now does not demand inclusion of any backend logic components.
"quill/Backend.h" - It can be included once to start the backend logging thread, typically in main.cpp
or in a wrapper library.
"quill/Frontend.h"` - Used to create or obtain a `Logger*` or a `Sink`. It can be included in limited
files, since an obtained `Logger*` has pointer stability and can be passed around.
"quill/Logger.h", "quill/LogMacros.h" - These two files are the only ones needed for logging and will have
to be included in every file that requires logging functionality.
- Backend formatting for user-defined and standard library types
One of the significant changes lies in the support for formatting both user-defined and standard library types.
Previously, the backend thread handled the formatting of these types sent by the frontend. It involved making a copy for
any object passed to the LOG_
macros as an argument using the copy constructor of a complex type instead of directly
serializing the data to the SPSC queue. While this method facilitated logging copy-constructible user-defined types with
ease, it also posed numerous challenges for asynchronous logging:
- Error-Prone Asynchronous Logging: Copying and formatting user-defined types on the backend thread in an
asynchronous logging setup could lead to errors. Previous versions attempted to address this issue with type
trait checks, which incurred additional template instantiations and compile times. - Uncertainty in Type Verification: It is challenging to confidently verify types, as some trivially copiable
types, such asstruct A { int* m; }
, could still lead to issues due to potential modifications by the user
before formatting. - Hidden Performance Penalties: Logging non-trivially copiable types could introduce hidden cache coherence
performance penalties due to memory allocations and deallocations across threads. For instance,
considerstd::vector<int>
passed as a log argument. The vector is emplaced into the SPSC queue by the frontend,
invoking the copy constructor dynamically allocating memory as the only members copied to SPSC queue
aresize
,capacity
, anddata*
. The backend thread reads the object, formats it, and then invokes the destructor,
which in turn synchronizes the
freed memory back to the frontend.
Additionally, after years of professional use and based on experience, it has been observed that user-defined types
are often logged during program initialization, with fewer occurrences on the hot path where mostly built-in types are
logged. In such scenarios, the overhead of string formatting on the frontend during initialization is not an issue.
In this new version, the use of the copy constructor for emplacing objects in the queue has been abandoned. Only POD
types are copied, ensuring that only raw, tangible data is handled without any underlying pointers pointing to other
memory locations. The only exception to this are the pointers to Metadata
, LoggerBase
and DecodeFunction
that are passed internally for each log message. Log arguments sent from the frontend must undergo
serialization beforehand. While this approach resolves the above issues, it does introduce more complexity when
dealing with user-defined or standard library types.
Built-in types and strings are logged by default, with the formatting being offloaded to the backend. Additionally,
there is built-in support for most standard library types, which can also be directly passed to the logger by
including the relevant header from quill/std
.
The recommendation for user-defined types is to format them into strings before passing them to the LOG_
macros using
your preferred method. You can find an example of this here.
It's also possible to extend the library by providing template specializations to serialize the user-defined types
and offload their formatting to the backend. However, this approach should only be pursued if you cannot tolerate the
formatting overhead in that part of your program. For further guidance, refer to this example.
- Header-Only library
The library is now header-only. This change simplifies exporting the library as a C++ module in the future. See
here on how to build a wrapper static library which includes the backend and will minimise the compile times.
- Preprocessor flags moved to template parameters
Most preprocessor flags have been moved to template parameters, with only a few remaining as CMake
options. This
change simplifies exporting the library as a C++ module in the future.
- Renamed Handlers to Sinks
To enhance clarity, handlers have been renamed to sinks.
- PatternFormatter moved to Logger
The PatternFormatter
has been relocated from Sink
to Logger
, enabling a logger object to log in a specific
format. This allows for different formats within the same output file, a feature not previously possible.
- Split Configuration
The configuration settings have been divided into FrontendOptions
and BackendOptions
.
- Refactoring of backend classes
MacroMetadata
and many backend classes have undergone refactoring, resulting in reduced memory requirements.
- Improved wide strings handling on Windows
The library now offers significant performance enhancements for handling wide strings on Windows platforms.
It's important to note that only wide strings containing ASCII characters are supported. Previously, wide strings were
converted to narrow strings at the frontend, impacting the critical path of the application.
With this update, the underlying wide char buffer is copied and the conversion to UTF-8 encoding is deferred to
the backend logging thread. Additionally, this update adds support for logging STL containers consisting of
wide strings
- Default logger removal
The default logger, along with the configuration inheritance feature during logger creation, has been removed. Now, when
creating a new logger instance, configurations such as the Sink
and log ...
v3.9.0
- Fix bug in
ConsoleHandler
when dynamic log level is used (#421) - Fix bug in
TransitEvent
when dynamic log level is used (#427) - Fix build error for Intel compiler classic (#414)
- Added
JsonConsoleHandler
(#413) - Fix fold expression argument evaluation sequence. This bug could occur when logging c style strings
v3.8.0
- Refactored
MacroMetadata
class to reduce its size. - Renamed some attributes in the
PatternFormatter
class for clarity. If you are using a custom format pattern, update the attribute names in your code to match the new names. - Improved accuracy of log statement timestamps. Previously, the timestamp was taken after checking if the queue had enough space to push the message, which could make it less accurate. Additionally, in the case of a blocking queue, the timestamp could be later in time. Now, the timestamp is taken and stored right after the log statement is issued, before checking for the queue size.
- Reduced template instantiations during logging operations on the hot path. Fold expressions are now used for encoding/decoding arguments, minimizing template recursion overhead.
- Removed compile-time format checks due to their significant impact on template instantiations, especially considering that only a few cases are invalid. For instance, while
fmt::format("{}", 1, 2)
is considered valid,fmt::format("{} {}", 1)
is deemed invalid. In cases where an invalid format string is detected, the backend worker thread catches the generated exception and logs an error. - The throughput of the backend worker thread has been improved by approximately 5%.
- Detect
tmux
as colour terminal. (#410)
v3.7.0
- Fixed crash triggered by insufficient space in the queue upon invocation of
flush()
. (#398) - Fixed windows clang-cl build error. (#400)
- Fixed compilation errors encountered on FreeBSD and extended
get_thread_id()
support to various other BSD operating systems. (#401) - Fix
open_file()
in theFileHandler
to also create the parent path before opening the file. (#395) - Enhance logic for backend thread's
flush()
invocation; it now triggers only if the handler has previously written data. (#395) - Address an uncaught exception in the backend thread that could occur when a user manually removes the log file from the terminal while the logger is running. (#395)
- Ensure that after a logger is removed, there are no subsequent calls to the Handler's
flush()
orrun_loop()
, provided the Handler is not shared. (#395) - Ignore the virtual destructor missing warning for the
CustomTags
class. (#402) - Update bundled
libfmt
tov10.2.1
v3.6.0
- Fixed
QUILL_LOGGER_CALL_NOFN_LIMIT
macros. (#381) - Resolved a bug that caused reading destructed arguments when structured logging format was used.
- Modified member access from
private
toprotected
inConsoleHandler
for potential inheritance purposes. - Eliminated redundant whitespaces within
JsonFileHandler
. - Fixed
JsonFileHandler
to notify the file event notifier before log message writes. - Implemented a new attribute called
%(structured_keys)
within thePatternFormatter
to facilitate the inclusion
of keys in messages when using structured log formatting. This addition is useful for instances where logging occurs
in both JSON and regular log formats, enabling the display of keys within the regular log-formatted messages.
See updated example_json_structured_log.cpp
v3.5.1
v3.5.0
- Fixed
LOG_TRACE_CFORMAT
macros. - Added support for compile-time custom tags in
quill::MacroMetadata
to enhance message filtering and incorporate
static information. New log macros suffixed with_WITH_TAGS
introduced for this feature.
Additionally,%(custom_tags)
parameter added toPatternFormatter
. (#349)
See example_custom_tags.cpp - Some code improvements aiming to reduce total compilation time
v3.4.1
- Reduce backend worker unnecessary allocation. (#368)
- Adjusted handling for empty
std::string_view
instances, addressing an issue where logging empty strings triggered an unintendedmemcpy
with zero size and a nullptr, leading to address sanitizer warnings. - Fix clang build error when using
-DQUILL_NO_EXCEPTIONS:BOOL=ON
. (#357)
v3.4.0
-
Resolved
bad_variant_access
error occurring when using Quill as a pre-compiled library with a distinct queue
type. (#276) -
Resolved a bug in
RotatingFileHandler
associated with logfiles located outside the working directory,
specifically when used with open_modea
. (#340) -
Added a
name()
method to the Logger class which provides the logger name. (#345) -
Fixed library and include paths in the pkg-config configuration. (#352)
-
Move
get_root_logger()
definition from cpp to the header file (#348) -
Introduced support for logging character arrays. You can now log character arrays, even when they don't contain a null-terminating character. Additionally, character arrays with null characters in the middle are supported, and the logger will capture the content until the null character is encountered. (#353)
For example
union { char no_0[2]; char mid_0[6]{'1', '2', '3', '4', '\0', 6}; } char_arrays; // only output "12" even if there's no '\0' at the end LOG_INFO(logger, R"(This is a log info example for char array without '\0': {})", char_arrays.no_0); // output "1234" until the '\0' LOG_INFO(logger, R"(This is a log info example for char array with '\0' in middle: {})", char_arrays.mid_0);
-
Minor improvements in
BoundedQueue
andUnboundedQueue
. Throughput is slightly improved.
Previous: 2.21 million msgs/sec average, total time elapsed: 1809 ms for 4000000 log messages.
New: 2.24 million msgs/sec average, total time elapsed: 1787 ms for 4000000 log messages. -
Disable
fmt::join(data, "")
at compile time. (#356) -
Fix compile error in Apple Clang 12. (#360)
-
Add guards for redefined preprocessor variables.
-
Fix
uint64_t
totime_t
implicit conversion error in Clang 18. -
Update bundled
libfmt
tov10.1.1