Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AVRO-3967] Replace boost::format with fmt library #2832

Merged
merged 3 commits into from
Apr 29, 2024

Conversation

mkmkme
Copy link
Contributor

@mkmkme mkmkme commented Apr 2, 2024

AVRO-3967

This reduces the binary size quite significantly and doesn't require an additional object creation during exception throwing.

What is the purpose of the change

This change replaces boost::format with fmt library. It increases the readability and reduces the binary size quite significantly (on my machine I noticed avrogencpp being 27M with boost::format and 14M with fmt.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Documentation

  • Does this pull request introduce a new feature? no

@github-actions github-actions bot added C++ Pull Requests for C++ binding build labels Apr 2, 2024
@martin-g martin-g requested a review from thiru-mg April 3, 2024 08:02
@martin-g
Copy link
Member

martin-g commented Apr 3, 2024

This change is a trivial rework

It might be trivial to you but to me it looks like it needs a JIRA issue (for the changelog) :-)

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 3, 2024

Sure! Sorry I hadn't added it earlier. Will do this now

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 3, 2024

I see that fmt::fmt-header-only was not found on ARM Linux. Will try to see what version of fmt is being installed on the host.

In general, I would like to see it updated at some pointed as it seems quite a bit outdated tbh

@mkmkme mkmkme changed the title [C++] Replace boost::format with fmt library [AVRO-3967] Replace boost::format with fmt library Apr 3, 2024
@martin-g
Copy link
Member

martin-g commented Apr 3, 2024

The ARM64 builders run on Ubuntu 20.04 with GCC 9.3.0

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 4, 2024

@martin-g so the quick research shows that on Ubuntu 20.04 the version of fmt is too old (6.1.2 while the latest one is 10.2.0).

As an alternative of relying to the system dependencies we can use FetchContent in order to download fmt during the configuration step. The downside of this is that we'll have to build fmt as part of building avro-cpp (although the library is very small and doesn't have dependencies on its own).
The upside of that is that we won't be relying on the system package libfmt-dev and developers wouldn't need to install it themselves.

WDYT?

@martin-g
Copy link
Member

martin-g commented Apr 4, 2024

The ARM64 CI is a nice to have so we can ignore it! But it will fail again for any following C++ PR too!
I've asked Apache Infra about upgrading the ARM64 CI runners to a newer Ubuntu but it might take some time...
Yet another option is to use some PPA to install newer libfmt-dev in the ARM64 CI job.

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 4, 2024

Thanks for the help! Let's hear from Apache Infra...

I'll try to find PPA for fmtlib, but it seems it's not that common, especially for ARM64.

@martin-g
Copy link
Member

martin-g commented Apr 4, 2024

https://ubuntu.pkgs.org/22.04/ubuntu-universe-arm64/libfmt-dev_8.1.1+ds1-2_arm64.deb.html
it depends on https://ubuntu.pkgs.org/22.04/ubuntu-universe-arm64/libfmt8_8.1.1+ds1-2_arm64.deb.html

apt install http://ports.ubuntu.com/pool/universe/f/fmtlib/libfmt8_8.1.1+ds1-2_arm64.deb
apt install http://ports.ubuntu.com/pool/universe/f/fmtlib/libfmt-dev_8.1.1+ds1-2_arm64.deb

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 4, 2024

Thanks! Let's see if fmt 8 will do the trick

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 12, 2024

Checked locally with Ubuntu 20.04 docker image, and it doesn't work unfortunately :(

The following packages have unmet dependencies:
 libfmt8 : Depends: libc6 (>= 2.33) but 2.31-0ubuntu9.14 is to be installed
           Depends: libstdc++6 (>= 11) but 10.5.0-1ubuntu1~20.04 is to be installed
E: Unable to correct problems, you have held broken packages.

Alternatively, I can try to build it just on that Ubuntu in CI. Shouldn't take long.

@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 15, 2024

Could you trigger the CI before I fix the merge conflicts? Wanna see if taking the latest one with FetchContent would do the trick.

@martin-g
Copy link
Member

I can't! You need to resolve the conflicts first

mkmkme added 2 commits April 16, 2024 09:41
This reduces the binary size quite significantly and doesn't require an
addtitional object creation during exception throwing.
@mkmkme
Copy link
Contributor Author

mkmkme commented Apr 23, 2024

Hey, I've pushed one more commit. Could this be approved to trigger the CI?

@@ -22,6 +22,7 @@
#include <cstdint>
#include <cstring>
#include <memory>
#include <vector>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this include needed/used ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I currently have a limited access to my laptop (until Friday). I'll double check this by next week.
IIRC I got compile error without it because I removed include of boost/format from one of the headers. But I'll definitely double check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martin-g so yeah this include is needed, because there's usage of std::vector in the signature of snapshot() on line 177. Without it, compilation failed for me.

@martin-g martin-g merged commit 19501fc into apache:main Apr 29, 2024
4 checks passed
@martin-g
Copy link
Member

Thank you, @mkmkme !

@marlowa
Copy link
Contributor

marlowa commented Apr 29, 2024

The fmt library was added to cpp in cpp20. So this change would break projects on an older dialect, such as cpp17.

@florianhumblot
Copy link

florianhumblot commented Apr 29, 2024

The fmt library was added to cpp in cpp20. So this change would break projects on an older dialect, such as cpp17.

@marlowa libfmt is not the same as std::fmt, the https://github.com/fmtlib/fmt project is compatible with older c++ standards than c++20 where <format> and std::fmt were introduced :)

@mkmkme mkmkme deleted the mkmkme/use-fmt branch April 30, 2024 07:57
RanbirK pushed a commit to RanbirK/avro that referenced this pull request May 13, 2024
* Replace boost::format with fmt library

This reduces the binary size quite significantly and doesn't require an
addtitional object creation during exception throwing.

* cmake: get fmt with FetchContent

* cmake: always use fmt-header-only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build C++ Pull Requests for C++ binding
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants