[RFC] About modernizing development tools

[derselbst]

Language Standard: C89

Being bound to C89 is getting more and more annoying. The most recent issue was the introduction of fluid_long_long_t. Now, MinGW complains about issues like that:

D:\a\1\s\src\sfloader\fluid_sfont.c:54:34: warning: too many arguments for format [-Wformat-extra-args]
             FLUID_LOG(FLUID_ERR, "EOF while attempting to read %lld bytes", count);
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

C89 doesn't know about %lld. But C99 does. Ofc, you can always find ways to make it work: Use non-standard %I64d or cast down to unsigned long. Which decreases maintainability and increases technical debts. And it's no fun either.

The effort for checking if certain C99 math functions or headers exist would also become redundant when upgrading to a newer C standard. Resulting in faster compilation on Windows.

Glib Dependency

Glib is a huge dependency. And it's becoming a pain when cross-compiling for e.g. Android.

Nowadays, the standard provides portable means that could replace glib. @chirs241097 has done some nice work to replace glib-threads with C++11 threading. His patch is basically only missing try / catch clauses around the C++ code and then would be instantly ready to use for fluidsynth. On the other hand, using C++ as OSAL in a project that is mainly written in C feels somewhat unclean. Using C11 threads feels better. Unfortunately MS has just started to support them in their recent VS 16.8, so, it's not a realistic option still doesn't support threading.

Yes, I know. Even once fluidsynth would be free of glib, glib wouldn't simply disappear. We still have libinstpatch which is using much more of glib's internals with much less abstraction than fluidsynth does. It might be even easier to adopt the DLS loading code from libinstpatch, rather than getting libinstpatch glib-free.

And dropping glib would also mean to drop support for ancient OSes like Win9x, OS/2, Solaris... WinXP might still work...

Auto-generated loookup tables

Having compile-time const lookup tables is really nice. The way we produce them is not. Again, we are using a sophisticated CMake-workaround to access the host compiler. Recently, someone has pointed out that Meson provides easier access to the host compiler. I don't want to access the host compiler at all. I don't want to generate any tables myself. I want to solve the problem rather than curing the symptom. Why can't we just use a C++11/14 constexpr lookup table? There is a nice project that provides constexpr C++ math functions, so no need to implement our own Newton iteration as we thought before.

fluid_long_long_t

Define it as int64 rather than long long, see related discussion in #1057.

CMake

EDIT: CMake modernization has been incorporated into fluidsynth 2.3.0!

Currently, the minimum required cmake version is 3.1.0. Bumping the minimum cmake version would allow for

• better OpenMP support,
• better clang-tidy support,
• CMAKE_MSVC_RUNTIME_LIBRARY.

Again: yes, to some extent it is possible to achieve this without bumping the minimum version, by injecting a bunch of if ( CMAKE_VERSION ) into the build system. Doesn't makes things easier...

Unrelated to the cmake version there are a few more TODOs to refurbish the build system a little:

• Using the more common GnuInstallDirs (requested already a few years ago, cannot find the issue anymore)
• Avoiding global functions (link_directories, include_libraries, etc)
• Remove enable-debug
• Rethink enable-fpe-check and enable-trap-on-fpe
• exported targets

---

The goals listed here are some huge milestones, which no one should expect to be achievable within a few weeks. I would be very happy, if we find a compromise that could be applied to fluidsynth 2.3.0 or so.

How do we find the best compromise between modernizing the tools and retaining backward compatibility? How to figure out which OSes are used out there? An answer to the latter question is essential for deciding on a technical implementation of the points above.

[This disussion will stay for a while. Please upvote it so it's attracting people's attention.]

[derselbst]

Oh no, not boost. Using Boost Threads requires compiling boost. Then we could also stay with glib.

PS: C++11 is available since VS2015, so getting a working build for WinXP wouldn't be a problem. Older OSes would need testing (and probably a statically linked CRT ;)

[MrSapps]

boost is even more bloated than glib :)

[sykhro]

dlib might be what you're looking for. Works all the way back to Windows 95, it's crossplatform, it might also replace usages of openmp since it has some parallelization utilities

[derselbst]

dlib might be what you're looking for

(Ab)using a library as OSAL while it was designed for machine learning doesn't seems right to me.

[pedrolcl]

A list of Fluidsynth versions, mostly Linux distributions who are shipping packages:
https://repology.org/projects/?search=fluidsynth

[mawe42]

@komh seems to maintain an OS/2 fork of Fluidsynth here: https://github.com/komh/fluidsynth-os2

Compiled versions seem to be published here: https://ecsoft2.org/fluidsynth

[derselbst]

(again, if they seem maintainable).

That's so vague and subjective. If you asked me I would consider any wrapper as unmaintainable. That's why we have standards nowadays.

I think that would be the main advantage: people could remove glib but don't need so many intrusive changes that their forks become unmaintainable and stale quickly

Yes. But this would only solve the problem for people who don't want to use DLS. But I think it's important to provide a solution that works as a whole for everyone. Hence my solution would be:

1. Add native DLS support to fluidsynth (alternatively: port libinstpatch to C++11, much more work though)
2. Migrate fluidsynth to C++11.

C++11 would require @jjceresa to use VS2013 to compile fluidsynth on WindowsXP. Alternatively MinGW could be used. And yet alternatively, the build artifacts from the CI pipeline could be used.

The only real downside I see is that @carlo-bramini would no longer be able to compile and run it on Win9x. Maybe it would still be possible via MinGW. I don't know.

[sykhro]

Small note: if you want constexpr arrays for DSP tables like mentioned in the OP, you'll need C++14, as C++11's std::array doesn't have anything constexpr

[mawe42]

That's so vague and subjective.

It is, as many other decisions we often make. Like which features we accept as contributions and which we reject, for example.

Add native DLS support to fluidsynth (alternatively: port libinstpatch to C++11, much more work though)

Why is native DLS support so important? FluidSynth doesn't need to be compiled with glib to be able to use a glib-libinstpatch, does it? Not arguing against including native DLS support, but I don't understand why it's so high on the list. I personally would solve the glib "problem" first, then address DLS.

Migrate fluidsynth to C++11.

You mean the whole codebase? Or just C++11 for a platform interface?

And as you write above:

But I think it's important to provide a solution that works as a whole for everyone.

Who would we loose with C++11? Would Solaris and OS/2 still work? (I honestly don't know)

[mawe42]

Just one more food for thought: staying with glib also has some larger drawbacks with regards to older platforms like OS/2: we are locked into very old glib versions if we want to keep OS/2 support, as OS/2 support has been removed in glib 2.39.2 (8 years ago).

[derselbst]

Small note: if you want constexpr arrays for DSP tables like mentioned in the OP, you'll need C++14, as C++11's std::array doesn't have anything constexpr

Noted, thanks. I wouldn't mind populating a constexpr C array. I know this can become ugly. We'll have to see.

FluidSynth doesn't need to be compiled with glib to be able to use a glib-libinstpatch, does it? Not arguing against including native DLS support, but I don't understand why it's so high on the list

I see it from a user perspective: A user wants to use fluidsynth without glib dependency. And the same user wants fluidsynth to support DLS files. Think of Android. Ergo, you would still have to compile glib. So why spending any effort on removing glib from fluidsynth? IMO, it would be in vain.

You mean the whole codebase? Or just C++11 for a platform interface?

As mentioned here this would need to be discussed separately.

But I think it's important to provide a solution that works as a whole for everyone.
Who would we loose with C++11? Would Solaris and OS/2 still work?

Perhaps I should have said "for everyone who (actively) uses fluidsynth". komh's comment above suggests that C++11 should work on OS/2. I haven't heard of anyone using fluidsynth on Solaris, but apparently C++11 seems to be available on Solaris, too.

[pedrolcl]

Language Standard

10 years ago, the MSVC compiler didn't accept many C99 features which were available in GCC, so people coding on Linux often contributed code that didn't compile in MSVC. That's why additional strict diagnostics were introduced when configuring for GCC (or compatible) compilers like -Wdeclaration-after-statement and -Werror=implicit-function-declaration. Nowadays, modern MSVC understands all C11 mandatory features and also C++11, so the justification is no longer valid. MSVC is not C99 compliant, though, because several mandatory features in C99 are optional in C11, like VLA. So, the TestVLA.cmake macro is still needed, and there is a glib replacement for it in fluidsynth_priv. Glib is hard to kill.

By the way, whatever the language(s) standard adopted, it should be prominently documented in CONTRIBUTING.md in addition to the wiki and other developer documentation.

CMake

In addition to Tom's list, my favorite request would be exported targets.

Why? because right now the only way for a client project using CMake to configure/search Fluidsynth as a dependency is using pkg-config. There is an option enable-pkgconfig when compiling fluidsynth, but we don't give a chance to our customers?

With modern CMake versions, you can already import pkg-config targets into your project , and Fluidsynth installs pkg-config support (fluidsynth.pc) but it would be nice to also provide support for find_package(Fluidsynth) in config mode.

[pedrolcl]

Multithreading and atomic primitives and types are optional C11 features.

[derselbst]

MSVC's C11 support is still very disappointing. Not only that they're missing most optional features. E.g. to use the <tgmath.h> you need the latest Windows 10 SDK Insider Preview Build 20206 for which you have to sign up for their preview program. Even if that would be available today, I know of at least one contributor who still runs WinXP, so we would still need some backward-compatible-fallback-mechanism.

I would really like to use C99. But it seems like MS skipped that standard completely (tgmath.h is a C99 feature after all).

Seems like C++11 is our best guess.

[sykhro]

Seems like C++11 is our best guess.

Would the plan be to switch everything to C++ or use it like in the patch?

[derselbst]

In short: I'm not sure.

Long story: The decision of whether or not to go with C++11 should be made in a separate discussion with pros and cons + involving the WinXP community, to see whether it's actually worth the effort.

And if we decide to do so we could either switch everything (=98%) to C++. I mean, not just by renaming the .c files to .cpp, but real C++, like

• using STL containers rather than our custom fluid_list, fluid_hash_table types,
• having an abstract Audio driver base class and derive specific drivers from it (to finally get of the very last "pointer-cast increases alignment" warnings),
• having constexpr lookup tables (to get rid of that gentable cmake dance),
• getting rid of glib because C++ STL will be the OSAL (how to deal with libinstpatch?).

Ofc, this would be a huge chunk of work.

Or we could only cherry-pick useful C++11 pieces and add them to our code. In this case I would be thinking of

• having constexpr lookup tables,
• perhaps a bit of C++ STL to remove or at least lower the glib dependency (again: how to deal with libinstpatch?),
• taking huge care that we are not causing undefined behaviour by intermixing the legacy C world with C++ (e.g. accessing unions works fine in C, in C++ it might be undef.b.; throwing exceptions into the legacy C world is undef.b., etc.)

I think the final decision will be made when discussing C++ and see for how many people it would break compilation on their legacy OS.

[pedrolcl]

The decision of whether or not to go with C++11 should be made in a separate discussion

I agree. This discussion does not progress on the original issues, and will likely stall if it becomes more dispersed.

[chirs241097]

Regarding my threading patch...

It was a terrible patch. It was a thing only because back then I wasn't able to figure out a "painless" way to build glib for Windows and Android. Now that we have stuff like msys2 and mxe, building for Windows is no longer that a mess to deal with. There are also folks providing Android binaries. So the patch isn't really needed anymore. I'm still occasionally maintaining it because I have the legacy setup still in use.

And the patch itself has a lot of issues. It breaks LADSPA because I didn't do anything about GModule. It has a completely broken g_shell_parse_argv implementation. Exceptions aren't handled as mentioned above. I just got the patched version "barely working" for my application.

I also personally don't like the idea of mixing C++ code into a code base that's mostly C. I have considered using C11 threading, but no major libc implementation has it back then (CRT on Windows still doesn't have it to this day afaik). A better alternative in my opinion is just using pthread (minimal dependency, won't randomly introduce a file in another language to the code base, and has decent support for legacy platforms as well). Atomic types in C11 can still be used (if we ignore msvc for a moment). GCC comes with C11 atomic types since 4.9, and mingw 4.9 definitely can target Windows 9x (I'm not sure whether atomic types would work with mingw of that age though).

If support for certain (very) legacy platforms is an absolute must, and there's a real need to remove glib as a dependency, adding multiple threading implementations and let the user decide which to use at compile time is potentially an option. (I'd say using glib only for concurrency is wasteful though.)

And I'm definitely a backer for adding exported targets to the CMake project file.

[MrSapps]

needing msys2 to build on windows certainly isn't "painless"

[mawe42]

Going with C11 for atomics and pthread for threading sounds like quite a good compromise to me. Then just special-case MSVC (which gets special casing in many places already, even with glib) and we're done. Sounds so simple... what am I missing? :-)

[derselbst]

Threading and atomics are optional for C11. The future will bring many more library implementations which claim to be C11 compliant, while not implementing those optional, yet crucial features. So, this would be an uncertain way to go.

[derselbst]

I would like to summarize the recent comments: When talking about removing glib, it should be possible to compile fluidsynth with all features without having to compile glib at all. I.e. glib is currently used by fluidsynth and libinstpatch (which provides DLS support). Thus the answer cannot be to move solely fluidsynth to another dlib, xlib, whateverlib. Doing so would simply add yet another dependency and glib would still be required through libinstpatch.

To truly remove glib, native DLS support should be added to fluidsynth (alternatively libinstpatch should be glib-free, very difficult). Then, glib should be replaced with a standardized API. Since many of C11's features we require are optional, I'm proposing to move to C++11 instead. Also, using C++11 maintains good compatibilty to old platforms (Solaris, OS/2, WinXP should work fine, only Win9x support is questionable).

This is a big effort and will take time. And before going into the implementation details, my questions to all stakeholders are:

• Do you support moving to C++11?
• Would it solve the glib dependency issue for you?
• Would this break anything I'm not seeing?

@paulsapps @sykhro @komh @pedrolcl @d3nwah @midi-pascal @carlo-bramini @jjceresa @chirs241097

[MrSapps]

I am for any option that removes the need for glib 😉 porting to C++11 does seem like quite a big change - I'm not sure it would cover all of the needed stuff though. the g_file_status or whatever I don't think exists in C++11 stdlib?

[derselbst]

I do wonder though if that layer would really work without any platform specific #ifdefs.

I don't think "zero-#ifdefs" should be the goal. Greatly reducing the number of #ifdefs in the code is already a big win. E.g. one issue that comes to my mind: C++ doesn't provide a portable way to specify a thread's priority. I.e. we would probably still need our fluid_thread_self_set_prio() dance. There may be a few other bits like that I'm currently not seeing. But IMO, that's only a minor drawback.

g_file_status or whatever I don't think exists in C++11 stdlib?

Correct, unfortunately. For this we could use an implementation of C++17 std::filesystem, something like this.

And while we are on it: A similar replacement for gmodule's library loading should be found.

[MrSapps]

Btw what is the dynamic code loading for? It seems kinda odd to have in a sound font synthesizer lib?

[sykhro]

Btw what is the dynamic code loading for? It seems kinda odd to have in a sound font synthesizer lib?

Audio effects via LADSPA.

As for moving to C++11, I'd happily contribute.

[MrSapps]

I would probably contribute also - btw that LADSPA is probably always gonna be a pain because some platforms don't even have a concept of shared objects/dlls.

[derselbst]

I'm not sure. Yes, the list of features is amazing. But: It the project in a major state? It's version is zero.zero.four released 3 years ago. It has a pretty trivial PR which is open for 2 years. I'm not convinced that is the horse we should set on to ride into the future.

Edit: it doesn't provide recursive mutex.

[mawe42]

I agree. Shame though... looked really promising.

[mawe42]

I would like to reiterate my proposal for a less intrusive approach with regards to glib-replacement and the modernization in general. Let me first explain where I'm coming from. Sorry, this is going to be a little longer, but I really wanted to get this off my chest. Skip to "Proposal" below if you don't want to read my whole sermon.

Motivation

glib ist great :-)

In my opinion, glib does a great job at abstracting away (most of) the hardware differences for FluidSynth. It's very actively maintained and extremely well tested due to it being the basis for GTK and Gnome, especially the low-level bits that we mostly use. And it has served us very well for the last few years, we have experience in using it.

What does glib provide us?

Please extend if I'm missing something, but to my knowledge, we use the following bits from glib:

• threading
• atomics
• command-line argument parsing
• file operations
• dynamic module loading

Reasons for removing glib

So far I've seen the following (good) reasons why we should remove glib or at least make it replaceable:

• glib is too large a dependency
• glib is difficult or impossible to compile on the target platform (or target configuration, e.g. static lib on Windows)

Who wants to remove glib?

Given the reasons above, I see users on the following platforms wanting to remove glib:

• Android
• Windows
• Other platforms not supported by glib (Playstation, XBox?)
• Other, more special use-cases like running FS on a ESP32

C or C++?

FluidSynth is written in C. The API design, the internal data structures, data flow, the whole architecture... everything is heavily influenced by the possibilities and limits of the C language.

Personally I feel comfortable in C. But in C++ I have only basic knowledge of the syntax. I know next to nothing about design patterns, data-flows, conventions for API design or data structures. And I have exactly zero experience writing C++ myself. I love learning new stuff, so I would be willing to start learning C++... but it would take me while until I would be confident enough to actually help reviewing patches again, let alone contributing new code.

But what really drives me to oppose a move to C++ is that I think such a move would only make sense if we are not just changing syntax and replacing some data structures, but actually make good use of that new language. That includes taking advantage of the great tools, patterns and data structures that C++ probably offers (so I've heard). Changing the architecture of FluidSynth to fit into the C++ world.

Only then would a move to C++ provide enough benefits to be worthwhile, in my opinion. But that would also probably mean rewriting FluidSynth from scratch. And that in turn would not only be a huge amount of work, it's also a lot of work for an unclear gain. And it would be a project with huge risks... just look at the jack/jack2 mess as an example of what could happen.

Yes, using C++ just for a platform interface would be much less intrusive. But even C++11 doesn't offer everything we need as built-ins, so that platform interface would contain a fair amount of C++ code. And I think that is the wrong direction for a C project.

Proposal

So, with that all said, I would like to propose to think about this differently. Rather than us providing a solution to the glib problem, let our users experiment with good alternatives to glib.

Restructure the FluidSynth code so that all calls and Macros used by our platform interface (glib) can be easily replaced. A little bit like what @paulsapps has proposed in #901, but with documentation, and a clear explanations on how users can write their own wrapper.

Then let that simmer for a few months, maybe a year. Watch what interfaces people are writing and how they are using them. Notice how well they maintain their interfaces, how much work it is to keep up with upstream. Then, when the dust has settled, decide if any of those wrappers should make it into FluidSynth.

Some of the benefits of this approach:

• No need for a new programming language in our codebase
• Even more flexibility for users wanting to target "obscure" platforms than a
  C++11 (or C11 with optionals) solution would offer
• But most importantly: No need for us to second-guess what our users want: we
  let them decide for themselves. Find the real-world use-cases.

How exactly this platform interface and glib-replacement could work on a technical level would obviously need to be discussed.

Ok, that's it. Sorry again for this wall of text, but I needed to get this out of my system. :-)

[derselbst]

we don't need to move away from glib. We could just make it easier for people to replace glib with whatever they prefer

All the forks I've seen have one motivation. Many discussions and requests of the past have the same reason: glib. Over and over again. And upstream's reaction has always been: "Well, no we keep it. But you can replace it. It's simple. It's only 3 files. Good luck." And even though it's simple (I do believe it's already simple!) people keep creating forks. glib was added in Nov. 2009. We are a living project and I think it's time to say good bye. Without feature regressions and without breaking changes, I'm convinced this could be done with C++11.
We could also leave it as is. But I'm not sure what this tells about this (reoccurring) discussion and our project in general.

Again, what's wrong with fluid_list and fluid_hash?

Technical depts. fluid_hash is a copy-paste of gmodule's hashmap. I guess nobody has ever had a close look at this code. It's probably fine. We also thought the sfloader code is fine and now we've had a CVE. That driver construct is a C-ish implementation of inheritance with an abstract base class (pretty similar to the fluid_defsfont, fluid_sfont and fluid_instpatch_font, btw.). Yes it works, because we've addressed alignment issues in the past. But it still leads to false-positive warnings like this one.

[mawe42]

And even though it's simple (I do believe it's already simple!) people keep creating forks.

I think that's partly because while it's simple, it feels like a ugly hack. And rather than creating ugly hacks, people prefer a "clean" fork and rewrite. Maybe by providing an official and clean way to replace glib, it would have the effect that instead of forking and rewriting or ripping out stuff, it would lead to cleaner forks with just two additional files. But I'm just guessing here.

And I agree, glib has been a constant issue over and over again. So leaving it as it is now is not really an option.

Technical depts. fluid_hash is a copy-paste of gmodule's hashmap. I guess nobody has ever had a close look at this code.

Ok, that's a motivation I can understand. However... we could also just have a good look at the <700 lines of code in fluid_hash.* and < 300 lines of code in fluid_list.* and be done with it. :)

You seem to be fairly decided on the matter, so I'll will drop out of this discussion. It's important that FluidSynth has an active maintainer and someone who wants to bring things forward. And I don't want to risk that by being the guy who always argues against stuff. If you do decide to go ahead with the C++ rewrite, then I'll watch with interest and will be there to support code I've contributed, of course. All in good spirits, but maybe it's time for me to move on... and maybe come back, once I've learned C++ :)

[pedrolcl]

glib was added in Nov. 2009.

It was A few months before that. I was there, but didn't protest because there was already code copied & pasted from glib into the fluidsynth sources, including bugs, and nobody was caring about it. Sorry for that. The alternative of depending on the original libraries was preferable for me at the time. And this discussion feels like a déjà vu.

My interest in fluidsynth is related to my projects, as a library dependency. I am a contributor to Qsynth, and prepare the Windows and macOS packages for it. Also, my drumstick libs, dmidiplayer and VMPK depend on fluidsynth. All these projects are C++/Qt based, and glib is a nuisance. I would be glad if we can get rid of it, without introducing a worse alternative.

C++11 as a replacement, albeit weird, would be acceptable for me. Would it be a solution to the nuisance? For me, and probably others like me, the ideal situation would be to have binaries of the fluidsynth releases, ready to use in packaging. The alternative is to compile fluidsynth and its dependencies, which may be a nuisance. For the latest Windows releases, the binaries distributed in Github didn't work for me, because the C++ runtime. I had to compile from sources...

For building in Windows there is the MS vcpkg, which provides a comfortable way to build fluidsynth using the MSVC++ compiler (but the MinGW alternative didn't work the last time I've tried). There is also msys2 if you prefer the MinGW compiler, but it may not provide the latest releases. Each C++ compiler requires its own C++ runtime, one for MSVC and two or more for MinGW, depending on the exception model: sjlj, dwarf and seh. It was really bad luck that the precompiled Qt binaries require a dwarf runtime, while the fluidsynth binaries require a sjlj one, so my only option was to compile fluidsynth myself.

[MrSapps]

I think a nice clean documented way of replacing glib would probably keep most people happy? I would probably replace glib with SDL2 in my game projects. The DAW style stuff is mostly using C++ and Qt? And I guess any C ones are happy to stay with glib because they're probably already using it.

If a move to C++ happened I'd assume the original C interface would stay so the public facing stuff is the same? I don't think you'd need to go as far as a rewrite for everything but if C++11 is going to mean more platform specific stuff being added I dunno if its really worth it?

[derselbst]

the original C interface would stay so the public facing stuff is the same?

Absolutely. API stability is a must.

but if C++11 is going to mean more platform specific stuff being added I dunno if its really worth it?

The idea of moving to C++11 is that we can remove platform specific stuff and other quirky-workarounds of the past. We probably cannot remove everything, e.g. thread-priorities, loading dynamic libs.

documented way of replacing glib would probably keep most people happy

I have no idea how this documentation should look like. I'd probably only say: "Re-implement the functions and macros in fluid_sys and fluidynth_priv.h".

The mindset of keeping everything as-is, is a bit like: "Good morning, you wanna drink coffee? Then go to the kitchen and make you some."
Don't get me wrong, it's completely fine for me: I don't drink any coffee. But frankly, the more I read you don't actually need a coffee-machine, the more drops my motivation to build a C++11-coffee-machine.

[carlo-bramini]

Although FluidSynth depends on GLib, I have to say that it is not an impossible task to make the synthesizer working without it. It tooks less than 20 minutes to change fluid_sys.c, fluid_sys.h and fluidsynth_priv.h when I compiled my first try on MSVC 6.0, which is C90.
I had also implemented a little stuff for getting rid of g_shell_parse_argv inside fluid_cmd.c, but it was also easy.

I did several tests and I was able to run FluidSynth natively on:

• posix (CYGWIN without GLib)
• Windows (32bit and 64bit)
• Windows CE (ARMv5, but it should be retested after multichannel addition to waveout driver)
• Embedded ARM Cortex M3 and M4 (with NXP LPC1788 and LPC4078 microcontrollers).
• MS-DOS DJGPP (SoundBlaster and MIDI UART).

So, I must say that the current code is highly configurable... at the moment, it's true that the code needs GLib but it is not too difficult to adjust it for your favourite system or if you want something different.

Talking about my version for Windows, later I did more changes to the code:

1. I implemented some functions for open, close, read, seek, etc... which solved lot of issues:

• file names with "strange" characters could not be open, now UTF8 encoded strings are accepted.
• can work with files larger than 4GB
• resolves internally links and shortcuts.
• can be used also with network sockets.

2. if you run FluidSynth on Windows 64bit and you try to use a socket, it will brutally crash because the hack used on the handle. In addition to point (1), I added two functions fluid_socket_free_istream() and fluid_socket_free_istream() for cleaning what the fluid_socket_get_istream() and fluid_socket_get_ostream() do.
3. C90 does not support variable length arrays, so in the past I suggested to include support for alloca() and _alloca(), that do exactly the same thing, but unfortunately that patch was not accepted:
   15b31be
   But it still exists in my sources, because I need it with MSVC 6.0.

When doing my DJGPP and Embedded ARM ports, I had added also 8bit sample output. It is true that you could easily obtain the same thing by converting a 16bit buffer to 8 bit, but this allowed to save some bandwidth especially on the second port.

More or less, this is what I did in my FluidSynth build: I created an abstraction layer that works with posix, winapi, or none (no threads and no atomics).

However, we should note that:

1. pthread is a standard, registered as IEEE Std 1003.1.
2. Windows API are also a standard, registered as ECMA_234.

but GLib is not: today, a function exists and tomorrow it could be deprecated or it could disappear.
Instead, the above two options are rock solid and they won't change.
Using GLib provides some benefits, but in my opinion it would be worth to consider also this aspect.

[derselbst]

Thanks Carlo. However, I don't yet fully understand your motivation to remove glib. I do understand that you want to remove it for your embedded ARM use-case. But why removing it for the Windows, DOS, posix world as well?

I understand that glib is no standardized API. That's one of the reasons I suggested moving to C++11 instead. Would that be a feasible solution for your use-cases?

[carlo-bramini]

Hello,

MY CONCLUSION, if you want to avoid the complete read of this reply:
it does not matter if you use C++11, GLib, or others, I would keep the system functions "isolated" by the synthesizer, in a similar manner it is done now, because by providing your own version of 'fluid_sys.h' and 'fluidsynth_priv.h' you can make working on the machine and the platform you want.

Thanks Carlo. However, I don't yet fully understand your motivation to remove glib. I do understand that you want to remove it for your embedded ARM use-case. But why removing it for the Windows, DOS, posix world as well?

well, if I can write my opinion, keeping GLib is not a problem at technical level. However, probably not for posix but at least in Windows, the situation is a bit different because FluidSynth depends on GLib, but GLib has also a good number of extra dependencies. So, at the end, for running the synthesizer, you must provided a number of DLLs that you do not use at all and you could avoid, at least in theory.
Just to make an example, SCUMMVM uses FluidSynth 1.0.9 to avoid these dependencies:

https://wiki.scummvm.org/index.php?title=Compiling_ScummVM/MinGW

However, with my ports I demonstrated myself that it is not impossible to make working Fluidsynth without GLib, this is just what I wanted to say. ;)

I understand that glib is no standardized API. That's one of the reasons I suggested moving to C++11 instead. Would that be a feasible solution for your use-cases?

About the threading support, this is a good idea but, since GLib thread API match very close to pthread and by looking how the code of FluidSynth is written at the time of writing, perhaps it would be worth to evaluate some aspects:

• GLib functions are very similar to pthread, so perhaps just doing a #define can do the work.
• MinGW-w64 implements std::thread by using winpthread. Since FluidSynth already links with the (big, argh!!!) stdc++, providing also winphread DLL seems unavoidable when using this compiler.
• MinGW does not support std:thread at all, at least the last time I used it.
• For Windows, thread support by using directly winapi can be also done, if someone wants (I did it).

Perhaps... we could expand the concept of "driver" also for the thread/mutex/atomic stuff, by letting CMake to detect which one to build at compile time, it's just an idea...

I have never used C++11, so I would like to ask a question: since MinGW-w64 uses winpthread for implementing std::thread (but some solutions with native winapi also exist on github), what can you do with C++11 that you cannot do with pthreads (or winapi if it runs on Windows)?
(please excuse my ignorance...)

[sykhro]

what can you do with C++11 that you cannot do with pthreads

Nothing, pretty much. std::thread is meant to be a portable abstraction, but it comes with some pitfalls compared to lower level APIs like pthreads (e.g. no way of setting priority). The way to do that is to use the std::thread::native_handle() which (optionally) returns a pointer to whatever that std::thread implementation uses internally. At that point it becomes a template hell to manage doing the same thing over different thread implementations, defeating the whole point of cross-platform threads imo. boost::thread doesn't have this issue but well, being boost the dependency size is even bigger than glib

[derselbst]

I would keep the system functions "isolated" by the synthesizer, in a similar manner it is done now, because by providing your own version of 'fluid_sys.h' and 'fluidsynth_priv.h' you can make working on the machine and the platform you want.

The reason for keeping that functionality in separate header(s) is because it's a custom API. In other words, it's a plain-old-C-ish manner of handling things. When using C++11 (or even C11) we would program against a standardized API. And if one wants to make fluidsynth run on any platform, one would need to provide minimal implementations of that standardized API on that platform (instead of fluidsynth's custom API). So, there would no longer be a need for keeping it separately.

Perhaps... we could expand the concept of "driver" also for the thread/mutex/atomic stuff, by letting CMake to detect which one to build at compile time, it's just an idea...

This in turn would mean to maintain low-level implementations for different platforms. I'm not willing to do this. This is what C++11 is meant to abstract in a portable manner (as already mentioned by @sykhro).

However, I wouldn't call it "template hell". Thread-priority could be cleanly #ifdefed out, just as we do at the moment. Or one could probably use function overloading as well. Yes, the native_handle type is implementation defined. But in the end you should always get the operating system's type.

[derselbst]

Ok, here's a propsal how to proceed regarding C++: Towards the end of the year I will port the sequencer to C++11. We already have lot's of C++ in the sequencer, so getting the rest ported seems like the natural way to go. I hope that by doing so we get a better understanding of what we can expect from such a step, what effort it takes, and what approach is best (PIMPL?, exception and error handling), etc. Also it's gonna be interesting how to set up the C API from the C++ sequencer class: Ideally, one would come up with a clever template or macro which generates the extern "C" functions during compilation and just forwards it to the C++ methods. All this will happen on a separate branch, so it shouldn't disturb anyone. Once ready, we'll review and decide how to proceed.

In the meantime, I welcome anybody to work on the CMake modernization points listed above. During this work it will probably become necessary to raise the minimum required CMake version. I would say that 3.13.3 is a reasonable minimum version, as it's the last version to officially support WinXP (and at least one of our contributors is a WinXP guy).

[jjceresa]

Ok, here's a proposal how to proceed regarding C++...(and at least one of our contributors is a WinXP guy).

The reasons I am using Win XP are to maintain compatibility (hardware, software) with my legacy silent hardware.

• XP is the only Windows version that can be installed on my hardware. It is installed on compact flash card disks, requires low RAM, and it boot fast enough (20 s).

Also, using C++11 maintains good compatibilty to old platforms (Solaris, OS/2, WinXP should work fine, only Win9x support is questionable).

Concerning WinXP, MSVC 2010 is the only one that can be installed on XP. Unfortunately MSVC 2010 doesn't support all C++11 features (for example, constexpr C++ isn't supported). That means that future fluidsynth with C++11 code build will probably fail when compiled on Win XP.

Do you support moving to C++11?

My applications (calling fluidsynth API and Windows GUI API) are actually pure C code. I am not interested by moving to C++ for calling fluidsynth library.

Would it solve the glib dependency issue for you?

As explained by Marcus (@mawe42 ) and Carlo (@carlo-bramini), I am rather favorable to define an C API that allows glib functions substitution by user custom functions (the interface selection could be a cmake option).

I would say that even I am not fan of C++, I understand Tom's motivations (as a maintainer). However, if any future version of fluidsynth shouldn't support compilation on Win XP, then this should be clearly documented in the Wiki.

[derselbst]

Thanks for the information.

I am not interested by moving to C++ for calling fluidsynth library.

Fluidsynth's API will stay C, so nothing changes for users.

That means that future fluidsynth with C++11 code build will probably fail when compiled on Win XP.

Compiling it yourself by using MinGW should work. And you have the option to consume the binaries produced by the CI builds.

[jjceresa]

To truly remove glib, native DLS support should be added to fluidsynth (alternatively libinstpatch should be glib-free, very difficult).

libinstpatch internal is complex and hugely based on GObject, so libinstpatch cannot be glib-free.
libinstpath is an option in fluidsynth. For users that doesn't need DLS, having fluidsynth without glib could be a valid option.

[derselbst]

That's the point: User who do need DLS would still need glib - think of Android. Either glib will be removed from the buildchain entirely or it will stay in both.

[Ghabry]

imo requiring glib for DLS is an acceptable compromise right now.

At least I only need SF2 support.

[Ghabry]

Was asked to comment on this in #942

I'm in favor of using C++11 here. When you put all the C++11 code in a cpp-file the external interface can still be C-only. You can also invent a rule that only this "glib-replacement" is allowed to use C++11 and everything else must stay C. There could be also a ifdef here that still uses glib to make the WinXP users happy.

The problem for me is that glib is hard to compile on some platforms and bloated (too huge increase in binary size).

I'm a maintainer of https://github.com/EasyRPG/Player . Thats a game engine compatible to RPG Maker 2000 and a lot of games use MIDI for there music so we need good support for this. The engine is available for Windows, Linux, macOS, Android, Wii*, 3DS*, Switch*, PS Vita* and Emscripten (WASM). We currently are about to move from FluidLite (https://github.com/divideconcept/FluidLite) to FluidSynth because we got many reports about incorrect instruments. To achieve this I had to patch out glib because it is close to impossible to compile. Most of these homebrew SDK are only "kinda POSIX" and certain functions are broken.

*: Homebrew only

About Windows: We ship a single binary (static linking) everywhere and static linking of glib is broken on Windows since 8 years! https://gitlab.gnome.org/GNOME/glib/-/issues/692

Also std::thread does not work on Emscripten due to technical reasons, but this is not something you should worry about (it will still compile and not all parts of fluidsynth use threads).

---

Note that my glib patching is extremely rudimentary: I simply removed the entire thread/atomic/mutex code because we only use FluidSynth as a synthesizer (load soundfont, send midi messages to it, sample into a buffer) on a single thread. MIDI processing and so on is done by us directly because RPG Maker 2000 has some non-standard features (e.g. looping and skipping of silence) that we must handle.

[derselbst]

Thanks Pedro! I just had a closer look at it, there're some comments in the PR. Some general points:

That means: if you need to use the library distributed as fluidsynth-2.2.x you need to request something like find_package(FluidSynth 3.0).

I would tend to use the project version instead. I understand that this would collide with the COMPATIBILITY SameMajorVersion. Perhaps we could just agree, that we'll never have a breaking change upon a minor bump again. This goes hand in hand with your idea of a single version identifier. In this case, we would probably have the use the library version for everything. I'm afraid that such a change would have to wait until we have another major version bump. On the other hand, I quite like the separate version identifiers. Maybe someone else has a comment on this.

FluidSynthConfig.cmake is generated from a template, [...] it could be enhanced, for instance, with the detection of the required dependencies and with some variables exposing the built-in capabilities

Ok, but I would prefer to wait for contirubtors to provide a concrete use-case for it.

[pedrolcl]

As an example for importing/detecting fluidsynth directly with cmake (requires the PR #943 ) I've updated one of my gists: https://gist.github.com/pedrolcl/5837161d8d5926f219dab3159a48bd8c

[sykhro]

Just a question, does calling it libfluidsynth work with all generators? I'm afraid that with some of them you might end up with a library called liblibfluidsynth

[pedrolcl]

@sykhro: If your question is about the exported/imported target called FluidSynth::libfluidsynth it is not creating any new library (on disk). The CMake target name has always been libfluidsynth. Exporting the target name does not change the file name of the DLL or shared object on the file system. Importing the target in a downstream project does not condition the file name of the library on the file system either. This is a cmake only thing.

Also, you may use the pkg-config method as usual, with a cmake build system or with another one.

[pedrolcl]

Modernizing CONTRIBUTING.md

My proposal is to add a "Languages" section to CONTRIBUTING.md just to declare the standard language dialects to use across the project. Initially, on the cmake_modernization branch, and also later on the wiki. Something like this:

Languages

This project uses the following language dialects across the project:

• C language: C89/C90 standard
  By default, all sources use this language. This is also the official API language

• C++ language: C++11 standard
  Some features may be internally implemented in this language

• CMake language: version 3.13
  The build system scripts are written in this language

[derselbst]

I'm not sure. The language dialects are already specified and enforced by cmake. Not respecting this should cause the build to break anyway. Having an additional documentation of those, however, will cause some redundancy. - What do others think?

[pedrolcl]

Motivation of my proposal: imagine an advanced user/developer that just discovered the project, and wants to know if he could contribute. He knows some programming tools already... Do you expect that he should write some code in his favorite language dialect, and try to integrate/build it to know if it is acceptable within this project? Or perhaps would be better to look first at some documentation like CONTRIBUTING.md and the Wiki to find which languages are used, before getting his hands really dirty?

[derselbst]

Sequencer C++ Port available

An MVP-like example of a C++ port of fluidsynth is now available, see #1011. Feedback is highly welcome!

[sykhro]

Thanks, I was wondering if this was actually going to happen. Will check it out!

[derselbst]

As of recently, glib can be linked statically on Windows. This means we've lost one argument for removing glib.

As described initially, removing glib is a huge amount of work. And the feedback for #1011 was less than I expected, which makes me think that it's not worth spending even more time on this in the future.

If one day somebody would come up with a full-featured C++11 port, which doesn't even require glib for DLS support, I would take it. Do bare in mind though, that there are nasty pit-falls that will break the build, e.g. https://stackoverflow.com/a/17986785 .

[DominusExult]

For what it's worth, I flat out "ported" an earlier fork that did away with GLib. By no means perfect, of course doesn't fix the GLib dependency of libinstpatch (no DLS support) and probably some dubios changes (e.g. JACK audio driver changes).
https://github.com/DominusExult/fluidsynth-sans-glib/tree/sans_glib
I successfully tested this only with Exult under macOS (working SF2 and SF3 support) and iOS (only SF2 support using SDL2 for audio output).

My main gripe with Glib is that it is hard to bundle and likely to just pull in more and more dependencies in the future which seems an overkill to me for the limited use FluidSynth gets out of it. But I'm certain you have a much better overview and know the pros and cons much much better than me :)

[Ghabry]

interesting. i have my own no-glib branch here: https://github.com/Ghabry/fluidsynth/tree/no-glib

But is also only tested for one application (EasyRPG Player) we have ports for across various platforms (Windows, Android, Emscripten, homebrew consoles like Wii etc.) so likely won't work for anything else. Getting rid of glib made all the porting possible without frustration of building glib :).

(also my patchset simply stubs all threading stuff as we don't need it)

[phamconganh]

@DominusExult would be appreciated if you could create a PR with the no-glib option in CMakeLists.txt and fix the code to hide glib if the no-glib option is on

[DominusExult]

I‘ll keep it in mind. So far my fork just ripped everything Glib out and didn‘t make no-glib optional.

[ebraminio]

HarfBuzz https://github.com/harfbuzz/harfbuzz is an important text related library used in different places including browsers, has an optional glib dependency which most embedders including Chrome and Firefox just don't use it, it's written in C++ and actually needs a C++11 compiler to compile but doesn't use C++'s standard libraries (STL) runtime dependency but uses C++11's atomics but with a compiler definition as described in CONFIG.md can go even without threading.

So my suggestion is to remove or make glib dependency optional for the ease of embedders, use C11 or C++11 atomic operations but don't switch to C++ or at least don't use C++'s standard libraries at all for the rest of the project now that you didn't need them all these years. And maybe provide a compiler configuration just like HB_NO_MT for non multithreaded environment if you want to support C11 builds without atomics. Or maybe just adopt this file from HarfBuzz.

My own use case:

I wanted to build fluidsynth for web using Emscripten, while compiling glib2 for Emscripten emscripten-core/emscripten#11066 isn't impossible https://gist.github.com/VitoVan/92ba4f2b68fec31cda803119686295e5 but it's hard and it most likely bloats resulting .wasm file and in web projects reducing .wasm result is beneficial so I wanted also to have fluidsynth without or with optional glib2 dependency. And here is result of my work based on Ghabry's work https://github.com/ebraminio/fluidsynth-no-glib-meson-wasm which builds with meson setup wasmbuild --cross-file emscripten-build.ini && ninja -Cwasmbuild which just build fluidsynth in WebAssembly, I'm yet to integrate to my open source project though.

[derselbst]

make the default +glib +DLS support and when the cmake switch NO_GLIB is used then disable DLS support

So, you're suggesting I should maintain two OSAL's, i.e. one with glib and one without. This will increase complexity and maintenance effort in the source code and buildsystem. I won't do that, sry. You're free to apply any no-glib patch out there and compile fluidsynth yourself and the result would be the same.

[DominusExult]

You are right, maintaining both is too draining. Bad advice.

If I understand correctly FS doesn't need glib itself when supporting DLS, right?
Glib is just a dependency of an optional dependency?

So drop glib from FS and let whover wants DLS support in their app worry about glib. That's how things work (and why there is this discussion: FS is an optional dependency that brings in the glib dependency).

But for FS itself it is apparent that glib is not really needed and could easily be dropped. It's just dependency creep (https://shutdownhook.com/2021/02/10/a-journey-to-dependency-with-csv-files/?amp=1) which might just over time add more and more dependency bloat (as it's out of your hand what else glib might pull in the future). And troublesome for some platforms.

I'm happily maintaining my simple fork. A bit sad about having lost DLS support but I can live with that and just leave the note that DLS (and I think sf3?) is not supported unless you build Exult yourself.

[derselbst]

If I understand correctly FS doesn't need glib itself when supporting DLS, right?

Correct.

So drop glib from FS and let whoever wants DLS support in their app worry about glib. That's how things work

That is a quick and dirty solution, which I am not going to support. Currently, the (simplified) dependency graph looks like this:

graph TD;
    fluidsynth-->glib;
    fluidsynth-->libinstpatch;
    libinstpatch-->glib;

Loading

You're suggesting to change it to

graph TD;
    fluidsynth-->OSAL;
    fluidsynth-->libinstpatch;
    libinstpatch-->glib;

Loading

For a user who wants / needs DLS support, how is this better? glib does have a few advantages, e.g. it can run on ancient platforms (e.g. Win9x - and yes, we have or had users who consider this to be a use-case). Imagine we do remove glib from fluidsynth itself, but users still have to compile glib because they need it for DLS support. They would feel kind of fooled.

Again, I'm not arguing against removing glib. I'm just demonstrating that doing it properly is a bunch of effort. IMO, the only way forward would be:

1. A solution for DLS support in fluidsynth (either modernize libinstpatch, or integrate DLS support into fluidsynth natively).
2. Remove glib from all CI pipelines and see which breaks.
3. Fix the CI by replacing glib with something else. My preference would be C++11, as it would also give us constexpr math, allowing us to get rid of the gentables at some time. And it should work on WinXP, so that JJC won't be cross with me.
4. See that all the CI pipelines succeed and glib isn't needed in any pipeline anymore.

Over the years I have people seen joining and leaving this discussion. But I haven't seen anyone willing to tackle this effort.

PS: Why do I care so much about DLS support and libinstpatch? Because libinstpatch was created by elementgreen, a former maintainer of fluidsynth. And bringing libinstpatch and fluidsynth together just makes sense for this project as they aim for the same target and wonderfully complement each other.

[DominusExult]

So drop glib from FS and let whoever wants DLS support in their app worry about glib. That's how things work

That is a quick and dirty solution, which I am not going to support. Currently, the (simplified) dependency graph looks like this:

For a user who wants / needs DLS support, how is this better? glib does have a few advantages, e.g. it can run on ancient platforms (e.g. Win9x - and yes, we have or had users who consider this to be a use-case). Imagine we do remove glib from fluidsynth itself, but users still have to compile glib because they need it for DLS support. They would feel kind of fooled.

For a user who wants DSL nothing changes, neither for better or worse. Currently, before even compiling FS, the libinstpatch dependency must be fulfilled. So you compile libinstpatch first, which pulls in glib and when it comes to compiling FS it doesn't even matter anymore whether FS needs glib. No one will feel fooled, it's the usual story with dependencies. BUT if there is a problem with glib on their platform they might feel annoyed (same s people are annoyed by FS' glib dependency).
But if FS would do away with glib they can still use FS even if they can't use DLS.

Again, I'm not arguing against removing glib. I'm just demonstrating that doing it properly is a bunch of effort.

I really think that removing glib FS is not the "quick and dirty" solution and pretty low effort one AND the way forward to get out of the chicken/egg situation (Libinstpatch: why remove glib, FS needs it anyway!, FS: why remove glib, libinstpatch needs it anyway!).

IMO, the only way forward would be:

1. A solution for DLS support in fluidsynth (either modernize libinstpatch, or integrate DLS support into fluidsynth natively).
2. Remove glib from all CI pipelines and see which breaks.
3. Fix the CI by replacing glib with something else. My preference would be C++11, as it would also give us constexpr math, allowing us to get rid of the gentables at some time. And it should work on WinXP, so that JJC won't be cross with me.
4. See that all the CI pipelines succeed and glib isn't needed in any pipeline anymore.

Over the years I have people seen joining and leaving this discussion. But I haven't seen anyone willing to tackle this effort.

Yes, that's annoying and I have to confess I can't be much help as well. I'm good at making existing code run, or rather clobber the build system into working and fixing minor problems, but I can't code enough to take this on.
My fork is based on another one's fork and I only maintain it to be up to dateISH with main FS.

PS: Why do I care so much about DLS support and libinstpatch? Because libinstpatch was created by elementgreen, a former maintainer of fluidsynth. And bringing libinstpatch and fluidsynth together just makes sense for this project as they aim for the same target and wonderfully complement each other.

I agree completely, I'm not happy about having to ditch DLS with my fork. Snipping off functionality is definitely not good.