Embed into other languages

[flip111]

Hey thanks for the library. Is there any possibility for a C version of this? Then it would be much easier to embed in other languages. A C-wrapper would also be a good compromise.

[gvinciguerra]

Hi @flip111, thank you for your interest!

Writing a C version would take quite some time, I'd rather go with wrappers from C++ to the target language. For example, for the Python wrapper, I used the pybind11 library and wrapped manually the main class pgm::PGMIndex. Do you have in mind any particular target language?

But if we want to scale, maybe we should look at automatic wrapper generators, such as SWIG and similar tools. But they have a learning curve too.

I'll leave this issue open. Maybe someone more experienced with these tools wants to help 😉

[flip111]

Going from C++ directly to the target language is not trivial due to ABI, that's why i was asking for a C wrapper. This doesn't need additional tools or libraries https://isocpp.org/wiki/faq/mixing-c-and-cpp

[darko20]

Although I agree that a C wrapper is preferable because most languages can fairly trivally adapt to C due to the standard ABI and there are a large number of possible languages, writing a C wrapper for a C++ program is not a trivial matter.

The C++ code uses language features that have no equivalent in C so there is a lot of work creating a C API that approximates those features. With things like C++ templates you have no hope and just have to only provide support for some basic instatiations, invoke some kind of C macro magic or resort to build-time hackery.

Having said that, tools like SWIG end up being a lot of work anyway and may not be worth the effort.

Personally I'd approach it by defining some minimalist C API for the entire library and build on that when and if required. I like this idea because it would make the library much more accessible to those who just want to exploit these powerful new algorithms.

[gvinciguerra]

I'll work on the C API in the upcoming week. Stay tuned 😉

[darko20]

Great, I'll be ready to test it when it's done.

I use LMDB right now and I'm very interested to see how it might perform in practice with PMG indexes replacing its B+Trees. LMDB's memory mapped approach is very fast and convenient and seems complementary to this.

[gvinciguerra]

I pushed a first version of the C interface and some examples of usage under /c-interface/examples :)

[gvinciguerra]

Here there's a re-implementation in Java!