Fix RNTuple writer after API break in ROOT

[tmadlener]

BEGINRELEASENOTES

• Fix the RNTupleWriter after ROOT has slightly changed the API for the RNTupleModel. Fixes RNTuple interface changed in ROOT #545

ENDRELEASENOTES

[tmadlener]

It looks like RNTuple is undergoing some API work on the ROOT side, so it probably makes sense fix things in one go, once that is done. Coming back to this, once that is the case.

[jmcarcell]

This isn't needed with ROOT 6.30.04 released a few days ago but the PRs that change the API are not in any tag yet

[tmadlener]

Yeah, they will presumably be part of 6.32. I am still waiting for the message from the root devs to see how this looks afterwards. Then we can see whether we can have both versions or whether we need to bump the minimal root version for RNTuple support.

[tmadlener]

Things should be more stable again in ROOT, so we could try to revive this.

[tmadlener]

I got things to work again for the writing side, where it was really mainly an API change as far as I can tell. I have kept backwards compatibility using pre-processor version checks for now.

On the reading side the API has changed a bit more drastically and things are not yet compiling, mainly because
RNTupleModel::GetDefaultEntry now returns a const REntry& instead of a Rentry*, so we cannot call BindRawPtr on it. Maybe @jblomer can shed some light on what we have to change conceptually. What we did until now was something along the lines of

auto reader = ROOT::Experimental::RNTupleReader{};
auto entry = reader.GetModel()->GetDefaultEntry();

auto buffers = BufferFactory::createBuffers("<coll-type>", schemaVersion);
dentry->CaptureValueUnsafe("<branch-name>", buffers.data);

reader.LoadEntry(entry);

buffers are podio::CollectionReadBuffers in this case, and buffers.data is a void*.

[jblomer]

Instead of using the default entry, you can create a dedicated entry with reader.GetModel().CreateEntry(). If you bind all the fields of your entry, then you can use reader.GetModel().CreateBareEntry(), which will not try to instantiate objects for the entry's fields.

On the entry, you can call entry->BindValue (requires an std::shared_ptr to the buffer) or entry->BindRawPtr()

[tmadlener]

Thanks for the pointers. Things are compiling again now, but there is an error in the test that reads back the rntuple file again. I have to figure out whether the problem is on the writing or on the reading side.

[tmadlener]

@jblomer do I have to do something more than CreateBareEntry and then binding all of the fields with BindRawPtr and then LoadEntry? It currently looks like everything is working on the writing side, but when I try to read back things, I effectively get only empty vectors. Those are as far as I can tell the empty ones we pass to BindRawPtr in the first place. Can (unique) ownership of these pointers play a role here?

[jblomer]

Apologies, this last comment slipped through!

What you describe sounds correct. You use CreateBareEntry and then bind all top-level fields with BindRawPtr.

I think what is missing is poassing that entry to LoadEntry() as a second parameter. Otherwise LoadEntry() fills the default entry.

[tmadlener]

No worries. The information would have been readily available from the documentation, if I had read that a bit more carefully. ;) This seems to do the trick. I am not entirely sure why but dev3 seems to be broken for other reasons at the moment, but I can build this locally against dev3 from last week.

I realized that we need to CreateEntry instead of CreateBareEntry, because we do not explicitly set all the fields in the entry.

Thanks again for your help.

[tmadlener]

Merging this since the RNTuple I/O is fixed with ROOT master. The remaining failing tests are unrelated and possibly due to other upstream ROOT changes. They are partially addressed in #575 and will be fully fixed in another PR.