Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supporting also vector<pod> in the RootTreeWriter #2645

Merged
merged 2 commits into from
Dec 9, 2019

Conversation

matthiasrichter
Copy link
Collaborator

@matthiasrichter matthiasrichter commented Dec 2, 2019

This is a first sketch for the functionality requested by @shahor02

A couple of things need to be polished and the unit test has to be completed.

It also depends on the correct implementation of `InputRecord::get, WIP in PR #2617

EDIT:

@shahor02
Copy link
Collaborator

shahor02 commented Dec 2, 2019

@matthiasrichter thanks! If you think the final implementation (including # #2525) will take more than a few days, let me know please, then I will use the trick with int wrapped to struct + dictionary.

Require the buffer to be of serialization Method 'None' unless a char
type is used which will allow to access the raw buffer in all cases.
Implementing a specialization for vector<messageable> retrieving the input
opject as span and copying the data to the storage variable which is then
stored in the branch. Implementation for Vectors of non messageable types
with ROOT dictionary is unchanged.

TODO: Need to investigate if the binary branch specialization can be extended
to any POD pointer and realized without copy. Add read-back check for binary
branch test.
@matthiasrichter matthiasrichter changed the title [WIP] Supporting also vector<pod> in the RootTreeWriter Supporting also vector<pod> in the RootTreeWriter Dec 9, 2019
@matthiasrichter
Copy link
Collaborator Author

With the latest update, the PR implements now the support for vectors of messageable types with and without ROOT dictionary. The latter need to be non serialized at the input. We can extend later to support both non and ROOT serialized messages as input for these types.

Storage in a ROOT tree always involves a copy. While ROOT serialized objects are deserialized and the object pointer is used for writing data to branch, the data of the non serialized messages needs to be copied to a storage object for writing. Maybe this can be avoided, but this is a task for later investigation.

@matthiasrichter matthiasrichter merged commit 2383383 into AliceO2Group:dev Dec 9, 2019
@matthiasrichter matthiasrichter deleted the dev-dpl-util branch December 9, 2019 18:53
@shahor02
Copy link
Collaborator

Hi @matthiasrichter

Looks like this PR screws up the POD subbranches, the updated o2-test-dplutils-RootTreeWriter produces _0.XX branches:

testtree->Print()
******************************************************************************
*Tree    :testtree  : testtree                                               *
*Entries :        1 : Total =           11401 bytes  File Size =       3446 *
*        :          : Tree compression factor = 1.00                       *
******************************************************************************
*Br    0 :intbranch : intbranch/I                                            *
*Entries :        1 : Total  Size=        585 bytes  File Size  =         84 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br    1 :containerbranch_0 : Int_t containerbranch_0_                       *
*Entries :        1 : Total  Size=       2482 bytes  File Size  =        104 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br    2 :containerbranch_0.mMember : Int_t mMember[containerbranch_0_]      *
*Entries :        1 : Total  Size=        781 bytes  File Size  =        112 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br    3 :containerbranch_0.mSecret : UInt_t mSecret[containerbranch_0_]     *
*Entries :        1 : Total  Size=        781 bytes  File Size  =        112 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br    4 :containerbranch_1 : Int_t containerbranch_1_                       *
*Entries :        1 : Total  Size=       2482 bytes  File Size  =        104 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
...
*Branch :trivvecbranch *
*Entries :        1 : BranchElement (see below)                              *
*............................................................................*
*Br   10 :_0        : Int_t _0_                                              *
*Entries :        1 : Total  Size=       2916 bytes  File Size  =         89 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br   11 :_0.mX     : UInt_t mX[_0_]                                         *
*Entries :        1 : Total  Size=        655 bytes  File Size  =         96 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br   12 :_0.mY     : UInt_t mY[_0_]                                         *
*Entries :        1 : Total  Size=        655 bytes  File Size  =         96 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................*
*Br   13 :_0.mSecret : UInt_t mSecret[_0_]                                   *
*Entries :        1 : Total  Size=        680 bytes  File Size  =        101 *
*Baskets :        1 : Basket Size=      32000 bytes Compression=   1.00     *
*............................................................................* 

@matthiasrichter
Copy link
Collaborator Author

@shahor02, yes, unfortunately I have been using the wrong pointer for the branch data. ROOT IO seems to be very flexible because it was handling this. Although not silently, I should have noticed the error message but it was hidden in the lengthy log.

Please try with the fix in PR #2687. For the TPC reco workflow I have verified it.

@wiechula
Copy link
Collaborator

I can confirm that this is fixed. Thanks!

rhaake added a commit to rhaake/AliceO2 that referenced this pull request Dec 12, 2019
rhaake added a commit to rhaake/AliceO2 that referenced this pull request Dec 12, 2019
shahor02 pushed a commit that referenced this pull request Dec 13, 2019
* [WIP][EMCAL-505] Porting EMCAL clusterizer to O2 framework

* [WIP][EMCAL-505] Requested improvements for pull request #2525

* [WIP][EMCAL-505] Implementation for clusterizer in workflow

* Fixing typo in cmake file

* Removed some orphans that accidentally CMakeLists

* Added fixes in run_clus_emcal macro + implemented BranchDefinition in workflow that currently breaks

* Applying workaround for related to PR #2645

* Removing workaround related to PR #2645, fix code checker error, apply clang-format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants