h.project() several times slower than Xarray.sum()

[reckoner]

I have a 13 dimensional dataset and I have noticed that although h = bh.Histogram(...) works beautifully, doing h.project(3,4) is three of four times slower than doing the exact same calculation in Xarray,

xa.sum([0,1,2,5,6,7,8,9,10,11,12])

Am I doing something wrong here or is this to be expected? I tried on both a Windows and Linux Ubuntu backend. The result is exactly the same in both cases. It's just the bh is much slower.

[andrzejnovak]

@reckoner are you on the latest version? I had a similar issue that went away recently (complicated slice on ~8ish dims)

[reckoner]

I have version 1.2.1

[henryiii]

Are you sure it's the same calculation? In boost-histogram, this is making a new histogram and computing the flow bins in the projections. In xarray, you are probably ignoring the flow bins. Even if you do include them, what xarray is doing is probably a little simpler, and it's not handling the axes. I think we avoid making an extra copy here in 1.2+, since the operations you are providing are all supported in a single fusion in Boost.Histogram (C++), but might be worth checking just to make sure.

Note to self: I wonder if we can improve the conversion to xarray to include nice indices based on the axes, at least in hist...

[reckoner]

I'm not using any flow bins. The resulting Numpy output array is the same in both cases.

[henryiii]

Do you have them turned off, or are you just avoiding them? I'd try underflow=False, overflow=False in all of your axes constructors; this will remove them from the histogram entirely.

[reckoner]

Init signature:
bh.axis.StrCategory(
    categories: Iterable[str],
    *,
    metadata: Any = None,
    growth: bool = False,
    __dict__: Optional[Dict[str, Any]] = None,
)

Is that the correct constructor? I don't see the keyword arguments that you mentioned.

[henryiii]

StrCategory doesn't come with a flow bin off version. Only Regular, Integer, and Variable do. It also only has one flow bin instead of two, so it's a bit less harmful to memory. Are all 13 axes Categorical?

[reckoner]

Yes. All categorical variables.