read<std::string>() returns all but the last character of a string

[nettels]

New to HighFive, I am trying to read some of the photon hdf5 example data from this site: http://photon-hdf5.github.io/

My code writes GROU instead of GROUP to the console.

HighFive::File file("test.hdf5", HighFive::File::ReadOnly);
std::cout << file.getAttribute("CLASS").read<std::string>() << "\n";

HDFView shows me for this attribute:

Name: CLASS
Type: String, length=5, padding=H5t_STR_NULLTERM, cset=H5T_CSET_UTF8
ArrayType: Scalar
Value: GROUP

Why is read<std::string>() not reading the last character while HDFView is doing it ?
I am useing HDF5 1.14.4 and the latest HighFive on Windows.
Thank you for your help!

[1uc]

Thank you for reporting the issue and providing the file that demonstrates the issue.

The cause is that HighFive has the assumption that a fixed length string, with padding H5T_STR_NULLTERM is in fact a null-terminated string. The provided example consists of a 5-byte, fixed length string with H5T_STR_NULLTERM padding. It uses all 5 bytes to store the string "GROUP", making it not null-terminated. The missing letter is the fifth one (which according to the assumption used in HighFive would have to be \0).

A quick search through the HDF5 manual only states:

H5T_STR_NULLTERM: Null terminate (as C does)

The format specification is slightly more wordy about it:

Null Terminate: A zero byte marks the end of the string and is guaranteed to be present after converting a long string to a short string. When converting a short string to a long string the value is padded with additional null characters as necessary.

-- Bits: String Bit Field Description https://support.hdfgroup.org/documentation/hdf5/latest/_f_m_t3.html

OTOH, the gold standard h5py reads it back with the P:

import h5py

with h5py.File("20dA_20dT_200mMgCl2.h5", "r") as h5:
    print(h5.attrs["CLASS"].decode())

[nettels]

This seems to be a tricky one. Thank you very much for your fast response!
Will you change HighFive such that it does the same as h5py? And in the meantime, do you have workaround for me? (I am really new to all of this...)

[1uc]

If you can use a branch, then you can use 1uc/not-so-null-terminated from this PR #1056.

[1uc]

If not, you can try to to use read_raw(ptr, dtype). Essentially, you need a pointer ptr to sufficient memory and the datatype dtype. You can probably get the datatype by doing something like file.get{Attribute,DataSet}(...).getDataType().

If you magically know that the string is N bytes long (excluding the null), you can maybe do:

std::string str(N, ' ');
read_raw(str.data(), dtype);

(or read into an std::vector and copy.)

If not you can get the number of bytes through dtype.getSize(), but check carefully that it actually does what it should:
https://portal.hdfgroup.org/documentation/hdf5/latest/group___h5_t.html#ga1b971589cd7a86f3e84affdee455564e
which sounds promising.

[nettels]

I used the branch. And it works! Thank you. I am very grateful that you solved the issue so rapidly!

[1uc]

Thanks for reporting and trying it out.