QGIS reads shp with 11 character attribute header

[pigreco]

What is the bug or the crash?

A colleague created, using matlab, an empty shapefile (only with one line with NULL attributes) and with an attribute with a header with 11 characters (consistency)
QGIS reads it and also allows you to add features, while ArcGIS PRO reads it but truncates the header at the tenth character.

I'm reporting this because it could be a bug!

Steps to reproduce the issue

1. load the shapefile into QGIS
2. and look at the attribute properties

Versions

OSGeo4W QGIS 3.34.8

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

header11.zip

[agiudiceandrea]

The DBF format uses 11 bytes to store a 0 terminated string as a field name, so the field name is actually up to 10 characters. It looks like matlab doesn't add the 0 as ending byte, allowing to use all the 11 characters to store the field name.

I don't think this is actually an issue. Anyway the OGR provider sanitizes the fields name to 10 characters max when exporting to the ESRI Shapefile format.
Ping @rouault.

[rouault]

Yes, this is the intended behavior. Cf following constants of the shapelib library:

/* Shapelib read up to 11 characters, even if only 10 should normally be used */
#define XBASE_FLDNAME_LEN_READ 11
/* On writing, we limit to 10 characters */
#define XBASE_FLDNAME_LEN_WRITE 10

[aborruso]

Yes, this is the intended behavior.

Thank you @rouault . Do you know why this choice was made, to object only to one character and not, for example, 3?

[rouault]

Do you know why this choice was made, to object only to one character and not, for example, 3?

Answered by @agiudiceandrea in #58152 (comment)

[aborruso]

Answered by @agiudiceandrea in #58152 (comment)

Thank you.

We could introduce an additional check during reading. If the eleventh byte is a null character (\0), treat it as the standard terminator. Otherwise, raise an error indicating that the field name is not correctly terminated.
Therefore, it would be acceptable to have 11 characters only if the eleventh character is the \0 terminator.

What do you think about

[rouault]

Therefore, it would be acceptable to have 11 characters only if the eleventh character is the \0 termin

The field name is encoded on 11 bytes: not less, not more. So it is nominally 10 characters or less + a \0 terminator (or the non-standard situation of 11 characters without a \0 terminator)

[aborruso]

or the non-standard situation of 11 characters without a \0 terminator

Please don't hate me: and why was it decided here that the software must accept a nonstandard situation, since it is easily mapped as wrong?
I probably do not understand you, because I am not technically prepared. And I apologize for that.

[rouault]

and why was it decided here that the software must accept a nonstandard situation

I don't know. Probably the common logic: "be lax on reading, and strict on writing". This dates back to the initial revision of shapelib 29 years ago: OSGeo/shapelib@4af7724#diff-dad95b297ecd1ec47ff997ebfc36e4be7104f688b01954f704f0393a58ba5e09R543

[aborruso]

But if it is 12 characters is it always lax on reading? 🙃

Okay, I'll stop here. For me it is something that should be changed, because the case is left open and it fits, because it may be 10 or 11, for the reasons said, but it is not mapped correctly.

Thank you for your time

[pigreco]

Thanks @agiudiceandrea for the reply and thanks @rouault for your time and thanks @aborruso for the time you wasted (it's wasted at this point)
I would just like to understand why Nyall put the label Won't fix, what does this label mean?

[nyalldawson]

It means there'll be no action taken