why do we force the 8th number to be zero in NF4 data type?

[lidh15]

what if the quantiles are fully symmetric and no zero appears in the quantiles? will it be better or worse?

[matthewdouglas]

This is briefly discussed in the QLoRA paper. The main reason is that the value 0 is actually quite important, and as such an inexact representation would likely result in a less desirable result.

A problem for a symmetric k-bit quantization is that this approach does not have an exact representation of zero, which is an important property to quantize padding and other zero-valued elements with no error. To ensure a discrete zeropoint of 0 and to use all $2^k$ bits for a k-bit datatype, we create an asymmetric data type by estimating the quantiles $q_i$ of two ranges $q_i$: $2^{k−1}$ for the negative part and $2^{k−1} + 1$ for the positive part and then we unify these sets of $q_i$ and remove one of the two zeros that occurs in both sets. We term the resulting data type that has equal expected number of values in each quantization bin k-bit NormalFloat (NFk), since the data type is information-theoretically optimal for zero-centered normally distributed data.