feat: add simple tests for weights

[drbh]

This PR adds a few more server tests for loading weights

[danieldk]

Really nice to have these tests!

Column gptq/marlin/exl2 and column-packed gptq/marlin tests seem to be missing?

[danieldk]

I think the quantizers should probably be tested with the real data types to check that there are no accidental dtype conversions happening. So int32 for qweight, qzeros, and g_idx, and f16 for scales.

Maybe also real shapes? (e.g. currently qzeros does not line up with just having one group) Probably less important.

[drbh]

great points, I've updated the types in the latest changes but the values are still not fully realistic, happy to follow up in this or a future PR

[drbh]

also I'm not 100% sure if col_packed_exl2 is suppose to be supported or not? I was a bit confused about the expected weight names ("weights" vs "qweights")... is this intended? I'll update the test accordingly 🙂

[danieldk]

Great question! This path is missing in weights, I'll create a ticket and look it. I can update the test here once it's properly implemented.

[drbh]

updated to include more test (21)

python -m pytest -vvv server/tests/utils/test_weights.py

============================================ test session starts ============================================
platform linux -- Python 3.11.1, pytest-7.4.0, pluggy-1.5.0 -- /home/ubuntu/Projects/text-generation-inference/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/Projects/text-generation-inference/server
configfile: pyproject.toml
plugins: asyncio-0.23.7, anyio-4.3.0, syrupy-4.0.1
asyncio: mode=Mode.STRICT
collected 23 items

server/tests/utils/test_weights.py::test_weights PASSED                                               [  4%]
server/tests/utils/test_weights.py::test_get_tensor PASSED                                            [  8%]
server/tests/utils/test_weights.py::test_get_weights_col_packed PASSED                                [ 13%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_block_size PASSED                     [ 17%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_block_size_arr PASSED                 [ 21%]
server/tests/utils/test_weights.py::test_get_multi_weights_col PASSED                                 [ 26%]
server/tests/utils/test_weights.py::test_get_multi_weights_row PASSED                                 [ 30%]
server/tests/utils/test_weights.py::test_get_weights_col_awq PASSED                                   [ 34%]
server/tests/utils/test_weights.py::test_get_weights_col_gtpq PASSED                                  [ 39%]
server/tests/utils/test_weights.py::test_get_weights_col_exl2 PASSED                                  [ 43%]
server/tests/utils/test_weights.py::test_get_weights_col_marlin PASSED                                [ 47%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_awq PASSED                            [ 52%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_exl2 SKIPPED (Review expected
functionality)                                                                                        [ 56%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_gptq PASSED                           [ 60%]
server/tests/utils/test_weights.py::test_get_weights_col_packed_marlin PASSED                         [ 65%]
server/tests/utils/test_weights.py::test_get_multi_weights_col_awq PASSED                             [ 69%]
server/tests/utils/test_weights.py::test_get_multi_weights_col_exl2 PASSED                            [ 73%]
server/tests/utils/test_weights.py::test_get_multi_weights_col_gptq PASSED                            [ 78%]
server/tests/utils/test_weights.py::test_get_multi_weights_col_marlin PASSED                          [ 82%]
server/tests/utils/test_weights.py::test_get_multi_weights_row_awq PASSED                             [ 86%]
server/tests/utils/test_weights.py::test_get_multi_weights_row_exl2 PASSED                            [ 91%]
server/tests/utils/test_weights.py::test_get_multi_weights_row_gptq PASSED                            [ 95%]
server/tests/utils/test_weights.py::test_get_multi_weights_row_marlin PASSED                          [100%]

------------------------------------------ snapshot report summary ------------------------------------------

======================================= 22 passed, 1 skipped in 3.74s =======================================