"Max Size" field in SHOW COLUMN STATS comes back as nulls

[wesm]

In [1]: import impala.dbapi as db

In [2]: con = db.connect()

In [3]: cur = con.cursor()

In [4]: cur.exe
cur.execute           cur.execute_async     cur.executemany       cur.execution_failed  

In [4]: cur.execute('SHOW COLUMN STATS ibis_testing.tpch_lineitem')

In [5]: cur.fetchall()
Out[5]: 
[('l_orderkey', 'BIGINT', 1511050, -1, None, 8.0),
 ('l_partkey', 'BIGINT', 272762, -1, None, 8.0),
 ('l_suppkey', 'BIGINT', 10380, -1, None, 8.0),
 ('l_linenumber', 'INT', 7, -1, None, 4.0),
 ('l_quantity', 'DECIMAL(12,2)', 51, -1, None, 8.0),
 ('l_extendedprice', 'DECIMAL(12,2)', 1011732, -1, None, 8.0),
 ('l_discount', 'DECIMAL(12,2)', 11, -1, None, 8.0),
 ('l_tax', 'DECIMAL(12,2)', 9, -1, None, 8.0),
 ('l_returnflag', 'STRING', 3, -1, None, 1.0),
 ('l_linestatus', 'STRING', 2, -1, None, 1.0),
 ('l_shipdate', 'STRING', 2332, -1, None, 10.0),
 ('l_commitdate', 'STRING', 2113, -1, None, 10.0),
 ('l_receiptdate', 'STRING', 2351, -1, None, 10.0),
 ('l_shipinstruct', 'STRING', 4, -1, None, 11.998638153076172),
 ('l_shipmode', 'STRING', 7, -1, None, 4.285304546356201),
 ('l_comment', 'STRING', 4648153, -1, None, 26.494169235229492)]

by contrast, in the impala shell

[localhost:21000] > show column stats ibis_testing.tpch_lineitem;
Query: show column stats ibis_testing.tpch_lineitem
+-----------------+---------------+------------------+--------+----------+-------------------+
| Column          | Type          | #Distinct Values | #Nulls | Max Size | Avg Size          |
+-----------------+---------------+------------------+--------+----------+-------------------+
| l_orderkey      | BIGINT        | 1511050          | -1     | 8        | 8                 |
| l_partkey       | BIGINT        | 272762           | -1     | 8        | 8                 |
| l_suppkey       | BIGINT        | 10380            | -1     | 8        | 8                 |
| l_linenumber    | INT           | 7                | -1     | 4        | 4                 |
| l_quantity      | DECIMAL(12,2) | 51               | -1     | 8        | 8                 |
| l_extendedprice | DECIMAL(12,2) | 1011732          | -1     | 8        | 8                 |
| l_discount      | DECIMAL(12,2) | 11               | -1     | 8        | 8                 |
| l_tax           | DECIMAL(12,2) | 9                | -1     | 8        | 8                 |
| l_returnflag    | STRING        | 3                | -1     | 1        | 1                 |
| l_linestatus    | STRING        | 2                | -1     | 1        | 1                 |
| l_shipdate      | STRING        | 2332             | -1     | 10       | 10                |
| l_commitdate    | STRING        | 2113             | -1     | 10       | 10                |
| l_receiptdate   | STRING        | 2351             | -1     | 10       | 10                |
| l_shipinstruct  | STRING        | 4                | -1     | 17       | 11.99863815307617 |
| l_shipmode      | STRING        | 7                | -1     | 7        | 4.285304546356201 |
| l_comment       | STRING        | 4648153          | -1     | 43       | 26.49416923522949 |
+-----------------+---------------+------------------+--------+----------+-------------------+
Fetched 16 row(s) in 0.02s

@caseyching @laserson any ideas?

[caseyching]

I'm pretty sure this is a bug in Impala's HS2 implementation. Alex Leblang filed a bug for this a while ago -- https://issues.cloudera.org/browse/IMPALA-1988. If this is a problem for you we can bump the priority.

[wesm]

Yes, it would be good to investigate this if the intent is to migrate completely to HS2