Skip to content

Commit

Permalink
Expand docs for Hive schema evolution type conversions
Browse files Browse the repository at this point in the history
  • Loading branch information
blvckcoffee authored and mosabua committed Oct 24, 2023
1 parent 4fb13d7 commit ee8682c
Showing 1 changed file with 35 additions and 8 deletions.
43 changes: 35 additions & 8 deletions docs/src/main/sphinx/connector/hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -628,14 +628,41 @@ Hive directly, most operations can be performed using Trino.

#### Schema evolution

Hive allows the partitions in a table to have a different schema than the
table. This occurs when the column types of a table are changed after
partitions already exist (that use the original column types). The Hive
connector supports this by allowing the same conversions as Hive:

- `VARCHAR` to and from `TINYINT`, `SMALLINT`, `INTEGER` and `BIGINT`
- `REAL` to `DOUBLE`
- Widening conversions for integers, such as `TINYINT` to `SMALLINT`
Hive table partitions can differ from the current table schema. This occurs when
the data types of columns of a table are changed from the data types of columns
of preexisting partitions. The Hive connector supports this schema evolution by
allowing the same conversions as Hive. The following table lists possible data
type conversions.

:::{list-table} Hive schema evolution type conversion
:widths: 25, 75
:header-rows: 1

* - Data type
- Converted to
* - `VARCHAR`
- `TINYINT`, `SMALLINT`, `INTEGER`, `BIGINT`, `TIMESTAMP`, `DATE`, as well as
narrowing conversions for `VARCHAR`
* - `CHAR`
- narrowing conversions for `CHAR`
* - `TINYINT`
- `VARCHAR`, `SMALLINT`, `INTEGER`, `BIGINT`
* - `SMALLINT`
- `VARCHAR`, `INTEGER`, `BIGINT`
* - `INTEGER`
- `VARCHAR`, `BIGINT`
* - `BIGINT`
- `VARCHAR`
* - `REAL`
- `DOUBLE`, `DECIMAL`
* - `DOUBLE`
- `FLOAT`, `DECIMAL`
* - `DECIMAL`
- `DOUBLE`, `REAL`, `VARCHAR`, `TINYINT`, `SMALLINT`, `INTEGER`, `BIGINT`, as
well as narrowing and widening conversions for `DECIMAL`
* - `TIMESTAMP`
- `VARCHAR`
:::

Any conversion failure results in null, which is the same behavior
as Hive. For example, converting the string `'foo'` to a number,
Expand Down

0 comments on commit ee8682c

Please sign in to comment.