-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
9853386
commit 1bc7b73
Showing
6 changed files
with
215 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
# Catalog management properties | ||
|
||
(prop-catalog-management)= | ||
## `catalog.management` | ||
|
||
- **Type:** [](prop-type-string) | ||
- **Allowed values:** `static`, `dynamic` | ||
- **Default value:** `static` | ||
|
||
When set to `static`, Trino reads catalog property files and configures | ||
available catalogs only on server startup. When set to `dynamic`, catalog | ||
configuration can also be managed using [](/sql/create-catalog) and | ||
[](/sql/drop-catalog). New worker nodes joining the cluster receive the current | ||
catalog configuration from the coordinator node. | ||
|
||
:::{warning} | ||
This feature is experimental only. Because of the security implications the | ||
syntax might change and be backward incompatible. | ||
::: | ||
|
||
:::{warning} | ||
Some connectors are known not to release all resources when dropping a catalog | ||
that uses such connector. This includes all connectors that can read data from | ||
HDFS, S3, GCS, or Azure, which are [](/connector/hive), | ||
[](/connector/iceberg), [](/connector/delta-lake), and | ||
[](/connector/hudi). | ||
::: | ||
|
||
:::{warning} | ||
The complete `CREATE CATALOG` query is logged, and visible in the [Web | ||
UI](/admin/web-interface). This includes any sensitive properties, like | ||
passwords and other credentials. See [](/security/secrets). | ||
::: | ||
|
||
## `catalog.prune.update-interval` | ||
|
||
- **Type:** [](prop-type-duration) | ||
- **Default value:** `5s` | ||
- **Minimum value:** `1s` | ||
|
||
Requires [](prop-catalog-management) to be set to `dynamic`. Interval for | ||
pruning dropped catalogs. Dropping a catalog does not interrupt any running | ||
queries that use it, but makes it unavailable to any new queries. | ||
|
||
(prop-catalog-store)= | ||
## `catalog.store` | ||
|
||
- **Type:** [](prop-type-string) | ||
- **Allowed values:** `file`, `memory` | ||
- **Default value:** `file` | ||
|
||
Requires [](prop-catalog-management) to be set to `dynamic`. When set to | ||
`file`, creating and dropping catalogs using the SQL commands adds and removes | ||
catalog property files on the coordinator node. Trino server process requires | ||
write access in the catalog configuration directory. Existing catalog files are | ||
also read on the coordinator startup. When set to `memory`, catalog | ||
configuration is only managed in memory, and any existing files are ignored on | ||
startup. | ||
|
||
## `catalog.config-dir` | ||
|
||
- **Type:** [](prop-type-string) | ||
- **Default value:** `etc/catalog/` | ||
|
||
Requires [](prop-catalog-management) to be set to `static` or | ||
[](prop-catalog-store) to be set to `file`. The directory with catalog property | ||
files. | ||
|
||
## `catalog.disabled-catalogs` | ||
|
||
- **Type:** [](prop-type-string) | ||
|
||
Requires [](prop-catalog-management) to be set to `static` or | ||
[](prop-catalog-store) to be set to `file`. Comma-separated list of catalogs to | ||
ignore on startup. | ||
|
||
## `catalog.read-only` | ||
|
||
- **Type:** [](prop-type-string) | ||
- **Default value:** `false` | ||
|
||
Requires [](prop-catalog-store) to be set to `file`. If true, existing catalog | ||
property files cannot be removed with `DROP CATALOG`, and now new catalog files | ||
can be written with identical names with `CREATE CATALOG`. As a result, a | ||
coordinator restart resets the known catalogs to the existing files only. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
# CREATE CATALOG | ||
|
||
## Synopsis | ||
|
||
```text | ||
CREATE CATALOG | ||
catalog_name | ||
USING connector_name | ||
[ WITH ( property_name = expression [, ...] ) ] | ||
``` | ||
|
||
## Description | ||
|
||
Create a new catalog using the specified connector. | ||
|
||
The optional `WITH` clause is used to set properties on the newly created | ||
catalog. Property names can be double quoted, which is required if they contain | ||
special characters, like `-`. Refer to the [connectors | ||
documentation](/connector) to learn about all available properties. All | ||
property values must be varchars (single quoted), including numbers and boolean | ||
values. | ||
|
||
The query fails in the following circumstances: | ||
|
||
* A required property is missing. | ||
* An invalid property is set, for example there is a typo in the property name, | ||
or a property name from a different connector was used. | ||
* The value of the property is invalid, for example a numeric value is out of | ||
range, or a string value doesn't match the required pattern. | ||
* The value references an environmental variable that is not set on the | ||
coordinator node. | ||
|
||
:::{warning} | ||
The complete `CREATE CATALOG` query is logged, and visible in the [Web | ||
UI](/admin/web-interface). This includes any sensitive properties, like | ||
passwords and other credentials. See [](/security/secrets). | ||
::: | ||
|
||
:::{note} | ||
This command requires the [catalog management type](/admin/properties-catalog) | ||
to be set to `dynamic`. | ||
::: | ||
|
||
## Examples | ||
|
||
Create a new catalog called `tpch` using the [](/connector/tpch): | ||
|
||
```sql | ||
CREATE CATALOG tpch USING tpch; | ||
``` | ||
|
||
Create a new catalog called `brain` using the [](/connector/memory): | ||
|
||
```sql | ||
CREATE CATALOG brain USING memory | ||
WITH ("memory.max-data-per-node" = '128MB'); | ||
``` | ||
|
||
Notice that the connector property contains dashes (`-`) and needs to quoted | ||
using a double quote (`"`). The value `128MB` is quoted using single quotes, | ||
because it is a string literal. | ||
|
||
Create a new catalog called `example` using the [](/connector/postgresql): | ||
|
||
```sql | ||
CREATE CATALOG example USING postgresql | ||
WITH ( | ||
"connection-url" = 'jdbc:pg:localhost:5432', | ||
"connection-user" = '${ENV:POSTGRES_USER}', | ||
"connection-password" = '${ENV:POSTGRES_PASSWORD}', | ||
"case-insensitive-name-matching" = 'true' | ||
); | ||
``` | ||
|
||
This example assumes that the `POSTGRES_USER` and `POSTGRES_PASSWORD` | ||
environmental variables are set as [secrets](/security/secrets) on the | ||
coordinator node. | ||
|
||
## See also | ||
|
||
* [](/sql/drop-catalog) | ||
* [](/admin/properties-catalog) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# DROP CATALOG | ||
|
||
## Synopsis | ||
|
||
```text | ||
DROP CATALOG catalog_name | ||
``` | ||
|
||
## Description | ||
|
||
Drops an existing catalog. Dropping a catalog does not interrupt any running | ||
queries that use it, but makes it unavailable to any new queries. | ||
|
||
:::{warning} | ||
Some connectors are known not to release all resources when dropping a catalog | ||
that uses such connector. This includes all connectors that can read data from | ||
HDFS, S3, GCS, or Azure, which are [](/connector/hive), | ||
[](/connector/iceberg), [](/connector/delta-lake), and | ||
[](/connector/hudi). | ||
::: | ||
|
||
:::{note} | ||
This command requires the [catalog management type](/admin/properties-catalog) | ||
to be set to `dynamic`. | ||
::: | ||
|
||
## Examples | ||
|
||
Drop the catalog `example`: | ||
|
||
``` | ||
DROP CATALOG example; | ||
``` | ||
|
||
## See also | ||
|
||
* [](/sql/create-catalog) | ||
* [](/admin/properties-catalog) |