Skip to content

Commit

Permalink
Merge remote-tracking branch 'unity/main'
Browse files Browse the repository at this point in the history
# Conflicts:
#	.gitignore
#	README.md
#	etc/db/h2db.mv.db
  • Loading branch information
pflooky committed Jun 17, 2024
2 parents a3cd4a7 + af5c342 commit 7f4f3e6
Show file tree
Hide file tree
Showing 38 changed files with 1,029 additions and 951 deletions.
29 changes: 29 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: Bug report
about: Create a bug report to help us improve
labels: "bug"
---

**Describe the bug**

<!-- A clear and concise description of what the bug is. -->

**To Reproduce**

Steps to reproduce the behavior:

1. ...
2. ...
3. ...

**Expected behavior**

<!-- A clear and concise description of what you expected to happen. -->

**System [please complete the following information]:**

- OS: e.g. [Ubuntu 18.04]

**Additional context**

<!-- Add any other context about the problem here. -->
17 changes: 17 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
name: Feature request
about: Suggest a new feature
labels: "enhancement"
---

**Is your feature request related to a problem? Please describe.**

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when ... -->

**Describe the solution you would like**

<!-- A clear and concise description of what you want to happen. -->

**Additional context**

<!-- Add any other context or screenshots about the feature request here. -->
10 changes: 10 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
**PR Checklist**

- [ ] A description of the changes is added to the description of this PR.
- [ ] If there is a related issue, make sure it is linked to this PR.
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added or modified a feature, documentation in `docs` is updated

**Description of changes**

<!-- Please state what you've changed and how it might affect the users. -->
3 changes: 0 additions & 3 deletions .github/workflows/sbt-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,3 @@ jobs:
cache: 'sbt'
- name: Run tests
run: sbt test
# Optional: This step uploads information to the GitHub dependency graph and unblocking Dependabot alerts for the repository
- name: Upload dependency graph
uses: scalacenter/sbt-dependency-submission@ab086b50c947c9774b70f39fc7f6e20ca2706c91
7 changes: 3 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
etc/db/h2db.mv.db
.idea
*target*
build/sbt-launch-*
.bsp
target/
.idea/
.bsp/
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Violating these terms may lead to a permanent ban.
### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
Expand Down
2 changes: 1 addition & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Notice - https://github.com/delta-io/delta/blob/master/NOTICE.txt

sbt/sbt-jupiter-interface - https://github.com/sbt/sbt-jupiter-interface/tree/main
Copyright sbt-jupiter-interface authors
LIcense - https://github.com/sbt/sbt-jupiter-interface/blob/main/LICENSE
License - https://github.com/sbt/sbt-jupiter-interface/blob/main/LICENSE

raphw/byte-buddy - https://github.com/raphw/byte-buddy
Copyright byte-buddy authors
Expand Down
35 changes: 20 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Let's take Unity Catalog for spin. In this guide, we are going to do the followi

OR

You have to ensure that you local environment has the following:
You have to ensure that your local environment has the following:
- Clone this repository.
- Ensure the `JAVA_HOME` environment variable your terminal is configured to point to JDK11+.
- Compile the project using `build/sbt package`
Expand All @@ -38,31 +38,31 @@ docker run -d -i --name unitycatalog -p 8081:8081 datacatering/unitycatalog:0.1.

In a terminal, in the cloned repository root directory, start the UC server.

```
```sh
bin/start-uc-server
```

For the rest of the steps, continue in a different terminal.

### Operate on Delta tables with the CLI
Let's list the tables.
```
```sh
bin/uc table list --catalog unity --schema default
```
You should see a few tables. Some details are truncated because of the nested nature of the data.
To see all the content, you can add `--output jsonPretty` to any command.

Next, let's get the metadata of one those tables.
Next, let's get the metadata of one of those tables.

```
```sh
bin/uc table get --full_name unity.default.numbers
```

You can see that it is a Delta table. Now, specifically for Delta tables, this CLI can
print snippet of the contents of a Delta table (powered by the [Delta Kernel Java](https://delta.io/blog/delta-kernel/) project).
Let's try that.

```
```sh
bin/uc table read --full_name unity.default.numbers
```

Expand All @@ -71,7 +71,7 @@ bin/uc table read --full_name unity.default.numbers
For trying with DuckDB, you will have to [install it](https://duckdb.org/docs/installation/) (at least version 1.0).
Let's start DuckDB and install a couple of extensions. To start DuckDB, run the command `duckdb` in the terminal.
Then, in the DuckDB shell, run the following commands:
```sh
```sql
install uc_catalog from core_nightly;
load uc_catalog;
install delta;
Expand All @@ -80,8 +80,8 @@ load delta;
If you have installed these extensions before, you may have to run `update extensions` and restart DuckDB
for the following steps to work.

Now that we have DuckDB all set up, let's trying connecting to UC by specifying a secret.
```sh
Now that we have DuckDB all set up, let's try connecting to UC by specifying a secret.
```sql
CREATE SECRET (
TYPE UC,
TOKEN 'not-used',
Expand All @@ -90,10 +90,10 @@ CREATE SECRET (
);
```
You should see it print a short table saying `Success` = `true`. Then we attach the `unity` catalog to DuckDB.
```sh
```sql
ATTACH 'unity' AS unity (TYPE UC_CATALOG);
```
Now we ready to query. Try the following
Now we are ready to query. Try the following

```sql
SHOW ALL TABLES;
Expand All @@ -114,19 +114,24 @@ See the full [tutorial](docs/tutorial.md) for more details.
- Compatibility and stability: The APIs are currently evolving and should not be assumed to be stable.

## Compiling and testing
- Install JDK 11 by whatever mechanism that is appropriate for your system, and
- Install JDK 11 by whatever mechanism is appropriate for your system, and
set that version to be the default Java version (e.g., by setting env variable
JAVA_HOME)
- To compile all the code without running tests, run the following:
```
```sh
build/sbt clean compile
```
- To compile and execute tests, run the following:
```
```sh
build/sbt clean test
```
- To update the API specification, just update the `api/all.yaml` and then run the following:
```
```sh
build/sbt generate
```
This will regenerate the OpenAPI data models in the UC server and data models + APIs in the client SDK.

### Using more recent JDKs

The build script [checks for a lower bound on the JDK](./build.sbt#L14) but the [current SBT version](./project/build.properties)
imposes an upper bound. Please check the [JDK compatibility](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html) documentation for more information
173 changes: 0 additions & 173 deletions dev/copybara.py

This file was deleted.

2 changes: 1 addition & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ This is an experimental feature and only supported for python functions that tak
It runs the functions using the python engine script at `etc/data/function/python_engine.py`.

Example:
Invoke a python sum function that take two integer inputs:
Invoke a python sum function that takes two integer inputs:
```sh
bin/uc function call --full_name my_catalog.my_schema.my_function --input_params "1,2"
```
Expand Down
2 changes: 1 addition & 1 deletion docs/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ bin/start-uc-server

## Configuration

The server config file is at the location `etc/conf/server.properties` (relative to the project root).
The server config file is at the location `etc/conf/server.properties` (relative to the project root).

- `server.env`: The environment in which the server is running. This can be set to `dev` or `test`. When
set to `test` the server will instantiate an empty in-memory h2 database for storing metadata.
Expand Down
Loading

0 comments on commit 7f4f3e6

Please sign in to comment.