deps: relax dependency constraints (#36)

This commit relaxes our dependency constraints by removing the lockfiles
and replacing them by broad ranges for our supported dependency
versions.

To test that the SDK works with each version, the test matrix was
adapted to run each test for each `(python_version,resolution_strategy)`
tuple, where `resolution_strategy` is how UV will choose each
dependency.
- By using `resolution_strategy=lowest`, we will test the lowest version
of our dependencies.
- By using `resolution_strategy=lowest-direct`, we will test the lowest version
of our direct dependencies, and the highest version of transitive dependencies.
- By using `resolution_strategy=highest`, we will test the highest version
of all dependencies.

Due to some transitive dependencies not working in certain Python
versions, a new `constraints/` directory was created, which contains
`.txt` files that specify our dependency ranges for transitive
dependencies.

These constraints files are not shipped with the library and are used
only for testing. Users are expected to define their transitive
constraints themselves depending on the Python version.

.changes/unreleased/Dependencies-20240809-170009.yaml

@@ -0,0 +1,3 @@
+kind: Dependencies
+body: Relaxed dependency constraints
+time: 2024-08-09T17:00:09.507206+02:00

.github/workflows/code-quality.yaml

@@ -19,11 +19,14 @@ jobs:
       - name: fetch server schema
         run: "hatch run dev:fetch-schema"
 
+      - name: show env info
+        run: "hatch run test:uv pip freeze"
+
       - name: unit tests
         run: "hatch run test:unit"
 
       - name: integration tests
-        run: "hatch run test.py3.12:integration"
+        run: "hatch run test:integration"
         env:
           SL_HOST: ${{ secrets.TEST_HOST }}
           SL_ENV_ID: ${{ secrets.TEST_ENV_ID }}

CONTRIBUTING.md

@@ -28,10 +28,12 @@ Hatch will manage your environments for you. Check out the list of available env
 
 For most use cases, you probably want to use the `dev` environment by running `hatch shell dev`.
 
+If you're having problems with your environment, check out the [troubleshooting](#troubleshooting) section.
+
 
 ### Upgrading dependencies
 
-We use locked `requirements.txt` files for dependency management. If you want to upgrade, remove or add a dependency, change the `deps/*.in` files and run `hatch run lock`. Don't edit `.txt` files manually, since those are autogenerated by [uv](https://github.com/astral-sh/uv/).
+If you want to upgrade, remove or add a dependency, change the `pyproject.toml` file. Make sure to include a reasonable min/max version range to facilitate installation by end users. Also make sure the entire test matrix passes.
 
 
 ### Making code changes
@@ -43,9 +45,9 @@ We use [changie](https://changie.dev/) to manage our [`CHANGELOG.md`](./CHANGELO
 
 ### Running tests
 
-Run tests by using `hatch run test:all`. This will run all our tests in all supported Python versions. Make sure they pass before you submit a PR.
+Run tests by using `hatch run test:all`. This will run all our tests in all supported Python versions and with a variety of dependency versions. Make sure they pass before you submit a PR.
 
-If you want to run unit test separately, run `hatch run test:unit`. For integration tests, run `hatch run test.py3.12:integration` to avoid running slow tests in all python versions.
+If you want to run unit test separately, run `hatch run test:unit`. For integration tests, run `hatch run test.py3.12-highest:integration` to avoid running slow tests in all python versions.
 
 The unit test suite requires a `tests/server_schema.gql` file to exist, since it checks that all GraphQL queries from the SDK will work against the server. That file must contain the schema of the GraphQL API, and you can obtain it by running our introspection script with `hatch run dev:fetch-schema`.
 
@@ -63,3 +65,17 @@ We don't require every commit to be perfect, but your pull-request as a whole mu
 
 If you think you've found a security vulnerability (i.e something that could leak customer data, compromise API keys, gain access to unauthorized resources), **do not open an issue in GitHub**. In that case, contact us directly at `[email protected]`, and wait until the vulnerability has been patched before publicly disclosing it.
 
+
+## Troubleshooting
+
+Here's a non-exhaustive list of problems that might occur during development.
+
+### `clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly`
+
+For compatibility issues, we support older versions of libraries such as PyArrow. However, those older versions might not compile properly in MacOS with the error above. This issue might arise especially when running tests in the environment with older versions.
+
+To solve it, you need to install [OpenBLAS](https://www.openblas.net/) and tell the compiler where it is:
+```
+brew install openblas
+export OPENBLAS=$(brew --prefix openblas)
+```

constraints/README.md

@@ -0,0 +1,7 @@
+# Dependency constraints and overrides
+
+All files in this directory specify transitive dependency constraints for usage in our test environments. The files in this directory are _not_ shipped with the package, and users should specify those constraints themselves, according to what works in their own system. Learn more about pip constraints [here](https://pip.pypa.io/en/stable/user_guide/#constraints-files).
+
+This is useful for specifying different minimum versions of a package for Python 3.12 and 3.9, since older versions might not support the newest Python and vice-versa.
+
+For direct dependencies, we solve this by using the `python_version` [environment marker](https://peps.python.org/pep-0508/#environment-markers) in our dependencies.

constraints/build-constraints/py312.txt

@@ -0,0 +1,3 @@
+cython>=3.0.11
+setuptools>=72.1.0
+wheel>=0.43.0

constraints/constraints/py312.txt

@@ -0,0 +1,6 @@
+numpy>=1.26.0
+requests>=2.32.0
+urllib3>=1.26.8
+yarl>=1.9.3
+frozenlist>=1.4.0
+aiohttp>=3.9.0

constraints/overrides/py312.txt

@@ -0,0 +1 @@
+pyarrow>=0.14.0

constraints/py310.txt

@@ -0,0 +1,3 @@
+numpy>=1.21.3,<2.0.0
+requests>=2.27.0
+urllib3>=1.26.0

constraints/py311.txt

@@ -0,0 +1,6 @@
+numpy>=1.24.0,<2.0.0
+requests>=2.28.0
+urllib3>=1.26.8
+yarl>=1.8.2
+frozenlist>=1.3.1
+aiohttp>=3.8.2

constraints/py312.txt

@@ -0,0 +1,6 @@
+numpy>=1.26.0,<2.0.0
+requests>=2.32.0
+urllib3>=1.26.8
+yarl>=1.9.3
+frozenlist>=1.4.0
+aiohttp>=3.9.0

constraints/py39.txt

@@ -0,0 +1 @@
+numpy>=1.21.0,<2.0.0

dbtsl/models/base.py

@@ -3,7 +3,7 @@
 from dataclasses import field as dc_field
 from functools import cache
 from types import MappingProxyType
-from typing import List, Set, Type, Union
+from typing import Any, List, Set, Type, Union
 from typing import get_args as get_type_args
 from typing import get_origin as get_type_origin
 
@@ -62,11 +62,11 @@ def gql_model_name(cls) -> str:
     # If we do that, we need to modify this method to memoize what fragments were already created
     # so that we exit the recursion gracefully
     @staticmethod
-    def _get_fragments_for_field(type: Type, field_name: str) -> Union[str, List[GraphQLFragment]]:
+    def _get_fragments_for_field(type: Union[Type[Any], str], field_name: str) -> Union[str, List[GraphQLFragment]]:
         if inspect.isclass(type) and issubclass(type, GraphQLFragmentMixin):
             return type.gql_fragments()
 
-        if get_type_origin(type) == list:
+        if get_type_origin(type) is list:
             inner_type = get_type_args(type)[0]
             return GraphQLFragmentMixin._get_fragments_for_field(inner_type, field_name)