airbyte-ci: build manifest only with local CDK when flag is passed #48818
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
This stack of pull requests is managed by Graphite. Learn more about stacking. |
alafanechere
force-pushed
the
augustin/use-local-cdk-manifest-only
branch
from
74866b5 to
f217461
Compare
alafanechere
changed the title
| @@ -238,15 +239,16 @@ def with_python_connector_source(context: ConnectorContext) -> Container: | |||
| return with_python_package(context, testing_environment, connector_source_path) | |||
|
|
|||
|
|
|||
| async def apply_python_development_overrides(context: ConnectorContext, connector_container: Container) -> Container: | |||
| def apply_python_development_overrides(context: ConnectorContext, connector_container: Container) -> Container: | |||
There was a problem hiding this comment.
As discussed - this is no longer a coroutine but the return value (Container) is still awaitable. 👍
| raise UsageError( | ||
| f"Local CDK not found at '{path_to_cdk}'. Please clone the CDK repository in a sibling directory to the airbyte repository. Or use --use-cdk-ref to specify a CDK ref." | ||
| ) |
| @@ -28,14 +28,6 @@ def connector_context(dagger_client): | |||
| return context | |||
|
|
|||
|
|
|||
| @pytest.mark.skip( | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
I think this is ready to go once tests are passing! I tested successfully in the CDK repo - forcing manifest-only connector tests to fail by removing a key part of the initialization in the CDK. This is sufficient evidence to me that the --use-local-cdk option is working as expected for manifest-only connectors.
Example failure here: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/12185161294/job/33990749733?pr=136
alafanechere
force-pushed
the
augustin/use-local-cdk-manifest-only
branch
from
798af31 to
42ec312
Compare
alafanechere
force-pushed
the
augustin/use-local-cdk-manifest-only
branch
from
42ec312 to
786360c
Compare
Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/10779
Resolves airbytehq/airbyte-python-cdk#64
Being tested here:
What
Closes airbytehq/airbyte-python-cdk#64
Our manifest only connectors were not built with the local CDK or CDK ref because their build step did not call the development override function.
Bonus: consider missing local cdk as a usage error, handle it in the build phase to have a clean report with a clear error message when the local cdk is not available in the path.