Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests not rendered with DBT_MANIFEST mode #631

Closed
fabiomx opened this issue Oct 26, 2023 · 2 comments
Closed

Tests not rendered with DBT_MANIFEST mode #631

fabiomx opened this issue Oct 26, 2023 · 2 comments
Labels
duplicate This issue or pull request already exists
Milestone

Comments

@fabiomx
Copy link
Contributor

fabiomx commented Oct 26, 2023

Hi,

I'm currently using Cosmos 1.2.1 and dbt-core 1.5.4 on Google Cloud Composer, with local execution mode.

When I set the load mode to DBT_MANIFEST, tests are not rendered. However, they are successfully rendered when I switch to the DBT_LS mode. Based on both the documentation and the contents of the manifest file, I believe the tests should be rendered correctly in both modes.

I could switch to using the DBT_LS mode exclusively, but I face the issue mentioned here.

Below are the corresponding sections of my code:

project_config = ProjectConfig(
    dbt_project_path="/home/airflow/gcs/dags/dbt",
    manifest_path="/home/airflow/gcs/dags/dbt/target/manifest.json",
)

def get_render_config(schedule_tag: str):
    return RenderConfig(load_method=LoadMode.DBT_MANIFEST,
                        select=[f"tag:{schedule_tag}"]
                        )

@dag(dag_id="dbt_hourly", schedule_interval=timedelta(hours=1), default_args=default_args, catchup=False)
def dbt_hourly_dag():
    start = EmptyOperator(task_id="start")

    dbt_tg = DbtTaskGroup(
        project_config=project_config,
        profile_config=profile_config,
        execution_config=execution_config,
        default_args=default_args,
        operator_args=operator_args,
        render_config=get_render_config("hourly")

    )

    end = EmptyOperator(task_id="end")

    start >> dbt_tg >> end

On the other hand, on the tasks log, I can see this:

[2023-10-26, 20:19:25 UTC] {log.py:232} WARNING - [�[34m2023-10-26T20:19:25.788+0000�[0m] {�[34msubprocess.py:�[0m94} INFO�[0m - �[35m(astronomer-cosmos)�[0m - �[0m20:19:25  Unable to do partial parsing because saved manifest not found. Starting full parse.�[0m
[2023-10-26, 20:19:25 UTC] {subprocess.py:94} INFO - �[0m20:19:25  Unable to do partial parsing because saved manifest not found. Starting full parse.

and I can't understand it since the load mode is set to DBT_MANIFEST, and it could render the models thanks to the manifest file, right? Or I'm missing something? :)

Thanks!

@fabiomx
Copy link
Contributor Author

fabiomx commented Oct 27, 2023

Ok, I've just seen this #615
So I suppose we can close this issue :)

@tatiana tatiana added the duplicate This issue or pull request already exists label Oct 27, 2023
@tatiana
Copy link
Collaborator

tatiana commented Oct 27, 2023

Thanks for reporting the issue, @fabiomx ! This is a duplicate of #580.

We'll be releasing the fix as part of Cosmos 1.2.2, next week!

@tatiana tatiana closed this as completed Oct 27, 2023
@tatiana tatiana added this to the 1.2.2 milestone Oct 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants