[CT-3378] [Bug] "External connection exception occurred: not a gzip file" error when running dbt deps

[Visya]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running dbt deps behind proxy (with HTTP_PROXY, HTTPS_PROXY and FTP_PROXY variables set up), I get an error:

External connection exception occurred: not a gzip file

I tried installing with git, which works, but as soon as I try to install any bigger package that depends on dbt_utils, it fails with "Found duplicate project "dbt_utils". This occurs when a dependency has the same project name as some other dependency."

Expected Behavior

Packages are able to be installed behind the proxy with dbt deps.

Steps To Reproduce

1. Set HTTP_PROXY, HTTPS_PROXY environment variables.
2. Add a package to packages.yml.

packages:
  - package: dbt-labs/dbt_utils
    version: 1.1.1

4. Run dbt deps.
5. Observe error "External connection exception occurred: not a gzip file".

Relevant log output

11:17:28  Running with dbt=1.7.1
11:17:29  Updating lock file in file path: [...]code\test\src/package-lock.yml
11:17:30  Installing dbt-labs/dbt_utils
11:17:39  Encountered an error:
External connection exception occurred: not a gzip file

Environment

- OS: Windows 10 Enterprise
- Python: 3.11.5
- dbt: 1.7.1

Which database adapter are you using with dbt?

postgres

Additional Context

I have python-certifi-win32 installed, which resolved issues similar to this issue.

[jtcohen6]

Hey @Visya! I see you're using v1.7.1 — is this working for you on earlier versions of dbt-core? We need to understand if this is a regression (due to some change in v1.7), versus that has never worked.

[Visya]

Hi @jtcohen6, I am setting up a new environment in a new system, so it never worked for me. But I have other colleagues in the same network, that are able to download packages behind the proxy.

[graciegoheen]

@Visya do you happen to know which version of dbt your colleagues are using?

[josezeta]

Currently we are running some workflows using DBT-core v 1.7.4 in which this error is happeing. This is happening ramdomly with no exact pattern.

[josezeta]

@Visya I do not think this is a BUG related to DBT-core, It could maybe that due the concurrency of getting these packages from github is causing some crash in some cases.

[jfo8001]

I saw this error for the first time today having upgraded our snowflake connector recently. DBT core has remained the same for the last 9 months.
dbt-core = "^1.5.2"
dbt-snowflake = "^1.5.2"
snowflake-connector-python = "^3.7.1"

Last week we upgraded from
snowflake-connector-python = "^3.1.0"

[dbeatty10]

@josezeta and @jfo8001 did each of you see this error for the first time today?

If so, was it intermittent, or did it happen every time to you ran dbt deps? Are you using a proxy like the original poster, by any chance?

Also, could you share your environment details?

- OS:
- Python:
- dbt:

Examples:

OS: Ubuntu 20.04
Python: 3.9.12 (python3 --version)
dbt-core: 1.1.1 (dbt --version)

[IL-Jerry]

Thanks @dbeatty10 for the detailed response.

[pranjalbhatt]

Hi,

I am receiving the same error ConnectionError: External connection exception occurred: not a gzip file while downloading via packages.yml file.

NOTE: Error only occurs while downloading dbt-utlis package, may be because it dependency for elementary. I have not mentioned dbt-utils in my packages.yml file. Other packages get installed quickly.

packages.yml file looks like-

packages:
  - package: calogica/dbt_expectations
    version: 0.10.4
  - package: brooklyn-data/dbt_artifacts
    version: 2.6.4
  - package: elementary-data/elementary
    version: 0.16.1

[dbeatty10]

@pranjalbhatt are you still seeing this error? Or did it resolve?

[pranjalbhatt]

@pranjalbhatt are you still seeing this error? Or did it resolve?

Yes. I am still seeing this error. Unable to download dbt-utils and elementary. Tried to change versions as well. Is there a fix ?

[dbeatty10]

@pranjalbhatt I am able to successfully install the elementary version 0.16.1 using dbt deps via both dbt-core 1.7 and 1.8.

Can you share the output of dbt --version so I can see which version of dbt-core you are using?

[pranjalbhatt]

@pranjalbhatt I am able to successfully install the elementary version 0.16.1 using dbt deps via both dbt-core 1.7 and 1.8.

Can you share the output of dbt --version so I can see which version of dbt-core you are using?

Yes, still unable to download the elementary. I am using dbt-core.

Core:
  - installed: 1.8.1

As a temporary fix, I have used git clone to do install the package.

[dbeatty10]

@pranjalbhatt I am not able to replicate the issue you are having, and I suspect it is distinct from the other reports in this issue. If you are continuing to experience this, could you open a new issue here?

EDIT: Nevermind! I see that you opened dbt-labs/dbt-utils#950.