Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fails fatally at startup iAND has misleading error message if update url is unreachable #67

Open
rob-1126 opened this issue Apr 29, 2024 · 2 comments

Comments

@rob-1126
Copy link

This plugin breaks astro dev start AND use in Astronomer Software if the Airflow is working from a place where it can't access the update url.

Worked around in the image by setting

ENV AIRFLOW__ASTRONOMER__UPDATE_INTERVAL=0

Here's the log that was sent. It's pretty scary and gnarly looking.

Can we just set a 2 second timeout on this url, catch basically any and all kinds of mischief from just that request, log a message, and keep-on-trucking?


Traceback (most recent call last):

File "/usr/local/lib/python3.11/site-packages/airflow/plugins_manager.py", line 249, in load_entrypoint_plugins

 plugin_class = entry_point.load()

                ^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/site-packages/importlib_metadata/init.py", line 211, in load

 module = import_module(match.group('module'))

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/importlib/init.py", line 126, in import_module

 return _bootstrap._gcd_import(name[level:], package, level)

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "", line 1204, in _gcd_import

File "", line 1176, in _find_and_load

File "", line 1147, in _find_and_load_unlocked

File "", line 690, in _load_unlocked

File "", line 940, in exec_module

File "", line 241, in _call_with_frames_removed

File "/usr/local/lib/python3.11/site-packages/airflow/providers/openlineage/plugins/openlineage.py", line 21, in

 from airflow.providers.openlineage.plugins.listener import get_openlineage_listener

File "/usr/local/lib/python3.11/site-packages/airflow/providers/openlineage/plugins/listener.py", line 24, in

 from openlineage.client.serde import Serde

File "/usr/local/lib/python3.11/site-packages/openlineage/client/init.py", line 5, in

 from openlineage.client.client import OpenLineageClient, OpenLineageClientOptions

File "/usr/local/lib/python3.11/site-packages/openlineage/client/client.py", line 11, in

 from openlineage.client.serde import Serde

File "/usr/local/lib/python3.11/site-packages/openlineage/client/serde.py", line 16, in

 import numpy

File "/usr/local/lib/python3.11/site-packages/numpy/init.py", line 130, in

 from numpy.__config__ import show as show_config

File "/usr/local/lib/python3.11/site-packages/numpy/config.py", line 4, in

 from numpy.core._multiarray_umath import (

File "/usr/local/lib/python3.11/site-packages/numpy/core/init.py", line 24, in

 from . import multiarray

File "/usr/local/lib/python3.11/site-packages/numpy/core/multiarray.py", line 10, in

 from . import overrides

File "/usr/local/lib/python3.11/site-packages/numpy/core/overrides.py", line 8, in

 from numpy.core._multiarray_umath import (

File "", line 1173, in _find_and_load

MemoryError

[2024-04-28T22:42:34.754+0000] {plugins_manager.py:257} ERROR - Failed to import plugin astronomer_version_check

Traceback (most recent call last):

File "/usr/local/lib/python3.11/site-packages/airflow/plugins_manager.py", line 249, in load_entrypoint_plugins

 plugin_class = entry_point.load()

                ^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/site-packages/importlib_metadata/init.py", line 211, in load

 module = import_module(match.group('module'))

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/importlib/init.py", line 126, in import_module

 return _bootstrap._gcd_import(name[level:], package, level)

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "", line 1204, in _gcd_import

File "", line 1176, in _find_and_load

File "", line 1147, in _find_and_load_unlocked

File "", line 690, in _load_unlocked

File "", line 940, in exec_module

File "", line 241, in _call_with_frames_removed

File "/usr/local/lib/python3.11/site-packages/astronomer/airflow/version_check/plugin.py", line 9, in

 from .update_checks import UpdateAvailableBlueprint

File "/usr/local/lib/python3.11/site-packages/astronomer/airflow/version_check/update_checks.py", line 20, in

 from flask import Blueprint, current_app, flash, redirect, render_template, request, g

File "/usr/local/lib/python3.11/site-packages/flask/init.py", line 4, in

 from . import json as json

File "/usr/local/lib/python3.11/site-packages/flask/json/init.py", line 8, in

 from ..globals import current_app

File "/usr/local/lib/python3.11/site-packages/flask/globals.py", line 4, in

 from werkzeug.local import LocalProxy

File "/usr/local/lib/python3.11/site-packages/werkzeug/init.py", line 1, in

 from .serving import run_simple as run_simple

File "/usr/local/lib/python3.11/site-packages/werkzeug/serving.py", line 27, in

 from ._internal import _log

File "/usr/local/lib/python3.11/site-packages/werkzeug/_internal.py", line 35, in

 _cookie_re = re.compile(

              ^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/init.py", line 227, in compile

 return _compile(pattern, flags)

        ^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/init.py", line 294, in _compile

 p = _compiler.compile(pattern, flags)

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_compiler.py", line 745, in compile

 p = _parser.parse(p, flags)

     ^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 989, in parse

 p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 464, in _parse_sub

 itemsappend(_parse(source, state, verbose, nested + 1,

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 872, in _parse

 p = _parse_sub(source, state, sub_verbose, nested + 1)

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 464, in _parse_sub

 itemsappend(_parse(source, state, verbose, nested + 1,

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 872, in _parse

 p = _parse_sub(source, state, sub_verbose, nested + 1)

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 464, in _parse_sub

 itemsappend(_parse(source, state, verbose, nested + 1,

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 872, in _parse

 p = _parse_sub(source, state, sub_verbose, nested + 1)

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/re/_parser.py", line 464, in _parse_sub

 itemsappend(_parse(source, state, verbose, nested + 1,

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

SystemError: error return without exception set

@rishkarajgi
Copy link

@rob-1126
Copy link
Author

rob-1126 commented May 7, 2024

I think this may turn out to be a different issue although it's possible that the error handling for one will handle the other as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants