You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The intent of that commit is that when Dagster is started up, the list of partitions is regenerated, in order to avoid inconsistencies with previous Dagster runs (e.g. if asset prefixes are changed / country assets are removed, the old partition names would persist).
However, it seems that the Dagster daemon also performs a complete code reload every 60 seconds. (You can observe this by adding a print statement at the top level of python/popgetter/__init__.py.)
What this means is that the list of partitions gets wiped and regenerated every 60 seconds. This is completely harmless in that it doesn't affect the publishing process itself, so this is almost a non-issue, but it can be a little bit disconcerting when viewing in the web UI.
The text was updated successfully, but these errors were encountered:
I noticed that this is a weird side effect of this commit (Urban-Analytics-Technology-Platform/popgetter@01f84e7) which in turn was part of #108 (just merged).
The intent of that commit is that when Dagster is started up, the list of partitions is regenerated, in order to avoid inconsistencies with previous Dagster runs (e.g. if asset prefixes are changed / country assets are removed, the old partition names would persist).
However, it seems that the Dagster daemon also performs a complete code reload every 60 seconds. (You can observe this by adding a print statement at the top level of
python/popgetter/__init__.py
.)What this means is that the list of partitions gets wiped and regenerated every 60 seconds. This is completely harmless in that it doesn't affect the publishing process itself, so this is almost a non-issue, but it can be a little bit disconcerting when viewing in the web UI.
The text was updated successfully, but these errors were encountered: