Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] br_ms_cnes #590

Merged
merged 59 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
e7016f0
feat: add new poetry depencies
folhesgabriel Dec 6, 2023
71b9e8b
re register flow
folhesgabriel Dec 6, 2023
1f63455
re register flow
folhesgabriel Dec 6, 2023
0ad8bc0
feat: change pyreadbc version config in pyproject.toml
folhesgabriel Dec 6, 2023
2c429e6
Merge branch 'main' into staging/cnes
folhesgabriel Dec 19, 2023
799392c
feat: test new cnes crawler
folhesgabriel Dec 19, 2023
02351e6
feat: add git to Dockerfile
folhesgabriel Dec 19, 2023
8bba184
Merge remote-tracking branch 'origin' into staging/cnes
folhesgabriel Dec 19, 2023
c9da977
Merge remote-tracking branch 'origin' into staging/cnes
folhesgabriel Jan 30, 2024
39aa1ba
feat: and new task
folhesgabriel Feb 1, 2024
95230b5
Merge branch 'main' into staging/cnes
folhesgabriel Feb 1, 2024
b2406ce
feat: regen poetry.lock file
folhesgabriel Feb 1, 2024
44e7801
feat: fix ci lint python action
folhesgabriel Feb 1, 2024
81f49f3
Merge branch 'main' of https://github.com/basedosdados/pipelines into…
folhesgabriel Feb 1, 2024
89dd0e9
feat: remove unnused modules
folhesgabriel Feb 1, 2024
27e6648
feat: fix bash path
folhesgabriel Feb 1, 2024
ad883a6
re-register flow
folhesgabriel Feb 2, 2024
7f4b14d
fix: decompress_dbc task
folhesgabriel Feb 5, 2024
50d43ed
fix: ci action
folhesgabriel Feb 5, 2024
fa7f844
fix: upstream tasks
folhesgabriel Feb 5, 2024
176d63a
check folder tree
folhesgabriel Feb 5, 2024
f7077d6
feat: add logs
folhesgabriel Feb 5, 2024
c76a6ab
feat: refactor convert2dbf.sh into a series of cli comands
folhesgabriel Feb 5, 2024
e9fc590
fix: agora vai
folhesgabriel Feb 5, 2024
6002b2f
fix: hopeless
folhesgabriel Feb 5, 2024
9769f4b
fix: is it blasting or not?
folhesgabriel Feb 5, 2024
8d81aa4
fix: debug decompress_dbc
folhesgabriel Feb 5, 2024
4970af5
feat: test prefect ShellTask
folhesgabriel Feb 6, 2024
2e42f3f
feat: add git to Dockerfile
folhesgabriel Feb 6, 2024
5e3f629
feat: add refactored decompress_dbc task
folhesgabriel Feb 6, 2024
5a60b2b
fix: ci action
folhesgabriel Feb 6, 2024
4836370
Merge branch 'main' into staging/cnes
mergify[bot] Feb 6, 2024
c9f7c22
fix: final modifications
folhesgabriel Feb 7, 2024
b9c66fa
fix: unnused imports
folhesgabriel Feb 7, 2024
42bea6e
feat: requested changes
folhesgabriel Feb 8, 2024
9f076b1
feat: add datasus deepcopy flow
folhesgabriel Feb 15, 2024
7ec9de3
feat: add final modifications
folhesgabriel Feb 16, 2024
11116be
Merge branch 'main' into staging/cnes
folhesgabriel Feb 16, 2024
02abb8f
fix: merge conflicts
folhesgabriel Feb 16, 2024
5f02cb9
Merge branch 'staging/cnes' of https://github.com/basedosdados/pipeli…
folhesgabriel Feb 16, 2024
efc21b7
fix: crawler_datasus template flow parameters
folhesgabriel Feb 16, 2024
b7c80c5
feat: delete old tasks & utils
folhesgabriel Feb 16, 2024
b6f9b26
Merge branch 'main' into staging/cnes
mergify[bot] Feb 16, 2024
5523a0e
fix: pre_process_files output to next task
folhesgabriel Feb 17, 2024
4de4623
feat: add sequential chunks to control requests made to datasus ftp
folhesgabriel Feb 18, 2024
a020f70
feat: goodbye
folhesgabriel Feb 19, 2024
30f84ac
fix: check_files_to_parse return task
folhesgabriel Feb 19, 2024
3efc9d6
Merge branch 'main' into staging/cnes
mergify[bot] Feb 20, 2024
422110a
Merge branch 'main' into staging/cnes
mergify[bot] Feb 20, 2024
c3d5287
feat: add sia flow
folhesgabriel Feb 28, 2024
90e882e
Merge remote-tracking branch 'origin' into staging/cnes
folhesgabriel Feb 28, 2024
cdd8768
Merge branch 'main' into staging/cnes
folhesgabriel Feb 28, 2024
9cf9416
fix: add dependencies
folhesgabriel Feb 28, 2024
3573629
Merge branch 'staging/cnes' of https://github.com/basedosdados/pipeli…
folhesgabriel Feb 28, 2024
75585ee
fix: concrete flow names
folhesgabriel Feb 28, 2024
80a9e40
feat: br_ms_cnes
folhesgabriel Feb 28, 2024
1f85fa1
add: deactivate regra_contratual schedule
folhesgabriel Feb 28, 2024
60980b0
feat: standardize flow names
folhesgabriel Feb 28, 2024
3181901
feat: final modifications
folhesgabriel Feb 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ ENV PYTHONUNBUFFERED 1
ARG PREFECT_VERSION=0.15.9
ENV PREFECT_VERSION $PREFECT_VERSION

# Install gcc, Google Chrome, CLI tools and R
# Install gcc, Google Chrome, CLI tools, git and R
RUN apt-get update && \
apt-get install --no-install-recommends -y wget gnupg && \
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
Expand All @@ -33,6 +33,7 @@ RUN apt-get update && \
python3-dev \
traceroute \
wget \
git \
&& \
apt-get install -y r-base && \
apt-get clean && \
Expand Down
Loading
Loading