diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..3c84e16 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,5 @@ +#terraform +.terraform +*.tfstate +*.tfstate.backup +*.tfvars diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..94ddb90 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,31 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: '' +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Desktop (please complete the following information):** + - OS: [e.g. iOS] + - Version [e.g. 22] + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..bbcbbe7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: '' +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..06e1527 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,27 @@ +Make sure you have checked _all_ steps below. + +### Issue + +- [ ] My PR addresses the following Issue (If an issue exists in JIRA or Github) + +### Description + +- [ ] Here are some details about my PR, including example of usage. + +### Licence + +- [ ] My PR adds the needed licence header to every added file: + +### Commits + +- [ ] I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": + 1. Subject is separated from body by a blank line + 2. Subject is limited to 50 characters + 3. Subject does not end with a period + 4. Subject uses the imperative mood ("add", not "adding") + 5. Body explains "what" and "why", not "how" + +### Documentation + +- [ ] In case of new functionality, my PR adds documentation that describes how to use it. + - All the public functions and the classes in the PR contain docstrings that explain what it does \ No newline at end of file diff --git a/.github/workflows/gitleaks.yml b/.github/workflows/gitleaks.yml new file mode 100644 index 0000000..849a02d --- /dev/null +++ b/.github/workflows/gitleaks.yml @@ -0,0 +1,16 @@ +name: gitleaks + +on: + push: + pull_request: + branches: + - master + - dev + +jobs: + gitleaks: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v1 + - name: gitleaks-action + uses: zricethezav/gitleaks-action@master diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..7aa04cb --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,32 @@ +# Contributing to one-click-mlflow +We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's: + +- Reporting a bug +- Discussing the current state of the code +- Submitting a fix +- Proposing new features +- Becoming a maintainer + +## We Develop with Github +We use github to host code, to track issues and feature requests, as well as accept pull requests. + +## We Use [Github Flow](https://guides.github.com/introduction/flow/index.html), So All Code Changes Happen Through Pull Requests +Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests: + +1. Fork the repo and create your branch from `master`. +2. If needed, update the documentation. +3. Make sure your code works. +4. Don't forget to add licence header to every file if you added any. +5. Issue that pull request! + +## Report bugs using Github's [issues](https://github.com/briandk/transcriptase-atom/issues) +We use GitHub issues to track public bugs. Report a bug by [opening a new issue](); it's that easy! + + +**Great Bug Reports** : + +Follow the GitHub issue template and give as many informations as you can. + +## License + +By contributing, you agree that your contributions will be licensed under its GNU LESSER GENERAL PUBLIC LICENSE. \ No newline at end of file diff --git a/IaC/main.tf b/IaC/main.tf index 1deb48c..c00b279 100644 --- a/IaC/main.tf +++ b/IaC/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. terraform { backend "gcs" { } @@ -35,3 +52,9 @@ module "mlflow" { network_self_link = module.network.network_self_link network_short_name = module.network.network_short_name } + +module "log_pusher" { + source = "./modules/mlflow/log_pusher" + project_id = var.project_id + depends_on = [module.mlflow] +} diff --git a/IaC/modules/mlflow/artifacts/main.tf b/IaC/modules/mlflow/artifacts/main.tf index e7c0fbd..0df49cc 100644 --- a/IaC/modules/mlflow/artifacts/main.tf +++ b/IaC/modules/mlflow/artifacts/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. resource "google_storage_bucket" "this" { name = var.bucket_name location = var.bucket_location @@ -13,4 +30,5 @@ resource "google_storage_bucket" "this" { type = "Delete" } } + uniform_bucket_level_access = var.storage_uniform } diff --git a/IaC/modules/mlflow/artifacts/outputs.tf b/IaC/modules/mlflow/artifacts/outputs.tf index c7af138..b2cb89a 100644 --- a/IaC/modules/mlflow/artifacts/outputs.tf +++ b/IaC/modules/mlflow/artifacts/outputs.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. output "url" { description = "gcs uri" value = google_storage_bucket.this.url diff --git a/IaC/modules/mlflow/artifacts/variables.tf b/IaC/modules/mlflow/artifacts/variables.tf index ec992d4..ddb6cc5 100644 --- a/IaC/modules/mlflow/artifacts/variables.tf +++ b/IaC/modules/mlflow/artifacts/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "bucket_name" { description = "Name of the bucket." type = string @@ -26,3 +43,8 @@ variable "module_depends_on" { type = any default = null } +variable "storage_uniform" { + type = bool + description = "Wether or not uniform level acces is to be activated for the buckets" + default = true +} diff --git a/IaC/modules/mlflow/database/main.tf b/IaC/modules/mlflow/database/main.tf index 167e7e7..b948e59 100644 --- a/IaC/modules/mlflow/database/main.tf +++ b/IaC/modules/mlflow/database/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. resource "random_id" "db_name_suffix" { byte_length = 5 } diff --git a/IaC/modules/mlflow/database/outputs.tf b/IaC/modules/mlflow/database/outputs.tf index 093dafd..8fb7675 100644 --- a/IaC/modules/mlflow/database/outputs.tf +++ b/IaC/modules/mlflow/database/outputs.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. output "instance_connection_name" { description = "Connection string used to connect to the instance" value = google_sql_database_instance.this_instance.connection_name diff --git a/IaC/modules/mlflow/database/variables.tf b/IaC/modules/mlflow/database/variables.tf index 6423cef..8f2370a 100644 --- a/IaC/modules/mlflow/database/variables.tf +++ b/IaC/modules/mlflow/database/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "instance_prefix" { type = string description = "Name of the database instance you want to deploy" diff --git a/IaC/modules/mlflow/log_pusher/main.tf b/IaC/modules/mlflow/log_pusher/main.tf new file mode 100644 index 0000000..0328a67 --- /dev/null +++ b/IaC/modules/mlflow/log_pusher/main.tf @@ -0,0 +1,33 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +resource "google_service_account" "log_pusher" { + account_id = "mlflow-log-pusher" + display_name = "mlflow log pusher" +} + +resource "google_project_iam_member" "log_pusher_iap" { + project = var.project_id + role = "roles/iap.httpsResourceAccessor" + member = "serviceAccount:${google_service_account.log_pusher.email}" +} + +resource "google_project_iam_member" "log_pusher_storage" { + project = var.project_id + role = "roles/storage.objectAdmin" + member = "serviceAccount:${google_service_account.log_pusher.email}" +} \ No newline at end of file diff --git a/IaC/modules/mlflow/log_pusher/variables.tf b/IaC/modules/mlflow/log_pusher/variables.tf new file mode 100644 index 0000000..adf7a33 --- /dev/null +++ b/IaC/modules/mlflow/log_pusher/variables.tf @@ -0,0 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +variable "project_id" { + type = string +} \ No newline at end of file diff --git a/IaC/modules/mlflow/main.tf b/IaC/modules/mlflow/main.tf index da253f6..583298b 100644 --- a/IaC/modules/mlflow/main.tf +++ b/IaC/modules/mlflow/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. module "artifacts" { source = "./artifacts" bucket_name = var.artifacts_bucket_name diff --git a/IaC/modules/mlflow/secret_manager/main.tf b/IaC/modules/mlflow/secret_manager/main.tf index 8b82ef4..c6e7b05 100644 --- a/IaC/modules/mlflow/secret_manager/main.tf +++ b/IaC/modules/mlflow/secret_manager/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. resource "google_secret_manager_secret" "secret" { provider = google-beta diff --git a/IaC/modules/mlflow/secret_manager/outputs.tf b/IaC/modules/mlflow/secret_manager/outputs.tf index 2ffa9b4..64b0700 100644 --- a/IaC/modules/mlflow/secret_manager/outputs.tf +++ b/IaC/modules/mlflow/secret_manager/outputs.tf @@ -1,4 +1,22 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. output "secret_value" { description = "Value of the created secret" value = google_secret_manager_secret_version.secret-version.secret_data + sensitive = true } diff --git a/IaC/modules/mlflow/secret_manager/variables.tf b/IaC/modules/mlflow/secret_manager/variables.tf index a262a7a..b995bb9 100644 --- a/IaC/modules/mlflow/secret_manager/variables.tf +++ b/IaC/modules/mlflow/secret_manager/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "secret_id" { type = string description = "Name of the secret you want to create" diff --git a/IaC/modules/mlflow/server/main.tf b/IaC/modules/mlflow/server/main.tf index 0928e97..0b55a17 100644 --- a/IaC/modules/mlflow/server/main.tf +++ b/IaC/modules/mlflow/server/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. locals { env_variables = merge( { @@ -23,32 +40,43 @@ resource "google_app_engine_application" "app" { } resource "google_project_iam_member" "cloudsql" { + depends_on = [google_app_engine_application.app] project = data.google_project.project.project_id role = "roles/cloudsql.client" - member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name) + member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id) } resource "google_project_iam_member" "secret" { + depends_on = [google_app_engine_application.app] project = data.google_project.project.project_id role = "roles/secretmanager.secretAccessor" - member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name) + member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id) } resource "google_project_iam_member" "gcs" { + depends_on = [google_app_engine_application.app] project = data.google_project.project.project_id role = "roles/storage.objectAdmin" member = format("serviceAccount:service-%s@gae-api-prod.google.com.iam.gserviceaccount.com", data.google_project.project.number) } +resource "google_project_iam_member" "gae_gcs" { + depends_on = [google_app_engine_application.app] + project = data.google_project.project.project_id + role = "roles/storage.objectViewer" + member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id) +} + resource "google_project_iam_member" "gae_api" { + depends_on = [google_app_engine_application.app] project = data.google_project.project.project_id role = "roles/compute.networkUser" - member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name) + member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id) } resource "google_app_engine_flexible_app_version" "myapp_v1" { service = var.service - version_id = "v1" + version_id = "v0" runtime = "custom" deployment { @@ -69,12 +97,13 @@ resource "google_app_engine_flexible_app_version" "myapp_v1" { automatic_scaling { cool_down_period = "120s" - max_total_instances = 1 - min_total_instances = 1 + max_total_instances = var.max_appengine_instances + min_total_instances = var.min_appengine_instances cpu_utilization { target_utilization = 0.5 } } + resources { cpu = 1 memory_gb = 2 @@ -88,7 +117,7 @@ resource "google_app_engine_flexible_app_version" "myapp_v1" { } noop_on_destroy = true - depends_on = [google_project_iam_member.gcs, google_project_iam_member.cloudsql, google_project_iam_member.secret, google_project_iam_member.gae_api] + depends_on = [google_project_iam_member.gcs, google_project_iam_member.gae_gcs, google_project_iam_member.cloudsql, google_project_iam_member.secret, google_project_iam_member.gae_api] } resource "google_iap_brand" "project_brand" { @@ -101,10 +130,9 @@ resource "google_iap_client" "project_client" { brand = google_iap_brand.project_brand.name } resource "google_iap_app_engine_service_iam_binding" "member" { - project = data.google_project.project.name - app_id = data.google_project.project.name + project = data.google_project.project.project_id + app_id = data.google_project.project.project_id service = google_app_engine_flexible_app_version.myapp_v1.service role = "roles/iap.httpsResourceAccessor" members = var.web_app_users - depends_on = [google_app_engine_flexible_app_version.myapp_v1] -} \ No newline at end of file +} diff --git a/IaC/modules/mlflow/server/variables.tf b/IaC/modules/mlflow/server/variables.tf index 7fcf354..fefa0d1 100644 --- a/IaC/modules/mlflow/server/variables.tf +++ b/IaC/modules/mlflow/server/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "server_name" { type = string description = "Name of your server" @@ -58,4 +75,16 @@ variable "web_app_users" { variable "service" { default = "default" } -variable "network_short_name" {} +variable "network_short_name" { + type = string +} +variable "max_appengine_instances" { + description = "The maximum number of app engine instances to scale up to" + type = number + default = 1 +} +variable "min_appengine_instances" { + description = "The minimum number of app engine instances to scale down to" + type = number + default = 1 +} diff --git a/IaC/modules/mlflow/variables.tf b/IaC/modules/mlflow/variables.tf index 6f98dc9..7774d5e 100644 --- a/IaC/modules/mlflow/variables.tf +++ b/IaC/modules/mlflow/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "artifacts_bucket_name" { description = "Name of the mlflow bucket created to store artifacts" type = string diff --git a/IaC/modules/network/main.tf b/IaC/modules/network/main.tf index f109a28..16e2bc1 100644 --- a/IaC/modules/network/main.tf +++ b/IaC/modules/network/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. resource "google_compute_network" "private_network" { count = length(var.network_name) > 0 ? 0 : 1 name = var.network_name_local diff --git a/IaC/modules/network/output.tf b/IaC/modules/network/output.tf deleted file mode 100644 index 4d9a7c9..0000000 --- a/IaC/modules/network/output.tf +++ /dev/null @@ -1,7 +0,0 @@ -output "network_self_link" { - value = data.google_compute_network.default_network.self_link -} - -output "network_short_name" { - value = data.google_compute_network.default_network.name -} \ No newline at end of file diff --git a/IaC/modules/network/outputs.tf b/IaC/modules/network/outputs.tf new file mode 100644 index 0000000..354a48c --- /dev/null +++ b/IaC/modules/network/outputs.tf @@ -0,0 +1,24 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +output "network_self_link" { + value = data.google_compute_network.default_network.self_link +} + +output "network_short_name" { + value = data.google_compute_network.default_network.name +} \ No newline at end of file diff --git a/IaC/modules/network/variables.tf b/IaC/modules/network/variables.tf index e84a075..76893af 100644 --- a/IaC/modules/network/variables.tf +++ b/IaC/modules/network/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "network_name" { type = string description = "Name of the network to attach to. If empty, a new network will be created" diff --git a/IaC/modules/services/main.tf b/IaC/modules/services/main.tf index 7891065..1aefb39 100644 --- a/IaC/modules/services/main.tf +++ b/IaC/modules/services/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. resource "google_project_service" "project" { count = length(var.services) project = var.project_id diff --git a/IaC/modules/services/variables.tf b/IaC/modules/services/variables.tf index d6d99bd..2e01e3a 100644 --- a/IaC/modules/services/variables.tf +++ b/IaC/modules/services/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "project_id" { description = "Id of your project" type = string diff --git a/IaC/prerequesites/main.tf b/IaC/prerequesites/main.tf index 299d188..5594afc 100644 --- a/IaC/prerequesites/main.tf +++ b/IaC/prerequesites/main.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. terraform { required_version = "~> 0.13.2" required_providers { @@ -13,6 +30,7 @@ module "services" { source = "./../modules/services" project_id = var.project_id services = [ + "cloudresourcemanager.googleapis.com", "container.googleapis.com", "servicenetworking.googleapis.com", "stackdriver.googleapis.com", @@ -30,4 +48,6 @@ module "bucket_backend" { bucket_location = var.backend_bucket_location number_of_version = var.backend_bucket_number_of_version storage_class = var.backend_bucket_storage_class + storage_uniform = var.storage_uniform + versioning_enabled = var.tfstate_versionning } \ No newline at end of file diff --git a/IaC/prerequesites/variables.tf b/IaC/prerequesites/variables.tf index 86b2805..ca7c170 100644 --- a/IaC/prerequesites/variables.tf +++ b/IaC/prerequesites/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "project_id" { description = "GCP project" type = string @@ -25,4 +42,14 @@ variable "backend_bucket_storage_class" { description = "Storage class of your bucket" type = string default ="STANDARD" +} +variable "storage_uniform" { + type = bool + description = "Wether or not uniform level acces is to be activated for the buckets" + default = true +} +variable "tfstate_versionning" { + type = bool + description = "Wether or not the remote TFstate should be versioned" + default = true } \ No newline at end of file diff --git a/IaC/variables.tf b/IaC/variables.tf index bfce900..fb02068 100644 --- a/IaC/variables.tf +++ b/IaC/variables.tf @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. variable "project_id" { description = "GCP project" type = string @@ -26,4 +43,9 @@ variable "web_app_users" { variable "network_name" { type = string description = "Name of the network to attach to. If empty, a new network will be created" +} +variable "storage_uniform" { + type = bool + description = "Wether or not uniform level acces is to be activated for the buckets" + default = true } \ No newline at end of file diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..153d416 --- /dev/null +++ b/LICENSE @@ -0,0 +1,165 @@ + GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. \ No newline at end of file diff --git a/README.md b/README.md index 4b0c071..01a8175 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,11 @@ A tool to deploy a mostly serverless MLflow on a GCP project with one command ## How to use ### Pre-requesites -- A GCP project -- Initialized gcloud SDK +- A GCP project on which you are owner +- Terraform >= 0.13.2 installed +- Initialized gcloud SDK with your owner account - Docker engine running +- No app engine application running and no consent screen already setup ### Deploying Fill out the `vars` file. @@ -14,15 +16,92 @@ Fill out the `vars` file. |Variable name|Description| |---|---| |`TF_VAR_project_id`|Name of the GCP project| -|`TF_VAR_backend_bucket`|Name of the terraform backend bucket. Should be unique. No `gs://` prefix| +|`TF_VAR_backend_bucket`|Name of the terraform backend bucket. Should be globally unique. No `gs://` prefix| |`TF_VAR_consent_screen_support_email`|Contact email address displayed by the SSO screen when the user trying to log in is not authorized. The address should be that of the user deploying mlflow (you) or a Cloud Identity group managed by this user| -|`TF_VAR_web_app_users`|List of authorized users/groups/domains. Should be a single quoted list of string such as '["user:jane@example.com", "group:people@example.com", "domain:example.com"]'| -|`TF_VAR_network_name`|The network the application and backend should attach to. If blank, a new network will be created.| +|`TF_VAR_web_app_users`|List of authorized users/groups/domains. Should be a single quoted list of string such as '["user:jane@example.com", "group:people@example.com", "domain:example.com"]'. Email addresses and domains must be associated with an active Google Account, G Suite account, or Cloud Identity account.| +|`TF_VAR_network_name`|The network the application and backend should attach to. If left blank, a new network will be created.| -Run `make one-clic-mlflow` and follow the prompts. +**Run `make one-click-mlflow` and follow the prompts.** + + +### What it does +- Enables the necessary services +- Builds and pushes the MLFlow docker image +- Creates a private IP CloudSQL (MySQL) database for the tracking server +- Creates an AppEngine Flex on the default service for the web UI, secured by IAP +- Manages all the network magic +- Creates the `mlflow-log-pusher` service account ### Other available make commands - `make deploy`: builds and pushes the application image and (re)deploys the infrastructure - `make docker`: builds and pushes the application image - `make apply`: (re)deploys the infrastructure - `make destroy`: destroys the infrastructure. **Will not delete the OAuth consent screen, and the app engine application** + + +### Pushing logs and artifacts + +You will need to specify the project id hosting the tracking server and the name of your MLFlow experiment: +- `export PROJECT_ID=` +- `export EXPERIMENT_NAME=` + +You may also need to get a service account key for `mlflow-log-pusher` if you lack the necessary permissions: +- `export GOOGLE_APPLICATION_CREDENTIALS=secrets/` + +To be able to push logs and artifacts to the tracking server, you will need to authenticate your request. +Simply paste the following snippet in your `config.py` or `__init__.py`. + +````python +import os + +import six +from mlflow import set_tracking_uri, set_experiment +from google.auth.transport.requests import Request +from google.oauth2 import id_token +import requests + + +def _get_client_id(service_uri): + redirect_response = requests.get(service_uri, allow_redirects=False) + if redirect_response.status_code != 302: + print(f"The URI {service_uri} does not seem to be a valid AppEngine endpoint.") + return None + + redirect_location = redirect_response.headers.get("location") + if not redirect_location: + print(f"No redirect location for request to {service_uri}") + return None + + parsed = six.moves.urllib.parse.urlparse(redirect_location) + query_string = six.moves.urllib.parse.parse_qs(parsed.query) + return query_string["client_id"][0] + + +PROJECT_ID = os.environ["PROJECT_ID"] +EXPERIMENT_NAME = os.environ["EXPERIMENT_NAME"] + +tracking_uri = f"https://{PROJECT_ID}.ew.r.appspot.com/" +client_id = _get_client_id(tracking_uri) +open_id_connect_token = id_token.fetch_id_token(Request(), client_id) +os.environ["MLFLOW_TRACKING_TOKEN"] = open_id_connect_token + +set_tracking_uri(tracking_uri) +set_experiment(EXPERIMENT_NAME) +```` + +You shoud then be able to push logs and artifacts with: +```python +import os +import mlflow + +# Log a parameter (key-value pair) +mlflow.log_param("param1", 42) + +# Log a metric; metrics can be updated throughout the run +mlflow.log_metric("foo", 42 + 1) +mlflow.log_metric("foo", 42 + 2) +mlflow.log_metric("foo", 42 + 3) + +# Log an artifact (output file) +mlflow.log_artifacts("artifact_file_path") +``` \ No newline at end of file diff --git a/config.py b/config.py deleted file mode 100644 index e69de29..0000000 diff --git a/lib/__init__.py b/lib/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/lib/entrypoint.py b/lib/entrypoint.py deleted file mode 100644 index 16cc9e4..0000000 --- a/lib/entrypoint.py +++ /dev/null @@ -1,2 +0,0 @@ -def run(): - raise NotImplementedError() diff --git a/secrets/.keep b/secrets/.keep deleted file mode 100644 index e69de29..0000000 diff --git a/tracking_server/run_tracking.sh b/tracking_server/run_tracking.sh index c2b33be..4fcf7bc 100644 --- a/tracking_server/run_tracking.sh +++ b/tracking_server/run_tracking.sh @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. #!/bin/bash DB_PASSWORD=$(gcloud beta secrets versions access --project=${GCP_PROJECT} --secret=${DB_PASSWORD_NAME} latest) diff --git a/tracking_server/tracking.Dockerfile b/tracking_server/tracking.Dockerfile index e6a3631..32e0de6 100644 --- a/tracking_server/tracking.Dockerfile +++ b/tracking_server/tracking.Dockerfile @@ -1,3 +1,20 @@ +# GNU Lesser General Public License v3.0 only +# Copyright (C) 2020 Artefact +# licence-information@artefact.com +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 3 of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. FROM python:3.7 WORKDIR /mlflow/ diff --git a/vars b/vars index 43044fd..4cab2cb 100644 --- a/vars +++ b/vars @@ -1,5 +1,5 @@ -export TF_VAR_project_id=two-click-mlflow -export TF_VAR_backend_bucket=tfstate-mlflow-atf-two -export TF_VAR_consent_screen_support_email=alexis.vialaret@artefact.com -export TF_VAR_web_app_users='["user:alexis.vialaret@artefact.com", "user:thomas.griseau@artefact.com"]' -export TF_VAR_network_name=default \ No newline at end of file +export TF_VAR_project_id= +export TF_VAR_backend_bucket= +export TF_VAR_consent_screen_support_email= +export TF_VAR_web_app_users= +export TF_VAR_network_name= \ No newline at end of file