diff --git a/.dockerignore b/.dockerignore
new file mode 100644
index 0000000..3c84e16
--- /dev/null
+++ b/.dockerignore
@@ -0,0 +1,5 @@
+#terraform
+.terraform
+*.tfstate
+*.tfstate.backup
+*.tfvars
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 0000000..94ddb90
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,31 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 0000000..bbcbbe7
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000..06e1527
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,27 @@
+Make sure you have checked _all_ steps below.
+
+### Issue
+
+- [ ] My PR addresses the following Issue (If an issue exists in JIRA or Github)
+
+### Description
+
+- [ ] Here are some details about my PR, including example of usage.
+
+### Licence
+
+- [ ] My PR adds the needed licence header to every added file:
+
+### Commits
+
+- [ ] I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
+ 1. Subject is separated from body by a blank line
+ 2. Subject is limited to 50 characters
+ 3. Subject does not end with a period
+ 4. Subject uses the imperative mood ("add", not "adding")
+ 5. Body explains "what" and "why", not "how"
+
+### Documentation
+
+- [ ] In case of new functionality, my PR adds documentation that describes how to use it.
+ - All the public functions and the classes in the PR contain docstrings that explain what it does
\ No newline at end of file
diff --git a/.github/workflows/gitleaks.yml b/.github/workflows/gitleaks.yml
new file mode 100644
index 0000000..849a02d
--- /dev/null
+++ b/.github/workflows/gitleaks.yml
@@ -0,0 +1,16 @@
+name: gitleaks
+
+on:
+ push:
+ pull_request:
+ branches:
+ - master
+ - dev
+
+jobs:
+ gitleaks:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v1
+ - name: gitleaks-action
+ uses: zricethezav/gitleaks-action@master
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 0000000..7aa04cb
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,32 @@
+# Contributing to one-click-mlflow
+We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's:
+
+- Reporting a bug
+- Discussing the current state of the code
+- Submitting a fix
+- Proposing new features
+- Becoming a maintainer
+
+## We Develop with Github
+We use github to host code, to track issues and feature requests, as well as accept pull requests.
+
+## We Use [Github Flow](https://guides.github.com/introduction/flow/index.html), So All Code Changes Happen Through Pull Requests
+Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests:
+
+1. Fork the repo and create your branch from `master`.
+2. If needed, update the documentation.
+3. Make sure your code works.
+4. Don't forget to add licence header to every file if you added any.
+5. Issue that pull request!
+
+## Report bugs using Github's [issues](https://github.com/briandk/transcriptase-atom/issues)
+We use GitHub issues to track public bugs. Report a bug by [opening a new issue](); it's that easy!
+
+
+**Great Bug Reports** :
+
+Follow the GitHub issue template and give as many informations as you can.
+
+## License
+
+By contributing, you agree that your contributions will be licensed under its GNU LESSER GENERAL PUBLIC LICENSE.
\ No newline at end of file
diff --git a/IaC/main.tf b/IaC/main.tf
index 1deb48c..c00b279 100644
--- a/IaC/main.tf
+++ b/IaC/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
terraform {
backend "gcs" {
}
@@ -35,3 +52,9 @@ module "mlflow" {
network_self_link = module.network.network_self_link
network_short_name = module.network.network_short_name
}
+
+module "log_pusher" {
+ source = "./modules/mlflow/log_pusher"
+ project_id = var.project_id
+ depends_on = [module.mlflow]
+}
diff --git a/IaC/modules/mlflow/artifacts/main.tf b/IaC/modules/mlflow/artifacts/main.tf
index e7c0fbd..0df49cc 100644
--- a/IaC/modules/mlflow/artifacts/main.tf
+++ b/IaC/modules/mlflow/artifacts/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
resource "google_storage_bucket" "this" {
name = var.bucket_name
location = var.bucket_location
@@ -13,4 +30,5 @@ resource "google_storage_bucket" "this" {
type = "Delete"
}
}
+ uniform_bucket_level_access = var.storage_uniform
}
diff --git a/IaC/modules/mlflow/artifacts/outputs.tf b/IaC/modules/mlflow/artifacts/outputs.tf
index c7af138..b2cb89a 100644
--- a/IaC/modules/mlflow/artifacts/outputs.tf
+++ b/IaC/modules/mlflow/artifacts/outputs.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
output "url" {
description = "gcs uri"
value = google_storage_bucket.this.url
diff --git a/IaC/modules/mlflow/artifacts/variables.tf b/IaC/modules/mlflow/artifacts/variables.tf
index ec992d4..ddb6cc5 100644
--- a/IaC/modules/mlflow/artifacts/variables.tf
+++ b/IaC/modules/mlflow/artifacts/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "bucket_name" {
description = "Name of the bucket."
type = string
@@ -26,3 +43,8 @@ variable "module_depends_on" {
type = any
default = null
}
+variable "storage_uniform" {
+ type = bool
+ description = "Wether or not uniform level acces is to be activated for the buckets"
+ default = true
+}
diff --git a/IaC/modules/mlflow/database/main.tf b/IaC/modules/mlflow/database/main.tf
index 167e7e7..b948e59 100644
--- a/IaC/modules/mlflow/database/main.tf
+++ b/IaC/modules/mlflow/database/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
resource "random_id" "db_name_suffix" {
byte_length = 5
}
diff --git a/IaC/modules/mlflow/database/outputs.tf b/IaC/modules/mlflow/database/outputs.tf
index 093dafd..8fb7675 100644
--- a/IaC/modules/mlflow/database/outputs.tf
+++ b/IaC/modules/mlflow/database/outputs.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
output "instance_connection_name" {
description = "Connection string used to connect to the instance"
value = google_sql_database_instance.this_instance.connection_name
diff --git a/IaC/modules/mlflow/database/variables.tf b/IaC/modules/mlflow/database/variables.tf
index 6423cef..8f2370a 100644
--- a/IaC/modules/mlflow/database/variables.tf
+++ b/IaC/modules/mlflow/database/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "instance_prefix" {
type = string
description = "Name of the database instance you want to deploy"
diff --git a/IaC/modules/mlflow/log_pusher/main.tf b/IaC/modules/mlflow/log_pusher/main.tf
new file mode 100644
index 0000000..0328a67
--- /dev/null
+++ b/IaC/modules/mlflow/log_pusher/main.tf
@@ -0,0 +1,33 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+resource "google_service_account" "log_pusher" {
+ account_id = "mlflow-log-pusher"
+ display_name = "mlflow log pusher"
+}
+
+resource "google_project_iam_member" "log_pusher_iap" {
+ project = var.project_id
+ role = "roles/iap.httpsResourceAccessor"
+ member = "serviceAccount:${google_service_account.log_pusher.email}"
+}
+
+resource "google_project_iam_member" "log_pusher_storage" {
+ project = var.project_id
+ role = "roles/storage.objectAdmin"
+ member = "serviceAccount:${google_service_account.log_pusher.email}"
+}
\ No newline at end of file
diff --git a/IaC/modules/mlflow/log_pusher/variables.tf b/IaC/modules/mlflow/log_pusher/variables.tf
new file mode 100644
index 0000000..adf7a33
--- /dev/null
+++ b/IaC/modules/mlflow/log_pusher/variables.tf
@@ -0,0 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+variable "project_id" {
+ type = string
+}
\ No newline at end of file
diff --git a/IaC/modules/mlflow/main.tf b/IaC/modules/mlflow/main.tf
index da253f6..583298b 100644
--- a/IaC/modules/mlflow/main.tf
+++ b/IaC/modules/mlflow/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
module "artifacts" {
source = "./artifacts"
bucket_name = var.artifacts_bucket_name
diff --git a/IaC/modules/mlflow/secret_manager/main.tf b/IaC/modules/mlflow/secret_manager/main.tf
index 8b82ef4..c6e7b05 100644
--- a/IaC/modules/mlflow/secret_manager/main.tf
+++ b/IaC/modules/mlflow/secret_manager/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
resource "google_secret_manager_secret" "secret" {
provider = google-beta
diff --git a/IaC/modules/mlflow/secret_manager/outputs.tf b/IaC/modules/mlflow/secret_manager/outputs.tf
index 2ffa9b4..64b0700 100644
--- a/IaC/modules/mlflow/secret_manager/outputs.tf
+++ b/IaC/modules/mlflow/secret_manager/outputs.tf
@@ -1,4 +1,22 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
output "secret_value" {
description = "Value of the created secret"
value = google_secret_manager_secret_version.secret-version.secret_data
+ sensitive = true
}
diff --git a/IaC/modules/mlflow/secret_manager/variables.tf b/IaC/modules/mlflow/secret_manager/variables.tf
index a262a7a..b995bb9 100644
--- a/IaC/modules/mlflow/secret_manager/variables.tf
+++ b/IaC/modules/mlflow/secret_manager/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "secret_id" {
type = string
description = "Name of the secret you want to create"
diff --git a/IaC/modules/mlflow/server/main.tf b/IaC/modules/mlflow/server/main.tf
index 0928e97..0b55a17 100644
--- a/IaC/modules/mlflow/server/main.tf
+++ b/IaC/modules/mlflow/server/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
locals {
env_variables = merge(
{
@@ -23,32 +40,43 @@ resource "google_app_engine_application" "app" {
}
resource "google_project_iam_member" "cloudsql" {
+ depends_on = [google_app_engine_application.app]
project = data.google_project.project.project_id
role = "roles/cloudsql.client"
- member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name)
+ member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id)
}
resource "google_project_iam_member" "secret" {
+ depends_on = [google_app_engine_application.app]
project = data.google_project.project.project_id
role = "roles/secretmanager.secretAccessor"
- member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name)
+ member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id)
}
resource "google_project_iam_member" "gcs" {
+ depends_on = [google_app_engine_application.app]
project = data.google_project.project.project_id
role = "roles/storage.objectAdmin"
member = format("serviceAccount:service-%s@gae-api-prod.google.com.iam.gserviceaccount.com", data.google_project.project.number)
}
+resource "google_project_iam_member" "gae_gcs" {
+ depends_on = [google_app_engine_application.app]
+ project = data.google_project.project.project_id
+ role = "roles/storage.objectViewer"
+ member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id)
+}
+
resource "google_project_iam_member" "gae_api" {
+ depends_on = [google_app_engine_application.app]
project = data.google_project.project.project_id
role = "roles/compute.networkUser"
- member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.name)
+ member = format("serviceAccount:%s@appspot.gserviceaccount.com", data.google_project.project.project_id)
}
resource "google_app_engine_flexible_app_version" "myapp_v1" {
service = var.service
- version_id = "v1"
+ version_id = "v0"
runtime = "custom"
deployment {
@@ -69,12 +97,13 @@ resource "google_app_engine_flexible_app_version" "myapp_v1" {
automatic_scaling {
cool_down_period = "120s"
- max_total_instances = 1
- min_total_instances = 1
+ max_total_instances = var.max_appengine_instances
+ min_total_instances = var.min_appengine_instances
cpu_utilization {
target_utilization = 0.5
}
}
+
resources {
cpu = 1
memory_gb = 2
@@ -88,7 +117,7 @@ resource "google_app_engine_flexible_app_version" "myapp_v1" {
}
noop_on_destroy = true
- depends_on = [google_project_iam_member.gcs, google_project_iam_member.cloudsql, google_project_iam_member.secret, google_project_iam_member.gae_api]
+ depends_on = [google_project_iam_member.gcs, google_project_iam_member.gae_gcs, google_project_iam_member.cloudsql, google_project_iam_member.secret, google_project_iam_member.gae_api]
}
resource "google_iap_brand" "project_brand" {
@@ -101,10 +130,9 @@ resource "google_iap_client" "project_client" {
brand = google_iap_brand.project_brand.name
}
resource "google_iap_app_engine_service_iam_binding" "member" {
- project = data.google_project.project.name
- app_id = data.google_project.project.name
+ project = data.google_project.project.project_id
+ app_id = data.google_project.project.project_id
service = google_app_engine_flexible_app_version.myapp_v1.service
role = "roles/iap.httpsResourceAccessor"
members = var.web_app_users
- depends_on = [google_app_engine_flexible_app_version.myapp_v1]
-}
\ No newline at end of file
+}
diff --git a/IaC/modules/mlflow/server/variables.tf b/IaC/modules/mlflow/server/variables.tf
index 7fcf354..fefa0d1 100644
--- a/IaC/modules/mlflow/server/variables.tf
+++ b/IaC/modules/mlflow/server/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "server_name" {
type = string
description = "Name of your server"
@@ -58,4 +75,16 @@ variable "web_app_users" {
variable "service" {
default = "default"
}
-variable "network_short_name" {}
+variable "network_short_name" {
+ type = string
+}
+variable "max_appengine_instances" {
+ description = "The maximum number of app engine instances to scale up to"
+ type = number
+ default = 1
+}
+variable "min_appengine_instances" {
+ description = "The minimum number of app engine instances to scale down to"
+ type = number
+ default = 1
+}
diff --git a/IaC/modules/mlflow/variables.tf b/IaC/modules/mlflow/variables.tf
index 6f98dc9..7774d5e 100644
--- a/IaC/modules/mlflow/variables.tf
+++ b/IaC/modules/mlflow/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "artifacts_bucket_name" {
description = "Name of the mlflow bucket created to store artifacts"
type = string
diff --git a/IaC/modules/network/main.tf b/IaC/modules/network/main.tf
index f109a28..16e2bc1 100644
--- a/IaC/modules/network/main.tf
+++ b/IaC/modules/network/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
resource "google_compute_network" "private_network" {
count = length(var.network_name) > 0 ? 0 : 1
name = var.network_name_local
diff --git a/IaC/modules/network/output.tf b/IaC/modules/network/output.tf
deleted file mode 100644
index 4d9a7c9..0000000
--- a/IaC/modules/network/output.tf
+++ /dev/null
@@ -1,7 +0,0 @@
-output "network_self_link" {
- value = data.google_compute_network.default_network.self_link
-}
-
-output "network_short_name" {
- value = data.google_compute_network.default_network.name
-}
\ No newline at end of file
diff --git a/IaC/modules/network/outputs.tf b/IaC/modules/network/outputs.tf
new file mode 100644
index 0000000..354a48c
--- /dev/null
+++ b/IaC/modules/network/outputs.tf
@@ -0,0 +1,24 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+output "network_self_link" {
+ value = data.google_compute_network.default_network.self_link
+}
+
+output "network_short_name" {
+ value = data.google_compute_network.default_network.name
+}
\ No newline at end of file
diff --git a/IaC/modules/network/variables.tf b/IaC/modules/network/variables.tf
index e84a075..76893af 100644
--- a/IaC/modules/network/variables.tf
+++ b/IaC/modules/network/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "network_name" {
type = string
description = "Name of the network to attach to. If empty, a new network will be created"
diff --git a/IaC/modules/services/main.tf b/IaC/modules/services/main.tf
index 7891065..1aefb39 100644
--- a/IaC/modules/services/main.tf
+++ b/IaC/modules/services/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
resource "google_project_service" "project" {
count = length(var.services)
project = var.project_id
diff --git a/IaC/modules/services/variables.tf b/IaC/modules/services/variables.tf
index d6d99bd..2e01e3a 100644
--- a/IaC/modules/services/variables.tf
+++ b/IaC/modules/services/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "project_id" {
description = "Id of your project"
type = string
diff --git a/IaC/prerequesites/main.tf b/IaC/prerequesites/main.tf
index 299d188..5594afc 100644
--- a/IaC/prerequesites/main.tf
+++ b/IaC/prerequesites/main.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
terraform {
required_version = "~> 0.13.2"
required_providers {
@@ -13,6 +30,7 @@ module "services" {
source = "./../modules/services"
project_id = var.project_id
services = [
+ "cloudresourcemanager.googleapis.com",
"container.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
@@ -30,4 +48,6 @@ module "bucket_backend" {
bucket_location = var.backend_bucket_location
number_of_version = var.backend_bucket_number_of_version
storage_class = var.backend_bucket_storage_class
+ storage_uniform = var.storage_uniform
+ versioning_enabled = var.tfstate_versionning
}
\ No newline at end of file
diff --git a/IaC/prerequesites/variables.tf b/IaC/prerequesites/variables.tf
index 86b2805..ca7c170 100644
--- a/IaC/prerequesites/variables.tf
+++ b/IaC/prerequesites/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "project_id" {
description = "GCP project"
type = string
@@ -25,4 +42,14 @@ variable "backend_bucket_storage_class" {
description = "Storage class of your bucket"
type = string
default ="STANDARD"
+}
+variable "storage_uniform" {
+ type = bool
+ description = "Wether or not uniform level acces is to be activated for the buckets"
+ default = true
+}
+variable "tfstate_versionning" {
+ type = bool
+ description = "Wether or not the remote TFstate should be versioned"
+ default = true
}
\ No newline at end of file
diff --git a/IaC/variables.tf b/IaC/variables.tf
index bfce900..fb02068 100644
--- a/IaC/variables.tf
+++ b/IaC/variables.tf
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
variable "project_id" {
description = "GCP project"
type = string
@@ -26,4 +43,9 @@ variable "web_app_users" {
variable "network_name" {
type = string
description = "Name of the network to attach to. If empty, a new network will be created"
+}
+variable "storage_uniform" {
+ type = bool
+ description = "Wether or not uniform level acces is to be activated for the buckets"
+ default = true
}
\ No newline at end of file
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..153d416
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,165 @@
+ GNU LESSER GENERAL PUBLIC LICENSE
+ Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc.
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+
+ This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+ 0. Additional Definitions.
+
+ As used herein, "this License" refers to version 3 of the GNU Lesser
+General Public License, and the "GNU GPL" refers to version 3 of the GNU
+General Public License.
+
+ "The Library" refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+ An "Application" is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+ A "Combined Work" is a work produced by combining or linking an
+Application with the Library. The particular version of the Library
+with which the Combined Work was made is also called the "Linked
+Version".
+
+ The "Minimal Corresponding Source" for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+ The "Corresponding Application Code" for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+ 1. Exception to Section 3 of the GNU GPL.
+
+ You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+ 2. Conveying Modified Versions.
+
+ If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+ a) under this License, provided that you make a good faith effort to
+ ensure that, in the event an Application does not supply the
+ function or data, the facility still operates, and performs
+ whatever part of its purpose remains meaningful, or
+
+ b) under the GNU GPL, with none of the additional permissions of
+ this License applicable to that copy.
+
+ 3. Object Code Incorporating Material from Library Header Files.
+
+ The object code form of an Application may incorporate material from
+a header file that is part of the Library. You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+ a) Give prominent notice with each copy of the object code that the
+ Library is used in it and that the Library and its use are
+ covered by this License.
+
+ b) Accompany the object code with a copy of the GNU GPL and this license
+ document.
+
+ 4. Combined Works.
+
+ You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+ a) Give prominent notice with each copy of the Combined Work that
+ the Library is used in it and that the Library and its use are
+ covered by this License.
+
+ b) Accompany the Combined Work with a copy of the GNU GPL and this license
+ document.
+
+ c) For a Combined Work that displays copyright notices during
+ execution, include the copyright notice for the Library among
+ these notices, as well as a reference directing the user to the
+ copies of the GNU GPL and this license document.
+
+ d) Do one of the following:
+
+ 0) Convey the Minimal Corresponding Source under the terms of this
+ License, and the Corresponding Application Code in a form
+ suitable for, and under terms that permit, the user to
+ recombine or relink the Application with a modified version of
+ the Linked Version to produce a modified Combined Work, in the
+ manner specified by section 6 of the GNU GPL for conveying
+ Corresponding Source.
+
+ 1) Use a suitable shared library mechanism for linking with the
+ Library. A suitable mechanism is one that (a) uses at run time
+ a copy of the Library already present on the user's computer
+ system, and (b) will operate properly with a modified version
+ of the Library that is interface-compatible with the Linked
+ Version.
+
+ e) Provide Installation Information, but only if you would otherwise
+ be required to provide such information under section 6 of the
+ GNU GPL, and only to the extent that such information is
+ necessary to install and execute a modified version of the
+ Combined Work produced by recombining or relinking the
+ Application with a modified version of the Linked Version. (If
+ you use option 4d0, the Installation Information must accompany
+ the Minimal Corresponding Source and Corresponding Application
+ Code. If you use option 4d1, you must provide the Installation
+ Information in the manner specified by section 6 of the GNU GPL
+ for conveying Corresponding Source.)
+
+ 5. Combined Libraries.
+
+ You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+ a) Accompany the combined library with a copy of the same work based
+ on the Library, uncombined with any other library facilities,
+ conveyed under the terms of this License.
+
+ b) Give prominent notice with the combined library that part of it
+ is a work based on the Library, and explaining where to find the
+ accompanying uncombined form of the same work.
+
+ 6. Revised Versions of the GNU Lesser General Public License.
+
+ The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License "or any later version"
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+ If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.
\ No newline at end of file
diff --git a/README.md b/README.md
index 4b0c071..01a8175 100644
--- a/README.md
+++ b/README.md
@@ -4,9 +4,11 @@ A tool to deploy a mostly serverless MLflow on a GCP project with one command
## How to use
### Pre-requesites
-- A GCP project
-- Initialized gcloud SDK
+- A GCP project on which you are owner
+- Terraform >= 0.13.2 installed
+- Initialized gcloud SDK with your owner account
- Docker engine running
+- No app engine application running and no consent screen already setup
### Deploying
Fill out the `vars` file.
@@ -14,15 +16,92 @@ Fill out the `vars` file.
|Variable name|Description|
|---|---|
|`TF_VAR_project_id`|Name of the GCP project|
-|`TF_VAR_backend_bucket`|Name of the terraform backend bucket. Should be unique. No `gs://` prefix|
+|`TF_VAR_backend_bucket`|Name of the terraform backend bucket. Should be globally unique. No `gs://` prefix|
|`TF_VAR_consent_screen_support_email`|Contact email address displayed by the SSO screen when the user trying to log in is not authorized. The address should be that of the user deploying mlflow (you) or a Cloud Identity group managed by this user|
-|`TF_VAR_web_app_users`|List of authorized users/groups/domains. Should be a single quoted list of string such as '["user:jane@example.com", "group:people@example.com", "domain:example.com"]'|
-|`TF_VAR_network_name`|The network the application and backend should attach to. If blank, a new network will be created.|
+|`TF_VAR_web_app_users`|List of authorized users/groups/domains. Should be a single quoted list of string such as '["user:jane@example.com", "group:people@example.com", "domain:example.com"]'. Email addresses and domains must be associated with an active Google Account, G Suite account, or Cloud Identity account.|
+|`TF_VAR_network_name`|The network the application and backend should attach to. If left blank, a new network will be created.|
-Run `make one-clic-mlflow` and follow the prompts.
+**Run `make one-click-mlflow` and follow the prompts.**
+
+
+### What it does
+- Enables the necessary services
+- Builds and pushes the MLFlow docker image
+- Creates a private IP CloudSQL (MySQL) database for the tracking server
+- Creates an AppEngine Flex on the default service for the web UI, secured by IAP
+- Manages all the network magic
+- Creates the `mlflow-log-pusher` service account
### Other available make commands
- `make deploy`: builds and pushes the application image and (re)deploys the infrastructure
- `make docker`: builds and pushes the application image
- `make apply`: (re)deploys the infrastructure
- `make destroy`: destroys the infrastructure. **Will not delete the OAuth consent screen, and the app engine application**
+
+
+### Pushing logs and artifacts
+
+You will need to specify the project id hosting the tracking server and the name of your MLFlow experiment:
+- `export PROJECT_ID=`
+- `export EXPERIMENT_NAME=`
+
+You may also need to get a service account key for `mlflow-log-pusher` if you lack the necessary permissions:
+- `export GOOGLE_APPLICATION_CREDENTIALS=secrets/`
+
+To be able to push logs and artifacts to the tracking server, you will need to authenticate your request.
+Simply paste the following snippet in your `config.py` or `__init__.py`.
+
+````python
+import os
+
+import six
+from mlflow import set_tracking_uri, set_experiment
+from google.auth.transport.requests import Request
+from google.oauth2 import id_token
+import requests
+
+
+def _get_client_id(service_uri):
+ redirect_response = requests.get(service_uri, allow_redirects=False)
+ if redirect_response.status_code != 302:
+ print(f"The URI {service_uri} does not seem to be a valid AppEngine endpoint.")
+ return None
+
+ redirect_location = redirect_response.headers.get("location")
+ if not redirect_location:
+ print(f"No redirect location for request to {service_uri}")
+ return None
+
+ parsed = six.moves.urllib.parse.urlparse(redirect_location)
+ query_string = six.moves.urllib.parse.parse_qs(parsed.query)
+ return query_string["client_id"][0]
+
+
+PROJECT_ID = os.environ["PROJECT_ID"]
+EXPERIMENT_NAME = os.environ["EXPERIMENT_NAME"]
+
+tracking_uri = f"https://{PROJECT_ID}.ew.r.appspot.com/"
+client_id = _get_client_id(tracking_uri)
+open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
+os.environ["MLFLOW_TRACKING_TOKEN"] = open_id_connect_token
+
+set_tracking_uri(tracking_uri)
+set_experiment(EXPERIMENT_NAME)
+````
+
+You shoud then be able to push logs and artifacts with:
+```python
+import os
+import mlflow
+
+# Log a parameter (key-value pair)
+mlflow.log_param("param1", 42)
+
+# Log a metric; metrics can be updated throughout the run
+mlflow.log_metric("foo", 42 + 1)
+mlflow.log_metric("foo", 42 + 2)
+mlflow.log_metric("foo", 42 + 3)
+
+# Log an artifact (output file)
+mlflow.log_artifacts("artifact_file_path")
+```
\ No newline at end of file
diff --git a/config.py b/config.py
deleted file mode 100644
index e69de29..0000000
diff --git a/lib/__init__.py b/lib/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/lib/entrypoint.py b/lib/entrypoint.py
deleted file mode 100644
index 16cc9e4..0000000
--- a/lib/entrypoint.py
+++ /dev/null
@@ -1,2 +0,0 @@
-def run():
- raise NotImplementedError()
diff --git a/secrets/.keep b/secrets/.keep
deleted file mode 100644
index e69de29..0000000
diff --git a/tracking_server/run_tracking.sh b/tracking_server/run_tracking.sh
index c2b33be..4fcf7bc 100644
--- a/tracking_server/run_tracking.sh
+++ b/tracking_server/run_tracking.sh
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#!/bin/bash
DB_PASSWORD=$(gcloud beta secrets versions access --project=${GCP_PROJECT} --secret=${DB_PASSWORD_NAME} latest)
diff --git a/tracking_server/tracking.Dockerfile b/tracking_server/tracking.Dockerfile
index e6a3631..32e0de6 100644
--- a/tracking_server/tracking.Dockerfile
+++ b/tracking_server/tracking.Dockerfile
@@ -1,3 +1,20 @@
+# GNU Lesser General Public License v3.0 only
+# Copyright (C) 2020 Artefact
+# licence-information@artefact.com
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
FROM python:3.7
WORKDIR /mlflow/
diff --git a/vars b/vars
index 43044fd..4cab2cb 100644
--- a/vars
+++ b/vars
@@ -1,5 +1,5 @@
-export TF_VAR_project_id=two-click-mlflow
-export TF_VAR_backend_bucket=tfstate-mlflow-atf-two
-export TF_VAR_consent_screen_support_email=alexis.vialaret@artefact.com
-export TF_VAR_web_app_users='["user:alexis.vialaret@artefact.com", "user:thomas.griseau@artefact.com"]'
-export TF_VAR_network_name=default
\ No newline at end of file
+export TF_VAR_project_id=
+export TF_VAR_backend_bucket=
+export TF_VAR_consent_screen_support_email=
+export TF_VAR_web_app_users=
+export TF_VAR_network_name=
\ No newline at end of file