Update mission-control version to 3.5.0

Move elasticsearch into mission-control pod
HA support for mission-control
HA support for elasticsearch
Upgrade to elasticsearch-oss 6.6.0 with searchguard

stable/mission-control/Chart.yaml

@@ -1,8 +1,8 @@
 apiVersion: v1
 name: mission-control
 description: A Helm chart for JFrog Mission Control
-version: 0.9.4
-appVersion: 3.4.3
+version: 1.0.0
+appVersion: 3.5.0
 home: https://jfrog.com/mission-control/
 icon: https://raw.githubusercontent.com/JFrogDev/artifactory-dcos/master/images/jfrog_med.png
 keywords:

stable/mission-control/README.md

@@ -29,6 +29,23 @@ helm repo add jfrog https://charts.jfrog.io
 ```bash
 helm install --name mission-control jfrog/mission-control
 ```
+### Create a unique MC Key
+Mission Control HA cluster uses a unique mc key. By default the chart has one set in values.yaml (`missionControl.mcKey`).
+
+**This key is for demo purpose and should not be used in a production environment!**
+
+You should generate a unique one and pass it to the template at install/upgrade time.
+```bash
+# Create a key
+export MC_KEY=$(openssl rand -hex 16)
+echo ${MC_KEY}
+
+# Pass the created master key to helm
+helm install --name mission-control --set missionControl.mcKey=${MC_KEY} jfrog/mission-control
+```
+
+**NOTE:** Make sure to pass the same mc key on all future calls to `helm install` and `helm upgrade`! In the first case, this means always passing `--set missionControl.mcKey=${MC_KEY}`.
+
 
 ## Set Mission Control base URL
 * Get mission-control url by running following commands:
@@ -45,13 +62,62 @@ helm upgrade --name mission-control --set missionControl.missionControlUrl=$MISS
 **NOTE:** It might take a few minutes for Mission Control's public IP to become available, and the nodes to complete initial setup.
 Follow the instructions outputted by the install command to get the Mission Control IP and URL to access it.
 
-### Updating Mission Control
+## Upgrade
 Once you have a new chart version, you can update your deployment with
 ```
 helm upgrade mission-control jfrog/mission-control
 ```
 
-### Use an external Database
+**NOTE:** Check for any version specific upgrade nodes in [CHANGELOG.md]
+
+### Non compatible upgrades
+In cases where a new version is not compatible with existing deployed version (look in CHANGELOG.md) you should
+* Deploy new version along side old version (set a new release name)
+* Copy configurations and data from old deployment to new one (The following instructions were tested for chart migration from 0.9.x to 1.0.0)
+```Copy data and config from old deployment to local filesystem :
+kubectl cp <elasticsearch-pod>:/usr/share/elasticsearch/data                                   /<local_disk_path>/mission-control-data/elastic_data               -n <old_namespace>
+kubectl cp <postgres-pod>:/var/lib/postgresql/data                                             /<local_disk_path>/mission-control-data/postgres_data              -n <old_namespace>
+kubectl cp <mission-control-pod>:/var/opt/jfrog/mission-control/etc/mission-control.properties /<local_disk_path>/mission-control-data/mission-control.properties -n <old_namespace> -c mission-control
+kubectl cp <mission-control-pod>:/var/opt/jfrog/mission-control/data/security/mc.key           /<local_disk_path>/mission-control-data/mc.key                     -n <old_namespace> -c mission-control
+```
+```Copy data and config from local filesystem to new deployment :
+kubectl cp /<local_disk_path>/mission-control-data/mc.key                     <mission-control-pod>:/var/opt/jfrog/mission-control/data/security/mc.key            -n <new_namespace> -c mission-control
+kubectl cp /<local_disk_path>/mission-control-data/mission-control.properties <mission-control-pod>:/var/opt/jfrog/mission-control/etc/mission-control.properties  -n <new_namespace> -c mission-control
+kubectl cp /<local_disk_path>/mission-control-data/elastic_data               <mission-control-pod>:/usr/share/elasticsearch                                       -n <new_namespace> -c elasticsearch
+kubectl cp /<local_disk_path>/mission-control-data/postgres_data              <postgres-pod>:/var/lib/postgresql                                                   -n <new_namespace>
+
+kubectl exec -it <postgres-pod> -n <new_namespace> -- bash
+    rm -fr /var/lib/postgresql/data
+    cp -fr /var/lib/postgresql/postgres_data/* /var/lib/postgresql/data/
+    rm -fr /var/lib/postgresql/postgres_data
+kubectl exec -it <mission-control-pod> -n <new_namespace> -c elasticsearch -- bash
+    rm -fr /usr/share/elasticsearch/data
+    cp -fr /usr/share/elasticsearch/elastic_data/* /usr/share/elasticsearch/data
+    rm -fr /usr/share/elasticsearch/elastic_data
+```
+* Restart the new deployment
+```bash
+kubectl scale deployment <postgres-deployment> --replicas=0 -n <new_namespace>
+kubectl scale statefulset <mission-control-statefulset> --replicas=0 -n <new_namespace>
+
+kubectl scale deployment <postgres-deployment> --replicas=1 -n <new_namespace>
+kubectl scale statefulset <mission-control-statefulset> --replicas=1 -n <new_namespace>
+```
+* Once the new release is up and ready, update mission-control base url with new DNS
+  * Login to mission-control pod,
+```bash
+kubectl exec -it <mission-control-pod> -n <new_namespace> -c mission-control -- bash
+```
+  * Update mission-control base url by running the api from [Mission Control Rest API](https://www.jfrog.com/confluence/display/MC/Mission+Control+REST+API#MissionControlRESTAPI-UpdateBaseURL)
+* A new mc.key will be generated after this upgrade, save a copy of this key. **NOTE**: This should be passed on all future calls to `helm install` and `helm upgrade`!
+```bash
+export MC_KEY=$(kubectl exec -it <mission-control-pod> -n <new_namespace> -c mission-control -- cat /var/opt/jfrog/mission-control/data/security/mc.key)
+```
+* Remove old release
+
+### Use external Database
+
+#### PostgreSQL
 There are cases where you will want to use an external **PostgreSQL** and not the enclosed **PostgreSQL**.
 See more details on [configuring the database](https://www.jfrog.com/confluence/display/MC/Using+External+Databases#UsingExternalDatabases-ExternalizingPostgreSQL)
 
@@ -67,7 +133,7 @@ This can be done with the following parameters
 ```
 **NOTE:** You must set `postgresql.enabled=false` in order for the chart to use the `database.*` parameters. Without it, they will be ignored!
 
-#### Use existing secrets for PostgreSQL connection details
+##### Use existing secrets for PostgreSQL connection details
 You can use already existing secrets for managing the database connection details.
 
 Pass them to the install command with the following parameters
@@ -84,6 +150,20 @@ export POSTGRES_PASSWORD_SECRET_KEY=
 ...
 ```
 
+##### Elasticsearch
+
+There are cases where you will want to use an external **Elasticsearch** and not the enclosed **Elasticsearch**.
+
+This can be done with the following parameters
+```bash
+...
+--set elasticsearch.enabled=false \
+--set elasticsearch.url=${ES_URL} \
+--set elasticsearch.username=${ES_USERNAME} \
+--set elasticsearch.password=${ES_PASSWORD} \
+...
+```
+
 ### Logger sidecars
 This chart provides the option to add sidecars to tail various logs from Mission Control containers. See the available values in `values.yaml`
 
@@ -116,28 +196,11 @@ The following table lists the configurable parameters of the mission-control cha
 | `initContainerImage`                         | Init Container Image                            | `alpine:3.6`                          |
 | `imagePullPolicy`                            | Container pull policy                           | `IfNotPresent`                        |
 | `imagePullSecrets`                           | Docker registry pull secret                     |                                       |
+| `replicaCount`                               | Number of replicas                              | `1`                                   |
 | `serviceAccount.create`                      | Specifies whether a ServiceAccount should be created | `true`                           |
 | `serviceAccount.name`                        | The name of the ServiceAccount to create             | Generated using the fullname template |
 | `rbac.create`                                | Specifies whether RBAC resources should be created   | `true`                           |
 | `rbac.role.rules`                            | Rules to create                                 | `[]`                                  |
-| `mongodb.enabled`                            | Enable Mongodb                                  | `false`                                |
-| `mongodb.image.tag`                          | Mongodb docker image tag                        | `3.6.8-debian-9`                      |
-| `mongodb.image.pullPolicy`                   | Mongodb Container pull policy                   | `IfNotPresent`                        |
-| `mongodb.persistence.enabled`                | Mongodb persistence volume enabled              | `true`                                |
-| `mongodb.persistence.existingClaim`          | Use an existing PVC to persist data             | `nil`                                 |
-| `mongodb.persistence.storageClass`           | Storage class of backing PVC                    | `generic`                             |
-| `mongodb.persistence.size`                   | Mongodb persistence volume size                 | `50Gi`                                |
-| `mongodb.livenessProbe.initialDelaySeconds`  | Mongodb delay before liveness probe is initiated   | `40`                               |
-| `mongodb.readinessProbe.initialDelaySeconds` | Mongodb delay before readiness probe is initiated  | `30`                               |
-| `mongodb.mongodbExtraFlags`                  | MongoDB additional command line flags           | `["--wiredTigerCacheSizeGB=1"]`       |
-| `mongodb.usePassword`                        | Enable password authentication                  | `false`                               |
-| `mongodb.db.adminUser`                       | Mongodb Database Admin User                     | `admin`                               |
-| `mongodb.db.adminPassword`                   | Mongodb Database Password for Admin user        | ` `                                   |
-| `mongodb.db.mcUser`                          | Mongodb Database Mission Control User           | `mission_platform`                    |
-| `mongodb.db.mcPassword`                      | Mongodb Database Password for Mission Control user | ` `                                |
-| `mongodb.db.insightUser`                     | Mongodb Database Insight User                   | `jfrog_insight`                       |
-| `mongodb.db.insightPassword`                 | Mongodb Database password for Insight User      | ` `                                   |
-| `mongodb.db.insightSchedulerDb`              | Mongodb Database for Scheduler                  | `insight_scheduler`                   |
 | `postgresql.enabled`                         | Enable PostgreSQL                               | `true`                                |
 | `postgresql.imageTag`                        | PostgreSQL docker image tag                     | `9.6.11`                              |
 | `postgresql.image.pullPolicy`                | PostgreSQL Container pull policy                | `IfNotPresent`                        |
@@ -192,11 +255,13 @@ The following table lists the configurable parameters of the mission-control cha
 | `elasticsearch.javaOpts.xms`                 | Elasticsearch ES_JAVA_OPTS -Xms                 | ` `                                   |
 | `elasticsearch.javaOpts.xmx`                 | Elasticsearch ES_JAVA_OPTS -Xmx                 | ` `                                   |
 | `elasticsearch.env.clusterName`              | Elasticsearch Cluster Name                      | `es-cluster`                          |
+| `elasticsearch.env.minimumMasterNodes`       | The value for discovery.zen.minimum_master_nodes. Should be set to (replicaCount / 2) + 1                      | `1`                          |
 | `logger.image.repository`                    | repository for logger image                     | `busybox`                             |
 | `logger.image.tag`                           | tag for logger image                            | `1.30`                                |
 | `missionControl.name`                        | Mission Control name                            | `mission-control`                     |
 | `missionControl.image`                       | Container image                                 | `docker.jfrog.io/jfrog/mission-control`     |
 | `missionControl.version`                     | Container image tag                             | `.Chart.AppVersion`                   |
+| `missionControl.mcKey`           | Mission Control mc Key. Can be generated with `openssl rand -hex 16` |`bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`|
 | `missionControl.customInitContainers`        | Custom init containers                          | ` `                                   |
 | `missionControl.service.type`                | Mission Control service type                    | `LoadBalancer`                        |
 | `missionControl.externalPort`                | Mission Control service external port           | `80`                                  |