Sunbird easy installation

Objective:

Make the sunbird installation initial setup more easy, reliable and efficient.

Concerns it is going to address

To reduce the complexity involved in setting up Sunbird in medium scale deployment(6weeks -> 3days -> 1.5hr) .This is achieved by below two key features:
- Main Config File (MCF) - file link
- Sunbird Setup Script (SSS) - file link

2)Make documentation simple and place in right order.

3)Scaling - Flexible to increase Ram, Replication .

4)Customized documentation for setting any service on VM.

5)SSL certs copying.

6)Backup and restore scripts.

MCF:

It is a single global configuration file (YML) which will be used in the Sunbird Setup Script (SSS). It is passed as an external config to all the ansible-playbook command using --extra-vars "@some_file.yml". All the variables in this config file are more generalized without any duplication. Variables mentioned in <implementation-name>-devops/ansible/inventories/<environment-name>/group_vars/<environment-name> file will be overridden by the values set in the main configuration file.
Purpose of doing this way is to reduce the effort of updating the repetitive values of the variables in the group_vars and to maintain idempotency. Explained in detail below with an example.

Example:

Problem Statement: Currently, on the medium scale deployment for sunbird, all the databases are running on the same machine. Sunbird consumer should update the group_vars file with the private ip of DB's (ex: 10.10.0.1) in many places as listed below.

cassandra_host:         10.10.0.1
elasticsearch_host:     10.10.0.1
postgresql_host:        10.10.0.1
mongodb_host:           10.10.0.1

Solution: Using Main Config File (MCF) we can overcome the problem. We will create a new variable inside the MCF.

-cassandra_host:         10.10.0.1
-elasticsearch_host:     10.10.0.1
-postgresql_host:        10.10.0.1
-mongodb_host:           10.10.0.1
-proxy_site_key: +|
-  -----BEGIN PRIVATE KEY-----
-  MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDWOp0w86dZCGpV
-  kY3BtoBQXUehV5t0mWsh8RP9FAQV/ei8+YRkkoI9qdIrmW9j41QY2qdzIvuTtKO7
-  V0/2I1UT7pVRlfCO1sQnkmfs1Oy/btPhm0Nqcu5mneHCULVvJoLvF1QKc7DXrXP9
-  cWS/dSG2bV7zWsxNK2aUkavXjQG3rZJj1tmiPQ1GLkBDEzuqQI06ejONyGs7CLPN
-  uqaTPnKMGhiSRb2VPr61ocTIeRWBp7sJ5xjxoMbBRghF6HUwS9hLp5v3g2QJmF3W
-  p+aIzZLaKdlI7gV3FgcWB7w5sOpWIq+Y8cFJ+GZLWO6NfoFWCXNKiyL2F1JdOsx0
-  EnJskFoDAgMBAAECggEABLUE/y5bXvlwXLD2DGuWJi3mhQUHxWgZkoVZixYFgPYz
-  RvO0lQEjG3e/xnV4nRlBn8ouI7mBDZ3KezNP7niYRffYEBLbVkghxzzQS8Ph79Lf
-  lPV0ZcJguqB9dWeQlQCjn91DW3dvS03kXo0/SXWpZ/i/PVmhY5CwkFMjE3llsEN1
-  yDG1m1OX6AOU0r6DZOwRw1PZKNdDg3ADNPB5NhLXX8I/djWExY5dMcCYsn0Z54s2
-  mq/VfOdJNc9bbMd/PKW+bOoFdtZoeGvwlVi9FMqUgIGTqA5Lqoqeo7kzRcbO6enl
-  Wmer2E/nma/de133bI/WGsTeeQLkaN/XHvh2GkaFAQKBgQDxYrZG5UXrUw6CQCkM
-  Kax2WHW9B0dHNoLgXHZD0veASgUwxy+s1sXTB3KJnrJrYUZXi/1AGQOpMO/dwcnt
-  nVf28q4AsDtAYjbDbwEfdEXopV9j2aA5BFIdVb69Zwmx81LuamcpSdIg5UeJ3GBF
-  i6QY4xKPmrevAHCXHLKOvJmNXQKBgQDjMv7Ur/O2pEggcUFFpabP4QUWZyPsEh4+
-  pqhFJIjSgcsWpTkXTvtGrpPF/hQbFBfDJ2+BP5ZW+A7V3p884bYtrM6EXsVq+RYL
-  9DdppxQ8JXzrMGEMTKuGfXgbcv6Qb3e1Che0cq7R6m38S4b7V1PP/EaTu9eQNDZM
-  Tupdgw2u3wKBgQDmuQWlSizEvATYNt+FQqgYDoGrhrN4u//o2DgqqBHuS9IeR6Wo
-  PTmldi7ioA/GV5pWv/X3OHGpXJ9JhZuvRX/5CQmMmuZnqfjSYN3f5NGVh8xfsnpV
-  RW3/oB/LBhvcPcn9HTMx4d+5la4FdeGHV+wn5ZOuM6Jy4aa8qa96g4b5+QKBgQCo
-  V02n66NJzdm/aAsz/rRlzNXHLq7YKOiOBVAiwxo513/SDKQLmDLowUuu5T6ifbXJ
-  nLuq6aGMWQos/2G/a4Jfv23ZeJsQUvU58D7qoqiElEZFwTf3NAvXf7nxpLfn7x+k
-  XZEI2/yafvf1e0TNkHwCXGsy5XioWXdt/P78QLFvwwKBgQCmMJ1RDUbJtU8DMa+z
-  AfCDEi1vQWir2UEcozTjmC8crGbHoiVlNVnwY86JGNGd50jrHqDczAxjH1Kh8q6J
-  yCAU8KkcgeqZAT1cUbmbVpWgWJY71NrIxNYG3CcX3pvMthXfw9B3xjdYTUObOfmF
-  kGko7GIteLziGT90IOz0Pno12g==
-  -----END PRIVATE KEY-----

-proxy_site_cert: +|
-  -----BEGIN CERTIFICATE-----
-  MIIFRzCCBC+gAwIBAgISA7B1ACbKuwBSCncb8zpLOIuCMA0GCSqGSIb3DQEBCwUA
-  MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
-  ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xODAxMjcwOTQyMjJaFw0x
-  ODA0MjcwOTQyMjJaMD0xOzA5BgNVBAMTMmFwcC10ZXN0cmVsZWFzZS0xNC5jZW50
-  cmFsaW5kaWEuY2xvdWRhcHAuYXp1cmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOC
-  AQ8AMIIBCgKCAQEA1jqdMPOnWQhqVZGNwbaAUF1HoVebdJlrIfET/RQEFf3ovPmE
-  ZJKCPanSK5lvY+NUGNqncyL7k7Sju1dP9iNVE+6VUZXwjtbEJ5Jn7NTsv27T4ZtD
-  anLuZp3hwlC1byaC7xdUCnOw161z/XFkv3Uhtm1e81rMTStmlJGr140Bt62SY9bZ
-  oj0NRi5AQxM7qkCNOnozjchrOwizzbqmkz5yjBoYkkW9lT6+taHEyHkVgae7CecY
-  8aDGwUYIReh1MEvYS6eb94NkCZhd1qfmiM2S2inZSO4FdxYHFge8ObDqViKvmPHB
-  SfhmS1jujX6BVglzSosi9hdSXTrMdBJybJBaAwIDAQABo4ICMjCCAi4wDgYDVR0P
-  AQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMB
-  Af8EAjAAMB0GA1UdDgQWBBSdOQ5OCuJ8ff183KarqtHJ1EpsZzAfBgNVHSMEGDAW
-  gBSoSmpjBH3duubRObemRWXv86jsoTBvBggrBgEFBQcBAQRjMGEwLgYIKwYBBQUH
-  MAGGImh0dHA6Ly9vY3NwLmludC14My5sZXRzZW5jcnlwdC5vcmcwLwYIKwYBBQUH
-  MAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlwdC5vcmcvMD0GA1UdEQQ2
-  MDSCMmFwcC10ZXN0cmVsZWFzZS0xNC5jZW50cmFsaW5kaWEuY2xvdWRhcHAuYXp1
-  cmUuY29tMIH+BgNVHSAEgfYwgfMwCAYGZ4EMAQIBMIHmBgsrBgEEAYLfEwEBATCB
-  1jAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwgasGCCsG
-  AQUFBwICMIGeDIGbVGhpcyBDZXJ0aWZpY2F0ZSBtYXkgb25seSBiZSByZWxpZWQg
-  dXBvbiBieSBSZWx5aW5nIFBhcnRpZXMgYW5kIG9ubHkgaW4gYWNjb3JkYW5jZSB3
-  aXRoIHRoZSBDZXJ0aWZpY2F0ZSBQb2xpY3kgZm91bmQgYXQgaHR0cHM6Ly9sZXRz
-  ZW5jcnlwdC5vcmcvcmVwb3NpdG9yeS8wDQYJKoZIhvcNAQELBQADggEBAAWRNPUc
-  1UU5izBT0EG/XYLEr7Uo2v/98KAKVOo6lreD8oR6hOWAk11gzopeLNwoumgIG6KO
-  aI20hQCWS4RwO+ro7hJAM+3zNyuVEqUSYfGdX9AfpzaAqxG0tH+UjUooOxBP5QWr
-  HMq0adLYbnuMiOZ+5LuQ2iratnD+kt1bNHmTucApZ+F1DGFpRstwGUPxUbSDLXZF
-  /HGT5hWSQyLciVZ6QF6YQLSEjyzuCJFSTwNxnKurZiJzirlF7Tga4tZLjzVcv+SL
-  HPaUFN6C9+4KjSrDzZCf1/6UJ/2HiAkIoZZNGdJHUKyqPzOQGbt4B+R5331ZzV88
-  iedhULT7ub1d1uw=
-  -----END CERTIFICATE-----

+db_ip: 10.10.0.1
+proxy_site_key: /home/rajesh/certs/key.pem
+proxy_site_cert: /home/rajesh/certs/site.crt

(for ex: db_ip: 10.10.0.1) and it will get updated in the group_var file. Below example describes how the MCF is configured.

Suppose, inside the MCF we assigned a new variable which is db_ip: 10.10.0.1. And the group_vars file will look like

cassandra_host:      "{{db_ip}}"
elasticsearch_host:  "{{db_ip}}"
postgresql_host:     "{{db_ip}}"
mongodb_host:        "{{db_ip}}"

Conclusion: Whenever we run the ansible-playbook command both MCF and group_var file is used. All the variables in MCF are injected into the group_var file.

SSS:

It is a Master shell script which will internally call and execute many other shell scripts to set up the sunbird. Sunbird consumer will be running only Sunbird Setup Script (SSS) and the deploy-core.sh (not part of SSS) scripts to bring the entire environment up and running. SSS reads the implementation-name and <env> from the Main Config File (MCF) and passes as Command Line arguments to the other shell scripts inside the Sunbird Setup Script (SSS). Sample SSS looks like below.

Example

imp=`awk '/implementation_name/{print $NF}'`
env=`awk '/env/{print $NF}'`
implementation=$imp"-devops"
./geneate_config.sh $imp $env
./deploy-apis.sh $implementation/ansible/inventories/$env

Options For SSS:

╰─$ ./sunbird_install.sh -h
Usage: ./sunbird_install.sh [ -s {config|dbs|apis|proxy|keycloak} ]

Once Sunbird Setup Script (SSS) is executed successfully, following items will be installed and configured.

generate_config.sh     (Create a configuration folder on app server)
install-depth.sh        (Install docker, Ansible, and other prerequisites)
install-dbs.sh         (Install Cassandra,elasticsearch and postgress databases)
init-dbs.sh            (Initialize databases)
deploy-api.sh          (provision api-manger, deploy api-manager, onboard consumers and API's). Make sure JWT 
                        token copied  to a seperate file. 
deploy_proxy.sh        (setup nginx)
provision-keycloak.sh  (Install keycloak)
deploy-keycloak-vm.sh  (Deploy keycloak jar file )

NOTE:

Keycloak realm import, SSO user creation, and deploy-core.sh are pending manual steps after successful completion of executing the Sunbird Setup Script (SSS).

Action items:

Create Main Config File (MCF).
Update the group_vars and hosts files.
Create Sunbird Setup Script (SSS).
Use getopts inside the SSS script to read arguments.
Ansible installation fixes.
JWT token copying to the file.
Logging of each subtask.

Provide feedback

Saved searches