project-sunbird · santhosh-tg · Mar 24, 2022 · Mar 28, 2022 · Mar 28, 2022
diff --git a/_data/sidebar.yaml b/_data/sidebar.yaml
@@ -7,8 +7,8 @@
             subfolderitems:
               - page: Pre-Requisites
                 url: developer-docs/server-installation/prerequisites/
-              - page: Optional Load Balancer Setup
-                url: developer-docs/server-installation/load-balancer-setup/  
+              - page: Infra Setup
+                url: developer-docs/server-installation/infra-setup/  
               - page: Ansible Variables
                 url: developer-docs/server-installation/ansible-variables/
               - page: Jenkins Setup

diff --git a/developer-docs/server-installation/ansible-variables.md b/developer-docs/server-installation/ansible-variables.md
@@ -14,16 +14,14 @@ Ansible is the configuration management system used in Sunbird. The infrastructu
 Use the following git commands sequentially to clone and update your private GitHub repository -
 
 ```bash
-git clone https://github.com/project-sunbird/sunbird-devops
-cd sunbird-devops
-git checkout tags/release-4.3.0 -b release-4.3.0
+   git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0
 ```
 
 - Copy the directory `sunbird-devops/private_repo/ansible` to your private repo
 
 - Update the files **common.yml**, **hosts**, and **secrets.yml** under **Core**, **KnowledgePlatform** and **DataPipeline** directories. After updating, push them to your private repo branch
 - Your private repo structure starting from the root path should be exactly as shown below
-  
+
 ```bash
 ansible
 └── inventory
@@ -137,4 +135,4 @@ ansible
     <td>log-es-1, log-es-backup, log-es</td>
   </tr>
   <tr></tr>
-</table>
+</table>
diff --git a/developer-docs/server-installation/build-and-deploy.md b/developer-docs/server-installation/build-and-deploy.md
@@ -10,56 +10,9 @@ allowSearch: true
 ## Overview
 
 
-To install Sunbird `release-4.6.0`, we will need to run many Jenkins jobs and a couple of manual steps. Some of the jobs need to be run in a specific order. This page details out the list of jobs, their order and the github release tags to be used for each job.
+To install Sunbird `release-4.7.0`, we will need to run many Jenkins jobs and a couple of manual steps. Some of the jobs need to be run in a specific order. This page details out the list of jobs, their order and the github release tags to be used for each job.
 
 
-### Installing Sunbird
-
-#### Create Plugin Containers / Folder Structure
-
-- We will create few folders in azure for storing sunbird plugins
-- Our deployment job always tries to delete the folder first before uploading, so we need to have the placeholders before running the jobs
-- We will create the folders by uploading random files from a random folder on your PC
-- Create a public container with the name you have set for the variable `sunbird_content_azure_storage_container` and run the below commands
-
-```bash
-az storage blob upload-batch --destination sunbird_content_azure_storage_container/collection-editor --source some_folder --account-name storage_account_name --account-key storage_account_key
-
-az storage blob upload-batch --destination sunbird_content_azure_storage_container/generic-editor --source some_folder --account-name storage_account_name --account-key storage_account_key
-
-az storage blob upload-batch --destination sunbird_content_azure_storage_container/content-editor --source some_folder --account-name storage_account_name --account-key storage_account_key
-
-az storage blob upload-batch --destination sunbird_content_azure_storage_container/v3/preview --source some_folder --account-name storage_account_name --account-key storage_account_key
-```
-
-- Upload the default T&C file to `sunbird_content_azure_storage_container` container in a folder named **terms-and-conditions**. You can get a copy of the sample T&C file from [here](https://sunbirdpublic.blob.core.windows.net/installation/terms-and-conditions/terms-and-conditions-v9.html). You are free to edit the HTML as per your requirements
-- Create a private container named `label` in Azure
-- Run the below command in your Jenkins VM where you cloned the sunbird-devops repo
-
-```bash
-cd sunbird-devops/utils/portal
-az storage blob upload-batch --destination label --source labels --account-name storage_account_name --account-key storage_account_key
-```
-
-#### Upload Initial Plugins
-
-- Upload the initial set of plugins to `sunbird_content_azure_storage_container` container
-- Download the initial Plugins from [here](https://sunbirdpublic.blob.core.windows.net/installation/content-plugins.zip)
-- Run the below command from the directory where **content-plugins.zip** is unzipped
-
-```bash
-az storage blob upload-batch --destination sunbird_content_azure_storage_container/content-plugins --source content-plugins --account-name storage_account_name --account-key storage_account_key
-```
-
-- Create a container named `public` in your Azure storage account from the Azure portal with public access level
-- Create a container named `artifacts` in your Azure storage account from the Azure portal with private access level
-
-
-#### Upload Maxmind Database to Azure Blob
-
-- Download the Maxmind city database in zip format from Maxmind website
-- Upload the zip file to the `artifacts` container in Azure
-
 #### Code Builds
 
 > **Note**:

diff --git a/developer-docs/server-installation/index.md b/developer-docs/server-installation/index.md
@@ -19,10 +19,13 @@ This document explains the procedure to set up Sunbird on your cloud infra. This
 * AKS (Azure Kubernetes Service)
 * Ubuntu 18.04 VM's
 
-### Infrastructure Components
+### Building Blocks
 
-Sunbird consists of 3 major subsystems
+Sunbird consists of these major building blocks
 
-* Knowledge Platform also knows as Learning Platform (Taxonomy and Content)
-* Data Pipeline (Data enrichment and Visualizations)
-* Core Services (User, Organization, Courses, Certificates, UI and Frontend)
+* Sunbird inQuiry
+* Sunbird Knowlg
+* Sunbird Lern
+* Sunbird Obsrv
+* Sunbird RC
+* Sunbird UCI
diff --git a/developer-docs/server-installation/infra-setup.md b/developer-docs/server-installation/infra-setup.md
@@ -0,0 +1,76 @@
+---
+title: Infra Setup
+page_title: Infra Setup
+description: Explains the infra setup and automation
+keywords: infra, setup, automation, install, sunbird
+allowSearch: true
+
+---
+
+
+
+#### Automation on Azure infrastructure creation
+
+You can run the following steps to create azure infrastructure using ansible.
+
+Easiest way to use the script will be to use azure cloud shell, as the cloud shell comes with all prerequisites bundled.
+- login to portal.azure.com
+- click on the cloudshell -> select bash ( if you're using it for the first time )
+
+If you want to run this on your local machine, Follow this [guide](https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=azure-cli#install-ansible-on-the-virtual-machine).
+
+
+#### Create service principal for kubernetes cluster
+
+Command to create SP:
+```bash
+az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<your_subscription_id>" --name <NameOfSP>
+
+# Replace the <> with your values
+```
+
+Note down the output of above command as we will need those to fill-up some variables in the ansible playbook
+
+#### Create Maxmind licence key
+```bash
+1. Signup at maxmind.com if you don't have an account
+2. Goto https://www.maxmind.com/en/accounts/current/license-key
+3. Click Generate a new license key.
+```
+
+#### Clone the repo and go to deploy directory
+
+```bash
+git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0
+
+cd sunbird-devops/deploy
+```
+
+####  Update variables in azure-provision.yaml file
+
+```bash
+version: "ChangeThis"  # sunbird version Ex: 4.7.0
+sp_client_id: "ChangeThis"  # substitute the value of `appId` from the output of the create SP command
+sp_client_secret: "ChangeThis"  # substitute the value of `password` from the output of the create SP command
+ssh_public_key_file_location: "ChangeThis"  # path to your ssh public key Ex: /home/sunbird/.ssh/my_pub.key
+geoip_licence_key: "ChangeThis"  # substitute the one that you generated at # Create Maxmind licence key step
+```
+
+#### Run the playbook
+```bash
+ansible-playbook -c local azure-provision.yaml
+
+# Resulting infrastructure information will be stored in sunbird-devops/deploy/azure-resources.txt file.
+```
+
+Running the playbook creates these azure resources
+
+* Resource group
+* VNET and subnet
+* 6 VMs (size: Standard_D4s_v4)
+* Public IP for Jenkins
+* AKS cluster with 4 nodes (size: Standard_D4s_v4)
+* Storage account
+* ACR  ( Image registry)
+
+and performs some steps needed for the sunbird installation like uploading maxmind database to blob storage etc. Please refer to the task section of the playbook to get the complete list
diff --git a/developer-docs/server-installation/jenkins-setup.md b/developer-docs/server-installation/jenkins-setup.md
@@ -14,9 +14,8 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos
 - SSH to the Jenkins server and enter the following commands -
 
      ```bash
-     git clone https://github.com/project-sunbird/sunbird-devops.git
-     cd sunbird-devops && git checkout tags/release-4.3.0 -b release-4.3.0
-     cd deploy/jenkins
+     git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0
+     cd sunbird-devops/deploy/jenkins
      sudo bash jenkins-server-setup.sh
      ```
 
@@ -59,9 +58,9 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos
   ```
   sudo su jenkins
   curl -SsL -o ~/jenkins.yaml https://raw.githubusercontent.com/project-sunbird/sunbird-devops/release-4.3.0/deploy/jenkins/jenkins.yaml
-  
+
   # Replace all placeholders ${VALUE} with proper values
-  # For example, 
+  # For example,
   # username: "${GH_USERNAME}" to  username: "mygithubusername"
   vim ~/jenkins.yaml
   ```
@@ -87,4 +86,4 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos
     jenkins ALL=(ALL) NOPASSWD:ALL
  ```
 
-- Reboot the Jenkins VM (`sudo reboot`)
+- Reboot the Jenkins VM (`sudo reboot`)
diff --git a/developer-docs/server-installation/prerequisites.md b/developer-docs/server-installation/prerequisites.md
@@ -20,38 +20,6 @@ To successfully complete Sunbird installation, you need to have:
 
 > **Note:** Sunbird is tested only on cloud hosted Ubuntu servers in Azure. We do not provide any support for installing Sunbird on other operating systems. Sunbird might work on other cloud providers, feel free to try it out and let us know. We are actively working on achieving cloud neutrality.
 
-### Provisioning Servers
-
-Before you start the installation process, ensure that you have the required infrastructure mentioned below. It is the bare minimum for a full fledged Sunbird setup. Every component in Sunbird can scale horizontally / vertically by adding additional resources.
-
-|Application|  Server           |Count|
-|-----------|-------------------|-----|  
-|Jenkins    | 4core 16G 250G HDD | 1 |
-| KP        | 4core 16G 60G HDD | 1 |
-| DP        | 4core 16G 60G HDD | 1 |
-| DB        | 4core 16G 60G HDD   | 1 |
-| Yarn      | 4core 16G 60G HDD | 1 |
-| Other Services | 4core 16G 60G HDD | 1 |
-| Basic Load Balancers | - | 2 (Optional) |
-
-
-#### Automation on Azure infrastructure creation
-
-You can run the following steps to create azure infrastructure using ansible.
-
-Easiest way to use the script will be to use azure cloud shell, as the cloud shell comes with all prerequisites bundled.
-- login to portal.azure.com
-- click on the cloudshell -> select bash ( if you're using it for the first time )
-
-If you want to run this on your local machine, Follow this [guide](https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=azure-cli#install-ansible-on-the-virtual-machine).
-
-```
-git clone https://github.com/project-sunbird/sunbird-devops -b release-4.3.0
-cd sunbird-devops/deploy
-# Update the necessary variables in playbook
-ansible-playbook -c local azure-provision.yaml
-# Resulting infrastructure infromation will be stored in sunbird-devops/deploy/azure-resources.txt file.
-```
 
 ### Infra Requirements
 
@@ -71,41 +39,17 @@ ansible-playbook -c local azure-provision.yaml
 - YouTube API Token (optional)
 - Slack account and slack bot with API Token (optional)
 
-### Creating the AKS cluster
 
-> **Note**  Follow the steps given below to create the Kubernetes cluster in Azure. For other clouds, please visit the respective cloud provider website
-The AKS cluster and VM's should be in same vnet. If they are in different vnet, you have to peer the vnets. To successfully peer, the IP address of the vnets should not overlap.
-
-- Create a service principal and assign contributor role to service principal
-- Get the secrets and client id of service principal
-- Click [here](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli) for more details
-- Create the AKS cluster either via Azure portal or using `az aks` command line
-- Refer to Azure documentation for all the available options
-- Below is a sample command which you can use -
-
- ```bash
-az aks create --resource-group <resouse-group-name> --node-resource-group <k8s-resource-group-name> --name <cluster name>  --node-count 4 --admin-username deployer --kubernetes-version 1.19.9 --service-principal "<service principal id>" --node-vm-size Standard_D4s_v3 --client-secret "<client id>" --network-plugin azure --ssh-key-value @deployer.pub -l <region> --vm-set-type VirtualMachineScaleSets --vnet-subnet-id /subscriptions/<subscription id>/resourceGroups/<resouse-group-name>/providers/Microsoft.Network/virtualNetworks/<vnet-name>/subnets/<subnet name>
-```
-
-> Note: Ensure you have allocated at least 1024 IP's for your Kubernetes subnet (CIDR notation as x.x.x.x/22)
-
-Get the kubeconfig file for your cluster with the below command -
-
-```bash
-az aks get-credentials --resource-group <resource group name> --name <cluster name> --file  k8s.yaml
-```
-
-### Configuring the Azure storage account
-
-- Update the CORS rule for the storage account as follows:
+### Provisioning Servers
 
- ```bash
-    Allowed Origins: *
-    Allowed Methods: GET,HEAD,OPTIONS, PUT
-    Allowed Headers: Access-Control-Allow-Origin,Access-Control-Allow-Method,Origin,x-ms-meta-qq,x-ms-blob-type,x-ms-blob-content-type,Content-Type
-    Exposed Headers: Access-Control-Allow-Origin,Access-Control-Allow-Methods
-    Max Age: 200
+Before you start the installation process, ensure that you have the required infrastructure mentioned below. It is the bare minimum for a full fledged Sunbird setup. Every component in Sunbird can scale horizontally / vertically by adding additional resources.
 
- ```
-
-- Disable **Secure transfer required** in storage account configuration
+|Application|  Server           |Count|
+|-----------|-------------------|-----|  
+|Jenkins    | 4core 16G 250G HDD | 1 |
+| KP        | 4core 16G 60G HDD | 1 |
+| DP        | 4core 16G 60G HDD | 1 |
+| DB        | 4core 16G 60G HDD   | 1 |
+| Yarn      | 4core 16G 60G HDD | 1 |
+| Other Services | 4core 16G 60G HDD | 1 |
+| Basic Load Balancers | - | 2 (Optional) |