DOP

Emerging from the agile culture, DevOps extremely emphasizes automation and heavily relies on tools in practice. Given the rapidly increasing number and diversity of the tools for DevOps, we aims to simplify the DevOps practices and enable development and operation efficiency by leveraging the combined benefits of tooling and automation. To achieve this aim, we developed and implemented a unified web-based DevOps platform (DOP), through integrating tools to DevOps process and continuous delivery pipeline in an automated manner. DOP provides five basic functionalities: code management, pipeline management, test management, container image management, and application management. Moreover, advanced features important for both practitioners and researchers are also available at DOP, e.g., visual demonstration of execution results of each phase in the whole pipeline for easier use, data collection through logs for process mining research, and logs’ visualization for quicker bug localization. Initial cases successfully adopting DOP indicate the effectiveness of our platform.

1. Architecture design

We designed DOP using microservices architecture and decomposed it into microservices based on its business logic and service capabilities. Most of the microservices of DOP communicate with each other through HTTP protocol, and only a small minority use the message queue. The architecture design of DOP includes eight layers:

Infrastructure layer：Applications managed by DOP and services of the DOP itself are all deployed in Kubernetes, and some tools are directly deployed on virtual machines or physical machines.

Persistence layer：MySQL and MongoDB are used for persistent storage, and Ceph is used to store distributed files,e.g., some stateful data applications in Kubernetes.

Middleware layer：RocketMQ is used as the message queue and Redis is used as the cache database.

Tool Layer：Tool layer mainly includes some open source tools, based on which to provide some basic capabilities of DOP. These capabilities can be combined in the Business Service Layer through business logic later. For example, some operations related to CI pipeline (pipeline parallelism, waiting) are implemented through wrapping the functions of Jenkins.

Fundamental Service Layer：This layer contains some services of DOP itself. Services in this layer could support some basic business logic, including service discovery, audit, login, authority management, user management, and message service, which are able to provide some common functions for business Layer services.

Business layer：This layer are partitioned into five services: testing, code, application management, pipeline management, and container image management. These services correspond to the various functions used by users, which are introduced later. See more explanations about the design of these services in this layer.

Access Layer：Access Layer is an API gateway, which mainly deals with two authentication processes: client OAuth 2.0 authentication and user authentication.

Interface Layer：This layer includes Web UI, SSO, and RESTful API used by users. External systems can call the platform API after it has been authenticated by the client OAuth 2.0.

DOP system uses the micro-service architecture. We build the basic layer services to support the micro-service architecture and basic business logic. According to the business context, we divide the business into micro-services such as testing, code, application management, pipeline and so on. Most of the services communicate through HTTP protocol, and a few communicate through message queue.

The relationship between business modules and DevOps processes is shown in the following figure:

Developers push the code to code repository of an application, triggering the WebHook of the code management service, which can send an HTTP request to the pipeline management service to start a pipeline. When the pipeline is started, the source code of an applicaiton is pulled from the code repository, after which a build task can be executed. Then a test task is submitted to the test service, and a Docker image could be built and pushed to the image management service. Finally, the application can be deployed to the target environment, whose latest container image is pulled by the Kubernetes cluster and starts running.

2. Usage

This chapter introduces the basic functions and usage of DOP. Before using it, you need to do some preparation work, such as creating an account and obtaining permissions

1. Visit：http://www.dop.clsaa.comSign up for a DOP account
2. After receiving the email of registration, click the link to activate the account, find the person in charge of DOP and add access to various functions of DOP
3. Deploy your project using DOP as described in 1.1 or 1.2

2.1. Routine use

2.1.1. Preparatory phase

Prepare the code repository

All code pulling in DOP systems is done using Git, so project code must be stored in Git repositories.

You can choose to use DOP's own code management module (code repository) or third-party code repository (such as GitHub, GitLab, etc.).It should be noted that the third-party code repository is used, and the visibility of the repository must be set to be publicly visible (currently the DOP platform is not connected to GitHub and GitLab account system);You can use a private repository if you use DOP's own code repository.

The code management module provided by DOP includes most of the functions of GitLab, and can basically complete daily code management work in the code management module of DOP.

1. Log in DOP and select Code Management

2. The first time you use DOP code management, it is better to add your computer's SSH public key to facilitate the subsequent development of PUSH code.

3. Create code repository

4. To view the repository, you can click on the repository name on the repository list page to view the repository.In the repository preview screen, you can see the repository name, icon, visibility, Git repository location, README file, and you can also download or set the repository.

5. To set up the project, click Settings to enter the project Settings page.In the Settings page, you can edit the basic information of the project (name, description, default branch, visible level and delete the project);You can set up the protection branch of the project (to prevent people with low permissions from modifying the content of important branches at will);You can set project members (add other users to the project and set the appropriate roles)

6. Browse files, select Files to enter the file browsing interface.You can view the file directory, file contents, the latest commit information and time of a file;You can click on a file name to view the contents of the file, or click on a commit information to view the commit changes.

7. Click Commits to enter the project submission record page.You can view the commit record list (commit message, time, sha);You can search for a commit;You can view commit by branch;You can view the file information for a commit;Click on a commit to see the changes to all the commit files.

8. Click the Branches to enter the branch page.Can view all branch information of the current project;Can create, delete, and search branches;You can see the latest commit for a branch.

9. Click the Tags to enter the tag page.You can view, search, delete, and create labels.You can see the commit corresponding to the label.

10. Click Merge Requests to enter merge request processing page.You can see the list of merges that need to be processed (name, time, operator);A merge request can be created;Click the merge name to view the details of the merge request;Can handle (open, close, merge) merge requests.

Prepare the container image repository

All deployment of DOP is completed with Docker and Kubernetes, so before deployment, the application Docker image needs to be stored, and after the Kubernetes cluster receives the deployment instruction, it will pull and run the image from the image repository.

Your application can use the image management module (image repository) that comes with DOP, or you can use the third-party image repository (DockerHub, ali cloud container image service, etc.), but the access right of the third-party image repository must be set to be public.

Here to explain image management module of the two concepts: namespace (usually on behalf of a project, is a classification of image repository), image repository (generally on behalf of an application, each application image every building general into the same image but with different version number of the repository), image (generally represent a certain version of an application)

docker pull registry.dop.clsaa.com/dop/application-server:2019032515435beb579

registry.dop.clsaa.com is the image repository domain name，

dop is namespace,representing project

application-server is image's name，representing application management service

2019032515435beb579 is image tag,representing a version of the docker image once

1. Click Image management to enter the image management page.You can create, retrieve, browse, and delete namespaces.Visibility of namespaces can be set (user roles must be the Namespace Manager of this Namespace, if they have permissions).

2. To enter the namespace, click the namespace name to enter the namespace page.You can browse, retrieve, and delete the mirror repository in the namespace.You can set namespace members and roles;You can view the operation log for namespaces.

3. To view the image repository, click the name of the image repository in the namespace to enter the image repository page.Browse and retrieve the list of different versions of a image (image name, Docker version, size, abstract, can be directly assigned to pull command);You can delete the repository.It is important to note that the image repository cannot be created directly, and when you create a namespace and push an image directly into it, it is automatically created in that namespace

1.1.2. Usage phase

project management

After the above code repository, image repository preparation, you need to create a project (a project can contain many applications, and an application usually corresponds to a service) where you can manage project members and applications.

Of course, you need to have access to DOP before creating the project. Please contact the responsible person for access.

1. To view all projects, click All Projects to enter the list of projects page.You can browse, retrieve, and create project.

2. View and set the project. Click the project name to enter the project details page.You can view project information, manage members (add and remove members), and manage applications (browse, retrieve, create and delete applications).

application management

After creating the project, you need to create an application in the project, which often represents a Java service, a NodeWeb, and so on

1. To view the list of applications, click the application TAB in project details.View all application information in the project;You can create applications;You can retrieve applications.

2. To create an application, click the Create an application.Fill in the application name and development mode;Fill in Git repository address (you can fill in the third-party Git address or fill in the DOP internal Git repository address);Fill in the image repository address (you can fill in the third party image repository address or the DOP internal image repository address);For the above two addresses, if you use the modules that come with DOP, you can search the repositories you have just created by searching directly.

3. To view the application, click the application name in the project.Can view and edit application basic information (application ownership, basic information, URL information, etc.);Browse, create, and delete application environments.You can browse, create, and delete application variables.

Application environment management

1. View the list of application environments (generally divided into development, test, integration and production), and click Envionment Profile when entering the application details page.By default, DOP creates a daily development environment for each application.You can create a new environment (currently only available for Kubernetes cluster deployment);You can click Deployment configuring for a specific environment configuration;You can click Deployment History to see the deployment history of an application in an environment.

2. Environment profile, click Deployment configuring to configure an environment in detail.

Configure the Kubernetes cluster

1. To authenticate the Kubernetes cluster, first fill in the authentication information of the Kubernetes cluster (URL https://masterIP:6443 and Token) and click submit to allow DOP to manipulate the Kubernetes cluster.
2. Select the release strategy, which currently supports only rolling upgrades
3. Select the Yaml file source
   1. Use configuration: DOP will configure you to automatically generate Yaml files based on your options
   2. Use the relative path: fill in a relative path, and DOP will get the Yaml file from the relative path of your project
4. When you choose to use configuration, select the namespace, and select existing or created services
   1. Choose to use the original service (without creating a new service): this way you can search for and use the original service.
   2. Select create new Service: this will create a new Service according to the configuration
5. When you choose to create a new service, you can select how the service is exposed to the outer network.

   1. Ingress: Access the service as a domain name. Once configured, go to the public cloud and set up DNS resolution to the Master node of the Kubernetes cluster. TargetPort is the port exposed by the container (your service)

   2. NodePort: Access the service in MasterIP+Port, and after configuration, access the service directly through MasterIP:Port. TargetPort is the Port exposed by the container (your service).

6. Set the number of replicas (how many instances this service runs)

Pipeline creation

1. Click pipeline management to enter the pipeline list page and browse all the pipelines.

2. Click "create pipeline" to create a pipeline, set the basic information of assembly line and trigger mode, and finally set the contents of each stage of assembly line as follows:

   1. Pull code: select the appropriate application /Git repository address
   2. Build: choose a Python project, Maven project, or Node project based on your project type
   3. Build Docker image: choose to build Docker image, select environment, image address and other information
   4. Push Docker image: select push Docker image, select environment, image address and other information
   5. Deploy: select deploy and do not do any other configuration

2.1.3. run phase

Running pipeline

1. Click Pipeline management to view pipeline list
2. Click to view a pipeline. If the pipeline has been executed, you can see the visual execution result of pipeline stage
3. Click Run pipeline, and the pipeline starts running

View the runtime log

1. Click the pipeline to run, the page will be updated in real time

2. You can click the pipeline stage in the page to see the log of a stage execution 3. If the pipeline stage is successfully executed, the stage is shown in green

   4. If the pipeline stage fails, the stage is shown in red

2.1.4.verification stage

View deployment history

1. After successful pipeline execution, you can view the application's deployment history (run id, commitId, mirror version number, snapshot of the environment, deployer, run status). There are three main pieces of information in the deployment history
   1. Click run id to view the pipeline running log
   2. Click commitId to see the corresponding code change for this pipeline run
   3. Click on the image version number to view the deployed docker image
   4. Mouse over the deployment environment to view a snapshot of the kubernetes environment for this deployment

View application information

1. Check the status of containers running in Kubernetes' Dashboard

2. test.dop.clsaa.com has been deployed successfully