Skip to content

v0.9 Release Blog

Jump to bottom
Shreyas Mishra edited this page Mar 2, 2023 · 17 revisions

K8s Operator-based install for KubeArmor

KubeArmor supports multiple modes of deployment today, including using manifests files, helm, and using karmor cli tool.

However, operator-based installation was desired for KubeArmor for the following reasons:

  1. To handle the scenario where the cluster contains multiple nodes supporting different LSM (Linux Security Modules). KubeArmor cannot set the AppArmor annotation in context to the workload deployed on the node not supporting AppArmor.
  2. There are certain services such as Kubearmor relay whose resource utilization depends on the number of nodes operating within the cluster.

Operator-based installation and subsequent monitoring simplify the handling of such scenarios.

With this release, the karmor cli tool or the helm/manifests will install the operator and then the operator will install the relevant Daemonset and services needed.

Consolidation of controllers

Benchmarking data

Config

  • Node: 4
  • Platform - AKS
  • Workload -> Sock-shop
  • replica: 1
  • Tool -> Apache-bench (request at front-end service)
  • Vm: DS_v2
Vm CPU Ram Data disks Temp Storage
DS2_v2 2 7 GiB 8 14 GiB

Without Kubearmor

Average

Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
no kubearmor 50000 5000 - - 2205.502 0.4534 0 401.1 287.3333333
Readings
Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
no kubearmor 50000 5000 - - 2246.79 0.445 0 380 239
-- -- -- -- -- -- -- -- -- --
no kubearmor 50000 5000 - - 2187.22 0.457 0 378 358
no kubearmor 50000 5000 - - 2244.16 0.446 0 451 258
no kubearmor 50000 5000 - - 2213.37 0.452 0 351 304
no kubearmor 50000 5000 - - 2131.19 0.469 0 380 251
no kubearmor 50000 5000 - - 2215.89 0.451 0 400 326
no kubearmor 50000 5000 - - 2172.19 0.46 0 428 332
no kubearmor 50000 5000 - - 2195.73 0.455 0 444 240
no kubearmor 50000 5000 - - 2206.41 0.453 0 385 278
no kubearmor 50000 5000 - - 2242.07 0.446 0 414 318
Average         2205.502 0.4534 0 401.1 287.3333333

With Kubearmor

Average

Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
with kubearmor 50000 5000 144.7142857 109.9 2152.563 0.4645 0 456.4 394
Readings
Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
with kubearmor 500000 5000 138 108 2150.07 0.465 0 429 446
with kubearmor 500000 5000 126 112 2177.17 0.459 0 479 408
with kubearmor 500000 5000 125 112 2186.66 0.457 0 520 418
with kubearmor 500000 5000 167 110 2141.53 0.467 0 466 417
with kubearmor 500000 5000 139 111 2161.54 0.463 0 422 384
with kubearmor 500000 5000 154 109 2117.48 0.472 0 505 344
with kubearmor 500000 5000 164 112 2160.88 0.463 0 430 331
with kubearmor 500000 5000 146 110 2112.76 0.473 0 453 450
with kubearmor 500000 5000 131 106 2162.98 0.462 0 420 364
with kubearmor 500000 5000 138 109 2154.56 0.464 0 440 378
Average     144.7142857 109.9 2152.563 0.4645 0 456.4 394

Kubearmor with discovered Policy Applied

Average

Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
with Policy  50000 5000   141.2 111.9 2169.358 0.4609 0 438.2 435.1
Readings
Scenario  Requests Concurrent Requests Kubearmor CPU (m) Kubearmor Memory (Mi) Throughput (req/s) Average time per req. (ms) # Failed requests Micro-service CPU (m) Micro-service Memory (Mi)
with Policy 500000 5000 131 113 2162.86 0.462 0 542 446
with Policy 500000 5000 139 111 2190.72 0.456 0 457 458
with Policy 500000 5000 145 112 2103.46 0.475 0 445 395
with Policy 500000 5000 149 108 2155.55 0.464 0 440 454
with Policy 500000 5000 129 113 2177.68 0.459 0 395 394
with Policy 500000 5000 160 122 2198.53 0.455 0 435 503
with Policy 500000 5000 156 117 2179.89 0.459 0 391 451
with Policy 500000 5000 134 119 2196.78 0.455 0 408 429
with Policy 500000 5000 129 114 2178.07 0.459 0 424 435
with Policy 500000 5000 140 112 2150.04 0.465 0 445 386
Average     141.2 111.9 2169.358 0.4609 0 438.2 435.1

Support for Unbreakable Enterprise Linux (UEK) used in Oracle Kubernetes Engine (OKE)

Explain what changes had to be done? What Enforcement logic had to be used?

Multiple controllers such as policy-controller and host-policy-controller were separately installed. The new release consolidates multiple controllers into a single pod reducing the overall number of kubearmor pods deployed in the cluster.

Support for AWS Amazon Linux 2

Clone this wiki locally