HA Kubernetes on 6 node 8GB Raspberry PI 5 cluster via kubeadm or Rancher k3s - bare metal simulation

[obriensystems]

Software

• https://k3s.io/
• https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
• https://datasheets.raspberrypi.com/rpi5/raspberry-pi-5-product-brief.pdf
• https://github.com/raspberrypi/rpi-imager
• Bare Metal Load Balancer - https://metallb.universe.tf/

Hardware

• see https://www.canakit.com/raspberry-pi-5
• get the board, power supply for 35+ watts and the dual passive/active cooler
• Raspberry PI 5 - 8GB LPDDR4X04267 ! 64bit Broadcom BCM2712, quad-core Cortex-A76 (ARM v8) 64-bit SoC @ 2.4GHz 512KB L2, 2MB shared L3, 800MHz VideoCore VII GPU 1 GBE
• https://datasheets.raspberrypi.com/rpi5/raspberry-pi-5-product-brief.pdf
• Raspberry PI 4 - 4GB! 64bit Broadcom BCM2711, quad-core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz 1 GBE
• https://www.raspberrypi.com/products/active-cooler/
  

References

• https://en.wikipedia.org/wiki/Raspberry_Pi#Raspberry_Pi
• https://github.com/kelseyhightower/kubernetes-the-hard-way

Current

• Implement Kubernetes IaC | helm | kpt - for spring boot and mysql 8 containers - statefulset and deployment biometric-backend#6

Review

• https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
• https://github.com/mhausenblas/kube-rpi
• k3s based https://www.zenko.io/how-i-made-a-kubernetes-cluster-with-a-couple-of-raspberry-pis/
• microk8s based http://sirchia.cloud/2019/12/kubernetes-on-64-bit-os-raspberry-pi-4/

Older 2019

• http://blog.kubernetes.io/2015/12/creating-raspberry-pi-cluster-running.html
• https://www.raspberrypi.com/news/five-years-of-raspberry-pi-clusters/
• https://blog.alexellis.io/serverless-kubernetes-on-raspberry-pi/
• https://gist.github.com/alexellis/fdbc90de7691a1b9edb545c17da2d975
• https://blog.zhaw.ch/icclab/supporting-container-based-application-deployment-to-heterogeneous-hardware-using-rancher/
• https://medium.com/@subhampradhan966/how-to-install-kubernetes-cluster-kubeadm-setup-on-ubuntu-22-04-step-by-step-guide-dfcf33253f5f
• https://rudimartinsen.com/2023/12/29/kubernetes-cluster-on-vms-2024/
• https://www.server-world.info/en/note?os=Ubuntu_24.04&p=kubernetes&f=2
• https://medium.com/@olorunfemikawonise_56441/simplifying-kubernetes-installation-on-ubuntu-using-a-bash-shell-script-d75fed68a31
• https://devops.datenkollektiv.de/raspberry-pi-5-a-kubernetes-cluster.html
• https://thechief.io/c/editorial/k3d-vs-k3s-vs-kind-vs-microk8s-vs-minikube/

[obriensystems]

[obriensystems]

[obriensystems]

config
192.168.16.14 (0) - 192.168.16.11 (pi53)

[obriensystems]

Simulate on AWS VMs

AWS

Create VPC: vpc-006e77544bb4dd4f6
Enable DNS hostnames
Enable DNS resolution
Verifying VPC creation: vpc-006e77544bb4dd4f6
Create S3 endpoint: vpce-0a061605eed305ba7
Create subnet: subnet-0a1043cded4b2fae1
Create subnet: subnet-0e578b478b16fcf4b
Create subnet: subnet-0ca3d194409fd41c3
Create subnet: subnet-043da12a418a7cde5
Create subnet: subnet-0f643b64bfbd69dbc
Create subnet: subnet-0aa6053f3a511a5d4
Create internet gateway: igw-0704a4df43c65427e
Attach internet gateway to the VPC
Create route table: rtb-0851d95a46d236365
Create route
Associate route table
Associate route table
Associate route table
Create route table: rtb-09bd7b0b3e35cc9bc
Associate route table
Create route table: rtb-06a5100aa92e7f6e0
Associate route table
Create route table: rtb-0b0e219acfe1f20cd
Associate route table
Verifying route table creation
Associate S3 endpoint with private subnet route tables: vpce-0a061605eed305ba7

RunInstances
{
  "MaxCount": 1,
  "MinCount": 1,
  "ImageId": "ami-0e86e20dae9224db8",
  "InstanceType": "t3a.medium",
  "KeyName": "obrie...15",
  "EbsOptimized": true,
  "UserData": "Z2l0IGNsb25lIGh0dHBzOi8vZ2l0aHViLmNvbS9PYnJpZW5sYWJzRGV2L2t1YmVybmV0ZXMtY2x1c3Rlci5naXQKY2Qga3ViZXJuZXRlcy1jbHVzdGVyL3NyYy9hd3MKLi9rdWJlcm5ldGVzLWF3cy1tYXN0ZXIuc2g=",
  "BlockDeviceMappings": [
    {
      "DeviceName": "/dev/sda1",
      "Ebs": {
        "Encrypted": false,
        "DeleteOnTermination": true,
        "Iops": 3000,
        "SnapshotId": "snap-0a00520ef60a68bed",
        "VolumeSize": 40,
        "VolumeType": "gp3",
        "Throughput": 125
      }
    }
  ],
  "NetworkInterfaces": [
    {
      "SubnetId": "subnet-0aa6053f3a511a5d4",
      "AssociatePublicIpAddress": false,
      "DeviceIndex": 0,
      "Groups": [
        "<groupId of the new security group created below>"
      ]
    }
  ],
  "CreditSpecification": {
    "CpuCredits": "unlimited"
  },
  "TagSpecifications": [
    {
      "ResourceType": "instance",
      "Tags": [
        {
          "Key": "Name",
          "Value": "kube-master-0"
        }
      ]
    }
  ],
  "MetadataOptions": {
    "HttpEndpoint": "enabled",
    "HttpPutResponseHopLimit": 2,
    "HttpTokens": "required"
  },
  "PrivateDnsNameOptions": {
    "HostnameType": "ip-name",
    "EnableResourceNameDnsARecord": false,
    "EnableResourceNameDnsAAAARecord": false
  }
}

CreateSecurityGroup
{
  "GroupName": "all-open-kube",
  "Description": "all-open-kube",
  "VpcId": "vpc-006e77544bb4dd4f6"
}

AuthorizeSecurityGroupIngress
{
  "GroupId": "<groupId of the security group created above>",
  "IpPermissions": [
    {
      "IpProtocol": "tcp",
      "FromPort": 22,
      "ToPort": 22,
      "IpRanges": [
        {
          "CidrIp": "0.0.0.0/0"
        }
      ]
    }
  ]
}

bastion

 ssh -J [email protected] [email protected]