k8s-1.30: Use kubelet's drop-in config files

I've verified that in k8s-1.30, removing all options set in
KUBELET_EXTRA_ARS, would result in the following diff:

    @@ -54 +53,0 @@
    -    "kubeletCgroups": "/systemd/system.slice",
    @@ -87 +86 @@
    -    "failSwapOn": false,
    +    "failSwapOn": true,

I've moved both kubeletCgroups and failSwapOn into kubelet's drop-in
config file and verified that it works and config is set.

I assume that --cgroup-driver=systemd is already the default in centos
stream or being set somewhere else, but to avoid regressions I've
included in the config file as well.

I don't know why --runtime-cgroups is not being shown in the node's
configz but it is the only command line option for the initial 4 that
is not marked as DEPRECATED in favor of Config file alternative, so
let's keep it as command line option, together with --config-dir.

Signed-off-by: Victor Toso <[email protected]>

cluster-provision/k8s/1.30/k8s_provision.sh

@@ -153,9 +153,19 @@ cp /tmp/local-volume.yaml /provision/local-volume.yaml
 kubelet_conf_d="/etc/kubernetes/kubelet.conf.d"
 mkdir -m 644 $kubelet_conf_d
 
-# TODO use config file! this is deprecated
+# Set our custom initializations to kubelet
+kubevirt_kubelet_conf="$kubelet_conf_d/50-kubevirt.conf"
+cat <<EOF >$kubevirt_kubelet_conf
+apiVersion: kubelet.config.k8s.io/v1beta1
+kind: KubeletConfiguration
+cgroupDriver: systemd
+failSwapOn: false
+kubeletCgroups: /systemd/system.slice
+EOF
+
+# Set only command line options not supported by config
 cat <<EOT >/etc/sysconfig/kubelet
-KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --runtime-cgroups=/systemd/system.slice  --fail-swap-on=false --kubelet-cgroups=/systemd/system.slice --config-dir=$kubelet_conf_d
+KUBELET_EXTRA_ARGS=--runtime-cgroups=/systemd/system.slice --config-dir=$kubelet_conf_d
 EOT
 
 # Enable userfaultfd for centos9 to support post-copy live migration.