test against k8s 1.24

[vreynolds]

internal slack thread

we're using an outdated k8s version in the e2e tests (through kind tool) - bump to latest version that runs k8s 1.24 (to be released April 19 2022)

check for deprecated features and consider upgrading our usage (e.g. sounds like /var/lib/docker/containers is deprecated in favor of /var/lib/containerd)

[ismith]

I decided to kick the tires a bit - the biggest worry I have re 1.24 is the move from docker to containerd.

Since my sandbox is EKS, the most recent version available is 1.22, but that defaults to containerd (containerd becomes mandatory in 1.24), so it seemed a useful check of metrics and logs.

Log tailing mostly worked - I set up a watcher for a nop parser, and saw data in honeycomb that looked right. Yay!

Metrics also put data into the appropriate dataset, though I'd want someone to take a look and make sure those values seem correct - I don't look at kubernetes-metrics often enough to be sure what should be in there.

So with those caveats, this seems like a good start. Doesn't guarantee that the kubernetes apis are all compatible (since, again, I tested in 1.22, not 1.24), but at least containerd vs docker doesn't seem to be a problem.

(I think to try 1.24 today we'd want to spin up a GKE cluster.)

[MikeGoldsmith]

We are using this internally in a later version of k8s (1.27). We will open issues as they come up.