[Docs] Kubernetes quickstart flow troubleshooting

[mykolaharmash]

We need a place in our docs to link users for troubleshooting issues with the kubectl kustomize command.

This could be a new section inside Scaling Elastic Agent on Kubernetes or a new dedicated page.

There is a Figma mockup with some potential troubleshooting steps that might be used as a starting point for the docs page.

[akhileshpok]

@bturquet, @gizas - Is there any existing doc for troubleshooting of the agent in K8s that we can refer to from within the new onboarding flow? Thanks

[gizas]

We dont have something specific!

For start, we can refer to below general ones:

• Troubleshooting Common Problems
• Frequently Asked Questions

We have this one as part of the current PR
https://github.com/elastic/elastic-agent/blob/d1a28379019332e27395259912e3675871720a0a/deploy/kubernetes/elastic-agent-kustomize/default/README.md

I would also suggest to include:

• Command to check the output of kustomize rendering:
  kubectl kustomize elastic-agent-kustomize/default/elastic-agent-standalone/ >> kustomize.yaml
• Command to get cluster status:
  kubectl get pods -n kube-system | grep elastic
• Command to describe agent:
  kubectl describe -n kube-system <name_of_elastic_agent_pod>
• Check logs of elastic-agent
  kubectl logs -n kube-system <name_of_elastic_agent_pod>
kubectl logs -n kube-system <name_of_elastic_agent_pod> | grep -i error

This could be a new section inside Scaling Elastic Agent on Kubernetes or a new dedicated page.

I would advise not to change the above page as it has specific goal.

My advise is to link the new page/ guide possibly to:

• https://www.elastic.co/getting-started/observability/monitor-kubernetes-clusters
• https://www.elastic.co/observability/kubernetes-monitoring
  (which are entry pages to our observability offering for k8s)

This is the repo where rest of elastic-agent pages for k8s leave

[akhileshpok]

Thanks @gizas. @mykolaharmash - Probably best to link the current 'Troubleshooting Common Problems' doc to the new onboarding flow. cc-@bmorelli25 - We should try to improve the current 'Troubleshooting Common Problems' doc based on the suggestions provided by @gizas.

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[flash1293]

@mykolaharmash @akhileshpok can we close this?

[mykolaharmash]

@flash1293 probably not, we still don't have anything about kustomize flow troubleshooting in the docs (like commands that @gizas provided)

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[elasticmachine]

Pinging @elastic/obs-ux-onboarding-team (Feature: Observability Onboarding)

[bmorelli25]

@eedugon is working on a proposal for improving our Kubernetes docs across Elastic (https://github.com/elastic/ingest-dev/issues/3633#issuecomment-2342936676).

Edu, can you take a look at this issue and make sure the gaps in our documentation explained here are included in your plan?

[eedugon]

@gizas , @mykolaharmash , @akhileshpok : I have a quick question before moving forward with this:

Why the new onboarding workflow in the UX is suggesting a manifest creation method (kustomize) that doesn't appear (or at least i can't find it) anywhere in the docs?

We are telling the users in the UI to use kustomize while all our existing docs still consist of downloading a manifest and updating it before being applied. I'm talking about Running Elastic Agent on Kubernetes and similar docs.

Shouldn't the UX be aligned with the rest of the docs? Are we planning to update also other docs in the near future and start using kustomize related manifests to prepare the users manifests?

Anyway, coming back to this issue:

• I will start by adding the troubleshooting commands and suggestions in a new section of the Troubleshooting Common Problems doc, called Kustomize based installations of Elastic Agent, and in case of needing to move them somewhere else in the future that will be easy.

• Then we can update the link of the onboarding UX to point to that new section of the troubleshooting doc (@mykolaharmash , would you change that or should I make that change too? ).

Let us know if that looks good and aligned with the expectations.

[gizas]

Heyyyy @eedugon (nice seeing you on the dark-side)

This is the reference: https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes#kustomize-templates
In more details see the updated readme

There was a new initiative/ project where we tried to ease the onboarding of users (and mainly install along with the manifest additional resources/assets needed like dashboards and kube-state-metrics with one command). I guess @akhileshpok and @flash1293 might have more documentation for the project goal.

[eedugon]

Thanks @gizas for sharing the initiative, and I think it's really great to use kustomize for this (with some caveats like maybe kube-state-metrics could be already installed, and other possible issues).

I'm going to proceed and create a troubleshooting section for possible issues related with this approach (as requested in the issue).

However let me add some comments about the lack of alignment between this new approach (which is great) and our official docs. @bmorelli25 , I'd like you and the team to analyze the following to see if we should create other follow-up issues:

• The new onboarding for k8s suggests an Standalone Elastic Agent installation with Kustomize, and our official doc to Run Standalone Elastic Agent on Kubernetes doesn't cover this type of installation, which could cause confusion on users and customers ---> I would suggest at some point, if we validate this kustomize installation, to include the details on the official doc.

  • @gizas has shared the current documentation we have around this type of deployment (https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes#kustomize-templates), so need to decide if it makes more sense to leave it in Github and link it from the docs, or to integrate it somehow in the docs.

• Eventually the details around kustomize related installations and possible issues might fit better in a dedicated doc (the doc that covers the details of this installation type and its possibilities), or in the same place shared by @gizas (https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes#kustomize-templates) than in the troubleshooting section of Ingest. The reason is that probably we have to provide a lot of details for such an installation.

Anyway I will start working on the troubleshooting details and propose something.

[eedugon]

@mykolaharmash : where is the Open documentation link of the screenshot you shared pointing?

I'm preparing a PR for ingest-docs to make troubleshooting content available at: /fleet-troubleshooting.html#agent-kubernetes-kustomize, so, something like:

https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html#agent-kubernetes-kustomize.

But if you just point to https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html that might be also ok.

[eedugon]

@gizas : I've created a PR elastic/ingest-docs#1409 that should add the content you mentioned at #186382 (comment), and with extra details. I'd like to get your review on it when possible.

@akhileshpok , @bmorelli25 , the PR is not very long, but considering Kubernetes is a complete platform and that we might want to add content to the troubleshooting agent on Kubernetes section in the future, it might be worthy to separate it to a new document.

For the moment I've just added the content to 'Troubleshooting Common Problems' as agreed at #186382 (comment)

[mykolaharmash]

@eedugon currently the link point to https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html but I think we should update it to point specifically to the troubleshooting section once the doc is live.