Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]1.0 - elasticsearch cluster is always in Creating status due to FailedScheduling #8402

Closed
tianyue86 opened this issue Nov 5, 2024 · 1 comment
Assignees
Labels
kind/bug Something isn't working
Milestone

Comments

@tianyue86
Copy link

Describe the bug

Kubernetes: v1.30.4+k3s1
KubeBlocks: 1.0.0-beta.0
kbcli: 1.0.0-alpha.0
elasticsearch                  1.0.0-alpha.0   community   Enabled    true 

To Reproduce
Steps to reproduce the behavior:

  1. Get the latest es cluster yaml
helm template esclu02 kubeblocks-addons/elasticsearch-cluster --version 1.0.0-alpha.0
---
# Source: elasticsearch-cluster/templates/cluster-multi-node.yaml
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: esclu02
  namespace: default
  labels: 
    helm.sh/chart: elasticsearch-cluster-1.0.0-alpha.0
    app.kubernetes.io/version: "8.8.2"
    app.kubernetes.io/instance: esclu02
  annotations:
    kubeblocks.io/extra-env: '{"mdit-roles":"master,data,ingest,transform","mode":"multi-node"}'
spec:
  terminationPolicy: Delete
  componentSpecs:
    - name: mdit
      componentDef: elasticsearch-8
      serviceVersion: 8.8.2
      serviceAccountName: kb-esclu02      
      schedulingPolicy:
        affinity:
          podAntiAffinity:
            preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/instance: esclu02
                    apps.kubeblocks.io/component-name: mdit
                topologyKey: kubernetes.io/hostname
              weight: 100
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/instance: esclu02
                  apps.kubeblocks.io/component-name: mdit
              topologyKey: kubernetes.io/hostname      
      replicas: 3      
      disableExporter: false      
      resources:
        limits:
          cpu: "1"
          memory: "2Gi"
        requests:
          cpu: "1"
          memory: "2Gi"      
      volumeClaimTemplates:
        - name: data # ref clusterDefinition components.containers.volumeMounts.name
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  1. Apply the yaml to create cluster
  2. Check the status of cluster - Creating
NAMESPACE   NAME          CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS     AGE
default     esclu02                            Delete               Creating   48m
  1. Check pod: Pending
tianyue@apeclouds-MacBook-Pro .kube % k get pod
NAME               READY   STATUS    RESTARTS   AGE
esclu02-mdit-0     0/3     Pending   0          58m
esclu02-mdit-1     0/3     Pending   0          58m
esclu02-mdit-2     0/3     Pending   0          58m
  1. Describe pod: FailedScheduling
tianyue@apeclouds-MacBook-Pro .kube % k describe pod esclu02-mdit-0
Events:
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  49m                default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: context deadline exceeded
  Warning  FailedScheduling  29m (x2 over 39m)  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: context deadline exceeded
  Warning  FailedScheduling  19m                default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: context deadline exceeded
  Warning  FailedScheduling  8m55s              default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: context deadline exceeded
  1. Describe cluster
tianyue@apeclouds-MacBook-Pro kbcli % k describe cluster esclu02
Name:         esclu02
Namespace:    default
Labels:       app.kubernetes.io/instance=esclu02
              app.kubernetes.io/version=8.8.2
              helm.sh/chart=elasticsearch-cluster-1.0.0-alpha.0
Annotations:  kubeblocks.io/extra-env: {"mdit-roles":"master,data,ingest,transform","mode":"multi-node"}
API Version:  apps.kubeblocks.io/v1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2024-11-05T06:32:23Z
  Finalizers:
    cluster.kubeblocks.io/finalizer
  Generation:        2
  Resource Version:  24685
  UID:               a6fdaa5a-4172-4e80-8ff5-fcf0434b00be
Spec:
  Component Specs:
    Component Def:     elasticsearch-8-1.0.0-alpha.0
    Disable Exporter:  false
    Name:              mdit
    Replicas:          3
    Resources:
      Limits:
        Cpu:     1
        Memory:  2Gi
      Requests:
        Cpu:     1
        Memory:  2Gi
    Scheduling Policy:
      Affinity:
        Pod Anti Affinity:
          Preferred During Scheduling Ignored During Execution:
            Pod Affinity Term:
              Label Selector:
                Match Labels:
                  app.kubernetes.io/instance:         esclu02
                  apps.kubeblocks.io/component-name:  mdit
              Topology Key:                           kubernetes.io/hostname
            Weight:                                   100
          Required During Scheduling Ignored During Execution:
            Label Selector:
              Match Labels:
                app.kubernetes.io/instance:         esclu02
                apps.kubeblocks.io/component-name:  mdit
            Topology Key:                           kubernetes.io/hostname
    Service Account Name:                           kb-esclu02
    Service Version:                                8.8.2
    Volume Claim Templates:
      Name:  data
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:   20Gi
  Termination Policy:  Delete
Status:
  Components:
    Mdit:
      Phase:  Creating
  Conditions:
    Last Transition Time:  2024-11-05T06:32:23Z
    Message:               The operator has started the provisioning of Cluster: esclu02
    Observed Generation:   2
    Reason:                PreCheckSucceed
    Status:                True
    Type:                  ProvisioningStarted
    Last Transition Time:  2024-11-05T06:32:23Z
    Message:               Successfully applied for resources
    Observed Generation:   2
    Reason:                ApplyResourcesSucceed
    Status:                True
    Type:                  ApplyResources
  Observed Generation:     2
  Phase:                   Creating
Events:                    <none>

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@tianyue86 tianyue86 added the kind/bug Something isn't working label Nov 5, 2024
@shanshanying
Copy link
Contributor

shanshanying commented Nov 6, 2024

running PreBind plugin "VolumeBinding":

What is the SC you are working with? Please check PV and PVC status.

@github-actions github-actions bot added this to the Release 0.9.2 milestone Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants