Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude NotReady nodes from Node Count #355

Open
morenod opened this issue Mar 28, 2022 · 1 comment
Open

Exclude NotReady nodes from Node Count #355

morenod opened this issue Mar 28, 2022 · 1 comment

Comments

@morenod
Copy link
Collaborator

morenod commented Mar 28, 2022

When generating the number of pods to create and labeling the nodes, we should exclude NotReady nodes, to avoid generating a number of pods higher to number of schedulable pods

ip-10-0-210-139.us-west-2.compute.internal NotReady,SchedulingDisabled worker 57m v1.23.5+b0357ed

lun 28 mar 2022 12:03:17 UTC Labeling 27 worker nodes with node-density=enabled
[...]
node/ip-10-0-210-139.us-west-2.compute.internal labeled
[...]

lun 28 mar 2022 12:03:47 UTC ###############################################
lun 28 mar 2022 12:03:47 UTC Workload: node-density
lun 28 mar 2022 12:03:47 UTC Workload template: workloads/node-pod-density/node-pod-density.yml
lun 28 mar 2022 12:03:47 UTC Metrics profile: metrics-profiles/metrics.yaml
lun 28 mar 2022 12:03:47 UTC Alerts profile:
lun 28 mar 2022 12:03:47 UTC QPS: 20
lun 28 mar 2022 12:03:47 UTC Burst: 20
lun 28 mar 2022 12:03:47 UTC Node count: 27
lun 28 mar 2022 12:03:47 UTC Pods per node: 245
lun 28 mar 2022 12:03:47 UTC ###############################################

lun 28 mar 2022 12:03:17 UTC Number of pods to deploy on nodes: 6181

Node count should be 26, not 27

@dry923
Copy link
Member

dry923 commented Mar 28, 2022

I disagree with this as an issue. During our testing we expect all nodes to be ready so we can accurately say we did a node density test on 27 nodes, for example. If nodes are not ready I expect this to fail. Perhaps we could expedite the failure by checking before starting the actual workload but otherwise I don't see an "issue" here IMHO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants