[bug] Spark Operator fails to create due to karpenter resources

[alanty]

Description

When creating the stack on latest this morning i saw a failure due to the karpenter resources that are created via karpenter_resources_helm_config

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Versions

• Module version [Required]: v1.0.3 (latest)

• Terraform version: v1.9.5

• Provider version(s):

• provider registry.terraform.io/gavinbunney/kubectl v1.14.0
• provider registry.terraform.io/hashicorp/aws v5.70.0
• provider registry.terraform.io/hashicorp/cloudinit v2.3.5
• provider registry.terraform.io/hashicorp/helm v2.15.0
• provider registry.terraform.io/hashicorp/kubernetes v2.32.0
• provider registry.terraform.io/hashicorp/random v3.3.2
• provider registry.terraform.io/hashicorp/time v0.12.1
• provider registry.terraform.io/hashicorp/tls v4.0.6

Reproduction Code [Required]

Steps to reproduce the behavior:
Pull the latest and try to create the spark operator v4 solution

Expected behavior

the stack creates successfully with the ./install.sh script

Actual behavior

The stack failed with the error below while creating the module.eks_data_addons.helm_release.karpenter_resources resources

│ Error: unable to build kubernetes objects from release manifest: [resource mapping not found for name: "spark-memory-optimized" namespace: "" from "": no matches for kind "EC2NodeClass" in version "karpenter.k8s.aws/v1"
│ ensure CRDs are installed first, resource mapping not found for name: "spark-memory-optimized" namespace: "" from "": no matches for kind "NodePool" in version "karpenter.sh/v1"
│ ensure CRDs are installed first]
│
│   with module.eks_data_addons.helm_release.karpenter_resources["spark-memory-optimized"],
│   on .terraform/modules/eks_data_addons/karpenter-resources.tf line 6, in resource "helm_release" "karpenter_resources":
│    6: resource "helm_release" "karpenter_resources" {
│
╵
╷
│ Error: unable to build kubernetes objects from release manifest: [resource mapping not found for name: "spark-vertical-ebs-scale" namespace: "" from "": no matches for kind "EC2NodeClass" in version "karpenter.k8s.aws/v1"
│ ensure CRDs are installed first, resource mapping not found for name: "spark-vertical-ebs-scale" namespace: "" from "": no matches for kind "NodePool" in version "karpenter.sh/v1"
│ ensure CRDs are installed first]
│
│   with module.eks_data_addons.helm_release.karpenter_resources["spark-vertical-ebs-scale"],
│   on .terraform/modules/eks_data_addons/karpenter-resources.tf line 6, in resource "helm_release" "karpenter_resources":
│    6: resource "helm_release" "karpenter_resources" {
│
╵

Additional context

Looks like aws-ia/terraform-aws-eks-data-addons#45 updated the resources to karpenter v1 spec, since the ones in spark operator are still beta i think we just need to update the nodepools/classes.

This can also take care of #611 for the spark operator solution 🥳

[askulkarni2]

This is a regression. The karpenter v1 changes should've been applied to all existing blueprints.

cc: @vara-bonthu @chrismld

[vara-bonthu]

Customers can pin the Data addons version to previous tag to work with old Karpenter version.

[alanty]

handled in #683