Heimdall is a dashboard for operating Flink jobs and deployments. Built-in Flink UI is extremely useful when dealing with a single job, but what if you have 10, 20 or 100 jobs? Heimdall helps to keep track of all your Flink jobs by searching, filtering, sorting and navigating them.
Read more in my blog: Heimdall: Making Operating Flink Deployments a Bit Easier.
Note: currently, Heimdall only supports Flink jobs deployed with Flink Kubernetes Operator.
kubectl apply -f https://raw.githubusercontent.com/sap1ens/heimdall/main/tools/k8s-operator/service-account.yaml
kubectl run heimdall --image=ghcr.io/sap1ens/heimdall:0.5.0 --port=8080 --overrides='{ "spec": { "serviceAccount": "heimdall-service-account" } }'
kubectl port-forward heimdall 8080:8080
open http://localhost:8080Heimdall is available as a Docker container here. Since only Flink Kubernetes Operator as available at the moment you'd typically deploy Heimdall as a pod or deployment in the same cluster.
Heimdall uses port 8080 and exposes liveness and readiness endpoints at /q/health/live and /q/health/ready.
Job locator (implements FlinkJobLocator interface) is a mechanism for discovering running Flink jobs in Heimdall.
Can be enabled with heimdall.joblocator.k8s-operator.enabled config. Currently, enabled by default. It loads
flinkdeployment custom resource (CR) created by the Flink Kubernetes Operator.
A service account with read-only access to flinkdeployment CR is required. See tools/k8s-operator/service-account.yaml for an example.
By default, all data received from a locator is cached in memory for 5 seconds. This can be disabled or configured to use a different interval (see this page).
| Environment variable | Default | Description |
|---|---|---|
| QUARKUS_HTTP_CORS_ORIGINS | http://localhost:5173 | Comma separated list of valid origins allowed for CORS. Change to http://localhost:8001 when using kubectl proxy. |
| HEIMDALL_PATTERNS_DISPLAY_NAME | $jobName | Pattern for showing Flink job name. Metadata fields (e.g. Kubernetes labels) can be accessed via $metadata.labelName syntax. |
| HEIMDALL_ENDPOINT_PATH_PATTERNS_FLINK_UI | http://localhost/$jobName/ui | Pattern for the Flink UI endpoint. $jobName will be substituted. Set to an empty string to disable. |
| HEIMDALL_ENDPOINT_PATH_PATTERNS_FLINK_API | http://localhost/$jobName/api | Pattern for the Flink API endpoint. $jobName will be substituted. Set to an empty string to disable. |
| HEIMDALL_ENDPOINT_PATH_PATTERNS_METRICS | http://localhost/$jobName/metrics | Pattern for the Metrics endpoint. $jobName will be substituted. Set to an empty string to disable. |
| HEIMDALL_ENDPOINT_PATH_PATTERNS_LOGS | http://localhost/$jobName/logs | Pattern for the Logs endpoint. $jobName will be substituted. Set to an empty string to disable. |
| Environment variable | Default | Description |
|---|---|---|
| HEIMDALL_JOBLOCATOR_K8S_OPERATOR_ENABLED | true | Is this locator enabled? |
| HEIMDALL_JOBLOCATOR_K8S_OPERATOR_NAMESPACE_TO_WATCH | default | Kubernetes namespace to use. |
This project uses:
You can run your application in dev mode that enables live coding using:
./gradlew quarkusDevThe application can be packaged using:
./gradlew buildIt produces the quarkus-run.jar file in the build/quarkus-app/ directory.
Be aware that it’s not an über-jar as the dependencies are copied into the build/quarkus-app/lib/ directory.
The application is now runnable using java -jar build/quarkus-app/quarkus-run.jar.
If you want to build an über-jar, execute the following command:
./gradlew build -Dquarkus.package.type=uber-jarThe application, packaged as an über-jar, is now runnable using java -jar build/*-runner.jar.
This project is sponsored by Goldsky ❤️.