[SPARK-49715] Add Java 21-based SparkCluster example

### What changes were proposed in this pull request? This PR aims to add `Java 21`-based `SparkCluster` example. ### Why are the changes needed? Apache Spark starts to publish Java 21-based image from Today (2024-09-19). This new example will illustrate how to use it. - apache/spark-docker#69 ### Does this PR introduce _any_ user-facing change? No, this is a new example. ### How was this patch tested? Manual review. ``` $ kubectl apply -f cluster-java21.yaml ``` ``` $ kubectl get sparkcluster NAME CURRENT STATE AGE cluster-java21 RunningHealthy 9s ``` ``` $ kubectl describe sparkcluster cluster-java21 Name: cluster-java21 Namespace: default Labels: <none> Annotations: <none> API Version: spark.apache.org/v1alpha1 Kind: SparkCluster Metadata: Creation Timestamp: 2024-09-19T04:25:30Z Finalizers: sparkclusters.spark.apache.org/finalizer Generation: 2 Resource Version: 96663 UID: 9421c957-380d-4a26-a266-379304bf83ee Spec: Cluster Tolerations: Instance Config: Init Workers: 3 Max Workers: 3 Min Workers: 3 Master Spec: Runtime Versions: Spark Version: 4.0.0-preview1 Spark Conf: spark.kubernetes.container.image: apache/spark:4.0.0-preview1-java21 spark.master.rest.enabled: true spark.master.rest.host: 0.0.0.0 spark.master.ui.title: Prod Spark Cluster (Java 21) spark.ui.reverseProxy: true Worker Spec: Status: Current Attempt Summary: Attempt Info: Id: 0 Current State: Current State Summary: RunningHealthy Last Transition Time: 2024-09-19T04:25:30.665095088Z Message: Cluster has reached ready state. State Transition History: 0: Current State Summary: Submitted Last Transition Time: 2024-09-19T04:25:30.640072963Z Message: Spark cluster has been submitted to Kubernetes Cluster. 1: Current State Summary: RunningHealthy Last Transition Time: 2024-09-19T04:25:30.665095088Z Message: Cluster has reached ready state. Events: <none> ``` ``` $ k get pod NAME READY STATUS RESTARTS AGE cluster-java21-master-0 1/1 Running 0 3m20s cluster-java21-worker-0 1/1 Running 0 3m20s cluster-java21-worker-1 1/1 Running 0 3m20s cluster-java21-worker-2 1/1 Running 0 3m20s spark-kubernetes-operator-778b9bbdc6-fqks9 1/1 Running 0 20m ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #130 from dongjoon-hyun/SPARK-49715. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

Author: dongjoon-hyun

examples/cluster-java21.yaml

@@ -0,0 +1,32 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+apiVersion: spark.apache.org/v1alpha1
+kind: SparkCluster
+metadata:
+ name: cluster-java21
+spec:
+ runtimeVersions:
+ sparkVersion: "4.0.0-preview1"
+ clusterTolerations:
+ instanceConfig:
+ initWorkers: 3
+ minWorkers: 3
+ maxWorkers: 3
+ sparkConf:
+ spark.kubernetes.container.image: "apache/spark:4.0.0-preview1-java21"
+ spark.master.ui.title: "Prod Spark Cluster (Java 21)"
+ spark.master.rest.enabled: "true"
+ spark.master.rest.host: "0.0.0.0"
+ spark.ui.reverseProxy: "true"