[SPARK-49714] Add Java 21-based SparkPi example

### What changes were proposed in this pull request? This PR aims to add `Java 21`-based SparkPi example. ### Why are the changes needed? Apache Spark starts to publish Java 21-based image from Today (2024-09-19). This new example will illustrate how to use it. - apache/spark-docker#69 ### Does this PR introduce _any_ user-facing change? No, this is a new example. ### How was this patch tested? Manual review. ``` $ kubectl apply -f examples/pi-java21.yaml ``` ``` $ kubectl get sparkapp NAME CURRENT STATE AGE pi-java21 ResourceReleased 28s ``` ``` $ kubectl describe sparkapp pi-java21 Name: pi-java21 Namespace: default Labels: <none> Annotations: <none> API Version: spark.apache.org/v1alpha1 Kind: SparkApplication Metadata: Creation Timestamp: 2024-09-19T04:08:16Z Finalizers: sparkapplications.spark.apache.org/finalizer Generation: 2 Resource Version: 95294 UID: 2bc46e8d-6339-4867-9a28-6552c6c471e5 Spec: Application Tolerations: Application Timeout Config: Driver Ready Timeout Millis: 300000 Driver Start Timeout Millis: 300000 Executor Start Timeout Millis: 300000 Force Termination Grace Period Millis: 300000 Termination Requeue Period Millis: 2000 Instance Config: Init Executors: 0 Max Executors: 0 Min Executors: 0 Resource Retain Policy: OnFailure Restart Config: Max Restart Attempts: 3 Restart Backoff Millis: 30000 Restart Policy: Never Deployment Mode: ClusterMode Driver Args: Jars: local:///opt/spark/examples/jars/spark-examples.jar Main Class: org.apache.spark.examples.SparkPi Runtime Versions: Scala Version: 2.13 Spark Version: 4.0.0-preview1 Spark Conf: spark.dynamicAllocation.enabled: true spark.dynamicAllocation.maxExecutors: 3 spark.dynamicAllocation.shuffleTracking.enabled: true spark.kubernetes.authenticate.driver.serviceAccountName: spark spark.kubernetes.container.image: apache/spark:4.0.0-preview1-java21-scala spark.log.structuredLogging.enabled: false Status: Current Attempt Summary: Attempt Info: Id: 0 Current State: Current State Summary: ResourceReleased Last Transition Time: 2024-09-19T04:08:33.316041381Z State Transition History: 0: Current State Summary: Submitted Last Transition Time: 2024-09-19T04:08:16.584629470Z Message: Spark application has been created on Kubernetes Cluster. 1: Current State Summary: DriverRequested Last Transition Time: 2024-09-19T04:08:17.269457304Z Message: Requested driver from resource scheduler. 2: Current State Summary: DriverStarted Last Transition Time: 2024-09-19T04:08:17.809898304Z Message: Driver has started running. 3: Current State Summary: DriverReady Last Transition Time: 2024-09-19T04:08:17.810393971Z Message: Driver has reached ready state. 4: Current State Summary: RunningHealthy Last Transition Time: 2024-09-19T04:08:17.828526471Z Message: Application is running healthy. 5: Current State Summary: Succeeded Last Transition Time: 2024-09-19T04:08:33.241514089Z Message: Spark application completed successfully. 6: Current State Summary: ResourceReleased Last Transition Time: 2024-09-19T04:08:33.316041381Z Events: <none> ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #129 from dongjoon-hyun/SPARK-49714. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

Author: dongjoon-hyun

examples/pi-java21.yaml

@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+apiVersion: spark.apache.org/v1alpha1
+kind: SparkApplication
+metadata:
+ name: pi-java21
+spec:
+ mainClass: "org.apache.spark.examples.SparkPi"
+ jars: "local:///opt/spark/examples/jars/spark-examples.jar"
+ sparkConf:
+ spark.dynamicAllocation.enabled: "true"
+ spark.dynamicAllocation.shuffleTracking.enabled: "true"
+ spark.dynamicAllocation.maxExecutors: "3"
+ spark.log.structuredLogging.enabled: "false"
+ spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
+ spark.kubernetes.container.image: "apache/spark:4.0.0-preview1-java21-scala"
+ applicationTolerations:
+ resourceRetainPolicy: OnFailure
+ runtimeVersions:
+ scalaVersion: "2.13"
+ sparkVersion: "4.0.0-preview1"