[SPARK-45497] Add a symbolic link file spark-examples.jar in K8s Docker images

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add a symbolic link file, spark-examples.jar, in the example jar directory.

Apache Spark repository is updated already via

• [SPARK-45497][K8S] Add a symbolic link file spark-examples.jar in K8s Docker images spark#43324

$ docker run -it --rm spark:latest ls -al /opt/spark/examples/jars | tail -n6 total 1620 drwxr-xr-x 1 root root 4096 Oct 11 04:37 . drwxr-xr-x 1 root root 4096 Sep 9 02:08 .. -rw-r--r-- 1 root root 78803 Sep 9 02:08 scopt_2.12-3.7.1.jar -rw-r--r-- 1 root root 1564255 Sep 9 02:08 spark-examples_2.12-3.5.0.jar lrwxrwxrwx 1 root root 29 Oct 11 04:37 spark-examples.jar -> spark-examples_2.12-3.5.0.jar 

Why are the changes needed?

Like PySpark example (pi.py), we can submit the examples without considering the version numbers which was painful before.

bin/spark-submit \ --master k8s://$K8S_MASTER \ --deploy-mode cluster \ ... --class org.apache.spark.examples.SparkPi \ local:///opt/spark/examples/jars/spark-examples.jar 10000 

The following is the driver pod log.

+ exec /usr/bin/tini -s -- /opt/spark/bin/spark-submit ... --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.examples.SparkPi local:///opt/spark/examples/jars/spark-examples.jar 10000 Files local:///opt/spark/examples/jars/spark-examples.jar from /opt/spark/examples/jars/spark-examples.jar to /opt/spark/work-dir/./spark-examples.jar 

Does this PR introduce any user-facing change?

No, this is an additional file.

How was this patch tested?

Manually build the docker image and do ls.

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

Could you review this PR, @viirya ?

[dongjoon-hyun]

Thank you, @viirya ! Merged to master.

[dongjoon-hyun]

For the record, the fixed images are released.

$ docker run -it --rm apache/spark:4.0.0-preview1 ls -al /opt/spark/examples/jars/ total 2636 drwxr-xr-x 2 spark spark 4096 Sep 18 23:19 . drwxr-xr-x 1 spark spark 4096 May 28 16:39 .. -rw-r--r-- 1 spark spark 232248 May 28 16:39 jackson-core-asl-1.9.13.jar -rw-r--r-- 1 spark spark 780664 May 28 16:39 jackson-mapper-asl-1.9.13.jar -rw-r--r-- 1 spark spark 80424 May 28 16:39 scopt_2.13-3.7.1.jar -rw-r--r-- 1 spark spark 1591049 May 28 16:39 spark-examples_2.13-4.0.0-preview1.jar lrwxrwxrwx 1 root root 38 Sep 18 23:19 spark-examples.jar -> spark-examples_2.13-4.0.0-preview1.jar