[SPARK-49714] Add Java 21-based SparkPi example

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add Java 21-based SparkPi example.

Why are the changes needed?

Apache Spark starts to publish Java 21-based image from Today (2024-09-19). This new example will illustrate how to use it.

• [SPARK-49703] Publish Java 21 Docker image for preview1 spark-docker#69

Does this PR introduce any user-facing change?

No, this is a new example.

How was this patch tested?

Manual review.

$ kubectl apply -f examples/pi-java21.yaml

$ kubectl get sparkapp
NAME        CURRENT STATE      AGE
pi-java21   ResourceReleased   28s

$ kubectl describe sparkapp pi-java21
Name:         pi-java21
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  spark.apache.org/v1alpha1
Kind:         SparkApplication
Metadata:
  Creation Timestamp:  2024-09-19T04:08:16Z
  Finalizers:
    sparkapplications.spark.apache.org/finalizer
  Generation:        2
  Resource Version:  95294
  UID:               2bc46e8d-6339-4867-9a28-6552c6c471e5
Spec:
  Application Tolerations:
    Application Timeout Config:
      Driver Ready Timeout Millis:            300000
      Driver Start Timeout Millis:            300000
      Executor Start Timeout Millis:          300000
      Force Termination Grace Period Millis:  300000
      Termination Requeue Period Millis:      2000
    Instance Config:
      Init Executors:        0
      Max Executors:         0
      Min Executors:         0
    Resource Retain Policy:  OnFailure
    Restart Config:
      Max Restart Attempts:    3
      Restart Backoff Millis:  30000
      Restart Policy:          Never
  Deployment Mode:             ClusterMode
  Driver Args:
  Jars:        local:///opt/spark/examples/jars/spark-examples.jar
  Main Class:  org.apache.spark.examples.SparkPi
  Runtime Versions:
    Scala Version:  2.13
    Spark Version:  4.0.0-preview1
  Spark Conf:
    spark.dynamicAllocation.enabled:                          true
    spark.dynamicAllocation.maxExecutors:                     3
    spark.dynamicAllocation.shuffleTracking.enabled:          true
    spark.kubernetes.authenticate.driver.serviceAccountName:  spark
    spark.kubernetes.container.image:                         apache/spark:4.0.0-preview1-java21-scala
    spark.log.structuredLogging.enabled:                      false
Status:
  Current Attempt Summary:
    Attempt Info:
      Id:  0
  Current State:
    Current State Summary:  ResourceReleased
    Last Transition Time:   2024-09-19T04:08:33.316041381Z
  State Transition History:
    0:
      Current State Summary:  Submitted
      Last Transition Time:   2024-09-19T04:08:16.584629470Z
      Message:                Spark application has been created on Kubernetes Cluster.
    1:
      Current State Summary:  DriverRequested
      Last Transition Time:   2024-09-19T04:08:17.269457304Z
      Message:                Requested driver from resource scheduler.
    2:
      Current State Summary:  DriverStarted
      Last Transition Time:   2024-09-19T04:08:17.809898304Z
      Message:                Driver has started running.
    3:
      Current State Summary:  DriverReady
      Last Transition Time:   2024-09-19T04:08:17.810393971Z
      Message:                Driver has reached ready state.
    4:
      Current State Summary:  RunningHealthy
      Last Transition Time:   2024-09-19T04:08:17.828526471Z
      Message:                Application is running healthy.
    5:
      Current State Summary:  Succeeded
      Last Transition Time:   2024-09-19T04:08:33.241514089Z
      Message:                Spark application completed successfully.
    6:
      Current State Summary:  ResourceReleased
      Last Transition Time:   2024-09-19T04:08:33.316041381Z
Events:                       <none>

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @viirya

[dongjoon-hyun]

Thank you! Merged to main.