Spark on K8S環境部署細節

https://www.cnblogs.com/lanrish/p/12267623.html

spark?history?helm部署

https://github.com/SnappyDataInc/spark-on-k8s/tree/master/charts/spark-hs?spm=5176.2020520152.0.0.2d5916ddP2xqfh

bin/spark-submit \--master k8s://https://<k8s-master-IP> \--deploy-mode cluster \--name spark-pi \--conf spark.kubernetes.namespace=spark \--class org.apache.spark.examples.SparkPi \--conf spark.eventLog.enabled=true \--conf spark.eventLog.dir=gs://spark-history-server/ \--conf spark.executor.instances=2 \--conf spark.hadoop.google.cloud.auth.service.account.json.keyfile=/etc/secrets/sparkonk8s-test.json \--conf spark.kubernetes.driver.secrets.history-secrets=/etc/secrets \--conf spark.kubernetes.executor.secrets.history-secrets=/etc/secrets \--conf spark.kubernetes.driver.docker.image=snappydatainc/spark-driver:v2.2.0-kubernetes-0.5.1 \--conf spark.kubernetes.executor.docker.image=snappydatainc/spark-executor:v2.2.0-kubernetes-0.5.1 \local:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.5.0.jar

在 K8S 部署一個 Spark History Server

https://blog.csdn.net/oscarun/article/details/100072656

總結

以上是生活随笔為你收集整理的Spark on K8S及history服务部署的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。