完整報(bào)錯(cuò)如下:

(Python3.6) appleyuchi@Desktop:target$ spark-submit --class WordCountLocal --master yarn --deploy-mode cluster ?--driver-memory 1g --executor-memory 1g --executor-cores 1 ?/home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar
20/04/26 18:12:33 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
?

?

解決方案:
(Python3.6) appleyuchi@Desktop:~$ hdfs dfs -mkdir /spark
(Python3.6) appleyuchi@Desktop:~$ hdfs dfs -mkdir /spark/jars
(Python3.6) appleyuchi@Desktop:~$ hdfs?dfs -put ?$SPARK_HOME/jars/* hdfs://Desktop:9000/spark/jars/

?

上傳后查看：
hdfs dfs -ls hdfs://Desktop:9000/spark/jars

?

spark-defaults.conf中寫(xiě)入

spark.yarn.jars hdfs://Desktop:9000/spark/jars

?

再次提交任務(wù):

spark-submit --class WordCountLocal --master yarn --deploy-mode cluster ?--driver-memory 1g --executor-memory 1g --executor-cores 1 ?/home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar

總結(jié)

以上是生活随笔為你收集整理的Neither spark.yarn.jars nor spark.yarn.archive is set的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。