?databricks加載MySQL數(shù)據(jù)，需要先安裝MySQL的驅(qū)動(dòng)包。可以從集群的Libraries安裝。上傳jar包即可。

查詢?nèi)缦?/p> driver = "com.mysql.jdbc.Driver" url = dbutils.secrets.get(scope = "db_test", key = "url") user = dbutils.secrets.get(scope = "db_test", key = "username") password = dbutils.secrets.get(scope = "db_test", key = "password") obj = spark.read.format("jdbc").option("driver", driver).\ option("url", url).option("user", user).option("password", password)table_sql = "(select * from students) tmp" df_students = obj.option("dbtable", table_sql).load() display(df_students)

?

創(chuàng)建臨時(shí)表

df_students.createOrReplaceTempView("test_student")

修改列名

df_students_new = df_students.withColumnRenamed("name","student_name" )

表連接

#全連接 df_union1 = df_students_new.join(df_students,"id","full") #左外連接 df_union2 = df_students_new.join(df_students,"id","left_outer") #右外連接 df_union3 = df_students_new.join(df_students,"id","right_outer")

排序

df_union1 = df_union1.sort("age")

?轉(zhuǎn)Pandas

df_union1 = df_union1.toPandas()

總結(jié)

以上是生活随笔為你收集整理的Databricks 加载MySQL数据的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。