Hive安装问题简述
??本文簡要介紹Hive安裝過程中的常見問題:版本下載地址、版本依賴、依賴jar包。重要的是,記錄Hive的元數據庫手動初始化時報錯的解決方案并對其進行反思
Hive的元數據庫手動初始化
當使用hive 2.x之前的版本時,也可以不做初始化,當 hive 進行第一次啟動時,會自動進行初始化,只是不會生成足夠多的元數據庫中的表,這些表需要在使用過程中慢慢生成。
如果使用的是Hive2.x版本,則Hive的元數據庫必須手動初始化。使用的命令和顯示界面如下:
在初始化元數據庫的過程中報錯
$cd /usr/local/java/hive/hive-2.1.0/bin # 進入hive安裝目錄 $./schematool -dbType mysql -initSchema # 對hive進行重新初始化報錯信息如下:
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ ./schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/java/hive-2.1.0/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/java/hadoop/hadoop-2.7.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://localhost:3306/db_hive?createDatabaseIfNotExist=true&serverTimezone=GMT Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: root Mon Nov 29 14:50:31 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failureThe last packet successfully received from the server was 625 milliseconds ago. The last packet sent successfully to the server was 611 milliseconds ago. SQL Error code: 0 Use --verbose for detailed stacktrace. *** schemaTool failed ***在啟動hive時報錯
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ hive報錯信息如下(僅截取部分報錯信息):
hadoop@ubuntu16:/usr/local/java/hive-2.1.0/bin$ ./hive SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/java/hive-2.1.0/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/java/hadoop/hadoop-2.7.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Logging initialized using configuration in jar:file:/usr/local/java/hive-2.1.0/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Mon Nov 29 14:18:57 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)因為ubuntu的終端控制臺都是白色字體,而且輸出內容很多,很難找到報錯信息,因此可以根據報錯信息選擇一種類型的文件,將報錯信息存至文件中。因為hadoop、hive基于java語言,所以.java為后綴名的文件可以看到字體高亮。
在初始化hive元數據庫時,出現(xiàn)hive的報錯信息*** schemaTool failed ***
源于上面其他的錯誤信息:org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
將第一個報錯信息hive.metastore.HiveMetaException: Failed to get schema version.復制并在瀏覽器中搜索,尋找解決方法
解決方法:設置mysql時區(qū),useSSL1
<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://localhost:3306/db_hive?createDatabaseIfNotExist=true&serverTimezone=GMT&useSSL=false</value><description>JDBC connect string for a JDBC metastore</description> </property>反思
Hive安裝過程中的其他問題
hive下載地址
APACHE官網: Index of /hive (apache.org)
推薦(版本較多):華為云開源軟件鏡像網站-APACHE: Index of apache-local/hive (huaweicloud.com)
hadoop和hive版本對應關系
如:hive2.3.8 依賴于hadoop2.x.y,其他版本信息見:Downloads (apache.org)
JDBC驅動下載 mysql-connector-java-x.y.z.jar
Download MySQL Connector/J (Archived Versions)
錯誤:org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version_葉蕖的博客-CSDN博客 ??
總結
以上是生活随笔為你收集整理的Hive安装问题简述的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: hadoop fs默认操作路径
- 下一篇: 字符串是否存在