kylin 部分问题
生活随笔
收集整理的這篇文章主要介紹了
kylin 部分问题
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
1.低版本kylin報錯 Failed to find metadata store by url: kylin_metadata@hbase
DEBUG [main] hbase.HBaseConnection:307 : HTable 'kylin_metadata' already exists Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbaseat org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:96)at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:108)at org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:94)at org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41)原因:3.0的kylin會自己創建元數據庫,2.6,2.5需要自己手動創建元數據庫
解決:進入hbase,創建kylin元數據庫 kylin_metadata
2.build報錯Call From hadoop02/192.168.146.201 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: 拒絕連接
org.apache.kylin.engine.mr.exception.MapReduceException: Exception: java.net.ConnectException: Call From hadoop02/192.168.146.201 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: 拒絕連接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused java.net.ConnectException: Call From dxt102/192.168.1.102 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: 拒絕連接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefusedat org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173)at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)原因:在使用kylin的時候需要開啟歷史服務器,Hadoop的historyserver服務沒有啟動
解決:啟動Hadoop的historyserver的進程服務
3.spark模式運行到Build Cube with Spark:報錯Container [pid=6263,containerID=container_1494900155967_0001_02_000001] is running beyond virtual memory limits
Diagnostics: Container [pid=6263,containerID=container_1494900155967_0001_02_000001] is running beyond virtual memory limits. Current usage: 107.3 MB of 1 GB physical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing container.原因:Container要用2.2GB的內存,而虛擬內存只有2.1GB,不夠用了,所以Kill了Container。
SPARK-EXECUTOR-MEMORY設置的是1G,即物理內存是1G,Yarn默認的虛擬內存和物理內存比例是2.1,也就是說虛擬內存是2.1G,小于了需要的內存2.2G。
解決辦法:把虛擬內存和物理內存比例增大,在yarn-site.xml中增加一個設置:
總結
以上是生活随笔為你收集整理的kylin 部分问题的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 深圳多区电闪雷鸣:暴雨突袭
- 下一篇: Vivo手机前置摄像头带补光功能吗