在Ubuntu中安装HBase
關(guān)于作者：
- 張丹(Conan), 程序員Java,R,PHP,Javascript
- weibo：@Conan_Z
- blog: http://blog.fens.me
- email: bsspirit@gmail.com
轉(zhuǎn)載請(qǐng)注明出處：
http://blog.fens.me/linux-hbase-install/
前言
HBase是Hadoop家族中的一個(gè)分布式數(shù)據(jù)庫(kù)產(chǎn)品，HBase支持高并發(fā)讀寫(xiě)，列式數(shù)據(jù)存儲(chǔ)，高效的索引，自動(dòng)分片，自動(dòng)Region遷移等許多優(yōu)點(diǎn)，已經(jīng)越來(lái)越多的被界業(yè)認(rèn)可并實(shí)施。
目錄
1 在Ubuntu中環(huán)境準(zhǔn)備
HBase是基于Java開(kāi)發(fā)的運(yùn)行Hadoop平臺(tái)上分布式NoSQL數(shù)據(jù)庫(kù)軟件，HBase沒(méi)有提供Windows系統(tǒng)安裝版本。我在這里也只介紹HBase在Linux Ubuntu系統(tǒng)中的安裝。
由于HBase是運(yùn)行在Hadoop平臺(tái)上面的，因此我們需要先安裝好Hadoop的環(huán)境，Hadoop的安裝請(qǐng)參考文章：[Hadoop歷史版本安裝](http://blog.fens.me/hadoop-history-source-install/)
HBase沒(méi)有提供apt的軟件源安裝，我們需要自己去官方網(wǎng)絡(luò)下載HBase軟件包進(jìn)行安裝。HBase下載頁(yè)：http://www.apache.org/dyn/closer.cgi/hbase/
系統(tǒng)環(huán)境：
- Linux Ubuntu 12.04.2 LTS 64bit server
- Java JDK 1.6.0_45
- Hadoop 1.1.2
2 HBase安裝
2.1 下載HBase
# 通過(guò)wget命令下載 ~ wget http://www.gaidso.com/apache/hbase/stable/hbase-0.94.18.tar.gz# 解壓HBase ~ tar xvf hbase-0.94.18.tar.gz# 移動(dòng)HBase目錄到文件夾 ~ mv hbase-0.94.18/ /home/conan/hadoop/# 進(jìn)入目錄 ~ cd /home/conan/hadoop/hbase-0.94.182.2 配置HBase
2.2.1 修改啟動(dòng)文件hbase-env.sh
~ vi conf/hbase-env.sh#打開(kāi)注釋 export JAVA_HOME=/home/conan/toolkit/jdk16 export HBASE_CLASSPATH=/home/conan/hadoop/hadoop-1.1.2/conf export HBASE_MANAGES_ZK=true2.2.2 修改配置文件 hbase-site.xml
~ vi conf/hbase-site.xml<configuration> <property> <name>hbase.rootdir</name> <value>hdfs://master:9000/hbase</value> </property><property> <name>hbase.cluster.distributed</name> <value>true</value> </property><property> <name>dfs.replication</name> <value>1</value> </property><property> <name>hbase.zookeeper.quorum</name> <value>master</value> </property><property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property><property> <name>hbase.zookeeper.property.dataDir</name> <value>/home/conan/hadoop/hdata</value> </property> </configuration>復(fù)制hadoop環(huán)境的配置文件和類(lèi)庫(kù)
~ cp ~/hadoop/hadoop-1.1.2/conf/hdfs-site.xml conf/ ~ cp ~/hadoop/hadoop-1.1.2/hadoop-core-1.1.2.jar lib/ ~ mkdir /home/conan/hadoop/hdata2.3 啟動(dòng)hadoop和hbase
~ /home/conan/hadoop/hadoop-1.1.2/bin/start-all.sh ~ /home/conan/hadoop/hbase-0.94.18/bin/start-hbase.sh# 查看hbase進(jìn)程 ~ jps 13838 TaskTracker 13541 JobTracker 15946 HMaster 16756 Jps 12851 NameNode 13450 SecondaryNameNode 13133 DataNode 15817 HQuorumPeer 16283 HRegionServer2.4 打開(kāi)HBase命令行客戶(hù)端訪問(wèn)Hbase
~ bin/hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.94.18, r1577788, Sat Mar 15 04:46:47 UTC 2014hbase(main):002:0> help HBase Shell, version 0.94.18, r1577788, Sat Mar 15 04:46:47 UTC 2014 Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command. Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.COMMAND GROUPS:Group name: generalCommands: status, version, whoamiGroup name: ddlCommands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, is_disabled, is_enabled, list, show_filtersGroup name: dmlCommands: count, delete, deleteall, get, get_counter, incr, put, scan, truncateGroup name: toolsCommands: assign, balance_switch, balancer, close_region, compact, flush, hlog_roll, major_compact, move, split, unassign, zk_dumpGroup name: replicationCommands: add_peer, disable_peer, enable_peer, list_peers, list_replicated_tables, remove_peer, start_replication, stop_replicationGroup name: snapshotCommands: clone_snapshot, delete_snapshot, list_snapshots, restore_snapshot, snapshotGroup name: securityCommands: grant, revoke, user_permissionSHELL USAGE: Quote all names in HBase Shell such as table and column names. Commas delimit command parameters. Type after entering a command to run it. Dictionaries of configuration used in the creation and alteration of tables are Ruby Hashes. They look like this:{'key1' => 'value1', 'key2' => 'value2', ...}and are opened and closed with curley-braces. Key/values are delimited by the '=>' character combination. Usually keys are predefined constants such as NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type 'Object.constants' to see a (messy) list of all constants in the environment.If you are using binary keys or values and need to enter them in the shell, use double-quote'd hexadecimal representation. For example:hbase> get 't1', "key\x03\x3f\xcd"hbase> get 't1', "key\003\023\011"hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added. For more on the HBase Shell, see http://hbase.apache.org/docs/current/book.html2.5 HBase簡(jiǎn)單命令操作
#創(chuàng)建一個(gè)新表student hbase(main):003:0> create 'student','info' 0 row(s) in 1.2680 seconds#查看所有的表 hbase(main):004:0> list TABLE student 1 row(s) in 0.0330 seconds#查看student的表結(jié)構(gòu) hbase(main):005:0> describe 'student' DESCRIPTION ENABLED'student', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', trueBLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS=> '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',BLOCKCACHE => 'true'} 1 row(s) in 0.1100 seconds#同student表中插入一條數(shù)據(jù) hbase(main):007:0> put 'student','mary','info:age','19' 0 row(s) in 0.0490 seconds#從student表中取出mary的數(shù)據(jù) hbase(main):008:0> get 'student','mary' COLUMN CELLinfo:age timestamp=1396366643298, value=19 1 row(s) in 0.0190 seconds#讓student表失效 hbase(main):009:0> disable 'student' 0 row(s) in 1.2400 seconds#列出所有表 hbase(main):010:0> list TABLE student 1 row(s) in 0.0310 seconds#刪除student表 hbase(main):013:0> drop 'student' 0 row(s) in 1.1100 seconds#列出所有表 hbase(main):014:0> list TABLE 0 row(s) in 0.0400 seconds3 Thrift安裝
安裝完成HBase后，我們還需要安裝Thrift，因?yàn)槠渌Z(yǔ)言調(diào)用HBase的時(shí)候，是通過(guò)Thrift連接的。
Thrift是需要本地編譯的，官方?jīng)]有提供二進(jìn)制安裝包，首先下載thrift-0.9.1，Thrift下載頁(yè)：http://thrift.apache.org/download
3.1 下載thrift
下載Thrift有兩種方式，直接下載源代碼發(fā)行包，或者通過(guò)git下載源代碼，請(qǐng)選擇其中一種方式下載。
3.1.1 直接下載源代碼發(fā)行包 thrift-0.9.1.tar.gz
~ wget http://apache.fayea.com/apache-mirror/thrift/0.9.1/thrift-0.9.1.tar.gz ~ tar xvf thrift-0.9.1.tar.gz ~ mv thrift-0.9.1/ /home/conan/hadoop/ ~ cd /home/conan/hadoop/注：后文中的各種錯(cuò)誤，都是這個(gè)包引起的
3.1.2 通過(guò)git下載源代碼
~ git clone https://git-wip-us.apache.org/repos/asf/thrift.git thrift-git ~ mv thrift-git/ /home/conan/hadoop/ ~ cd /home/conan/hadoop/為了避免各種出錯(cuò)，建議使用git下載源代碼安裝
3.2 通過(guò)thrift-0.9.1.tar.gz 發(fā)行包安裝Thrift
Thrift是需要本地編譯的，在Thrift解壓目錄輸入./configure，會(huì)列Thrift在當(dāng)前機(jī)器所支持的語(yǔ)言環(huán)境。
3.2.1 安裝Thrift的依賴(lài)包
sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev如果只是為了連接rhbase，默認(rèn)配置就可以了。如果除了希望支持rhbase訪問(wèn)，還支持PHP,Python,C++等語(yǔ)言的訪問(wèn)，就需要在系統(tǒng)中，裝一些額外的類(lèi)庫(kù)。大家可以根據(jù)自己的要求，安裝對(duì)應(yīng)的軟件包并設(shè)置Thrift的編譯參數(shù)。
生成配置腳本
~ ./configure//省略部分日志輸出thrift 0.9.1Building C++ Library ......... : yes Building C (GLib) Library .... : no Building Java Library ........ : yes Building C# Library .......... : no Building Python Library ...... : yes Building Ruby Library ........ : no Building Haskell Library ..... : no Building Perl Library ........ : no Building PHP Library ......... : no Building Erlang Library ...... : no Building Go Library .......... : no Building D Library ........... : noC++ Library:Build TZlibTransport ...... : yesBuild TNonblockingServer .. : yesBuild TQTcpServer (Qt) .... : noJava Library:Using javac ............... : javacUsing java ................ : javaUsing ant ................. : /home/conan/toolkit/ant184/bin/antPython Library:Using Python .............. : /usr/bin/pythonIf something is missing that you think should be present, please skim the output of configure to find the missing component. Details are present in config.log.我本機(jī)的已支持C++, Java與Thrift的通信。
3.2.2 增加Python語(yǔ)言的通信
雖然Python已被顯示支持與Thrift但在后面編譯過(guò)程中，還是缺少一些Python的庫(kù)，我們需要再增加Python的依賴(lài)庫(kù)
安裝Python的依賴(lài)包
sudo apt-get install python-all python-all-dev python-all-dbg3.2.3 增加PHP語(yǔ)言的通信
安裝PHP的依賴(lài)包
sudo apt-get install php5-dev php5-cli phpunit生成配置腳本
~ ./configure --enable-thrift_protocol//省略部分日志輸出thrift 0.9.1Building C++ Library ......... : yes Building C (GLib) Library .... : no Building Java Library ........ : yes Building C# Library .......... : no Building Python Library ...... : yes Building Ruby Library ........ : no Building Haskell Library ..... : no Building Perl Library ........ : no Building PHP Library ......... : yes Building Erlang Library ...... : no Building Go Library .......... : no Building D Library ........... : noC++ Library:Build TZlibTransport ...... : yesBuild TNonblockingServer .. : yesBuild TQTcpServer (Qt) .... : noJava Library:Using javac ............... : javacUsing java ................ : javaUsing ant ................. : /home/conan/toolkit/ant184/bin/antPython Library:Using Python .............. : /usr/bin/pythonPHP Library:Using php-config .......... : /usr/bin/php-configIf something is missing that you think should be present, please skim the output of configure to find the missing component. Details are present in config.log.我們看到Thrift的配置中，增加了對(duì)PHP語(yǔ)言的支持。
3.2.4 編譯和安裝
# 編譯Thrift ~ make//省略部分日志make[5]: 正在進(jìn)入目錄 `/home/conan/hadoop/thrift-0.9.1/lib/php/src/ext/thrift_protocol' make[5]: *** 沒(méi)有指明目標(biāo)并且找不到 makefile。 停止。 make[5]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/lib/php/src/ext/thrift_protocol' make[4]: *** [src/ext/thrift_protocol/modules/thrift_protocol.so] 錯(cuò)誤 2 make[4]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/lib/php' make[3]: *** [all-recursive] 錯(cuò)誤 1 make[3]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/lib/php' make[2]: *** [all-recursive] 錯(cuò)誤 1 make[2]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/lib' make[1]: *** [all-recursive] 錯(cuò)誤 1 make[1]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1' make: *** [all] 錯(cuò)誤 2在make生成過(guò)程，出現(xiàn)PHP的編譯錯(cuò)誤。從Thrift的錯(cuò)誤列表中，我們可以找到錯(cuò)誤描述( https://issues.apache.org/jira/browse/THRIFT-2265 )，這是由于Thrift-0.9.1發(fā)行包，打包時(shí)缺少了PHP擴(kuò)展文件造成的錯(cuò)誤，并在Thrift-0.9.2版本中修復(fù)。
所以，我們?nèi)绻€想繼續(xù)使用Thrift-0.9.1版本，則不能支持PHP語(yǔ)言。
# 生成配置信息，不包括PHP模塊 ~ ./configure --without-php_extension# 編譯Thrift ~ make編譯過(guò)程中，又出現(xiàn)了C++編譯錯(cuò)誤。
Makefile:832: 警告：覆蓋關(guān)于目標(biāo)“gen-cpp/ThriftTest.cpp”的命令 Makefile:829: 警告：忽略關(guān)于目標(biāo)“gen-cpp/ThriftTest.cpp”的舊命令 /bin/bash ../../libtool --tag=CXX --mode=link g++ -Wall -g -O2 -L/usr/lib -o libtestgencpp.la ThriftTest_constants.lo ThriftTest_types.lo ../../lib/cpp/libthrift.la -lssl -lcrypto -lrt -lpthread libtool: link: ar cru .libs/libtestgencpp.a .libs/ThriftTest_constants.o .libs/ThriftTest_types.o ar: .libs/ThriftTest_constants.o: No such file or directory make[3]: *** [libtestgencpp.la] 錯(cuò)誤 1 make[3]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/test/cpp' make[2]: *** [all-recursive] 錯(cuò)誤 1 make[2]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1/test' make[1]: *** [all-recursive] 錯(cuò)誤 1 make[1]:正在離開(kāi)目錄 `/home/conan/hadoop/thrift-0.9.1' make: *** [all] 錯(cuò)誤 2對(duì)于上面的2個(gè)編譯錯(cuò)誤，我決定換成git源代碼的版本重新操作。
3.2 通過(guò)git源代碼安裝Thrift
運(yùn)行安裝命令
# 進(jìn)行thrift-git目錄 ~ cd /home/conan/hadoop/thrift-git# 復(fù)制0.9.1標(biāo)簽到新分支thrift-0.9.1 ~ git checkout -b thrift-0.9.1 0.9.1# 產(chǎn)生配置腳本 ~ ./bootstrap.sh# 生成配置信息 ~ ./configure# 編譯Thrift ~ make# 安裝Thrift ~ sudo make install走了許多彎路，終于使用git源代碼版本安裝好了Thrift。
查看thrift版本
~ thrift -version Thrift version 0.9.1接下來(lái)，我們啟動(dòng)HBase的Thrift Server服務(wù)
# 啟動(dòng)HBase的Thrift服務(wù) ~ /home/conan/hadoop/hbase-0.94.18/bin/hbase-daemon.sh start thrift starting thrift, logging to /home/conan/hadoop/hbase-0.94.18/bin/../logs/hbase-conan-thrift-master.out# 檢查系統(tǒng)進(jìn)程 ~ jps 13838 TaskTracker 13541 JobTracker 15946 HMaster 32120 Jps 12851 NameNode 13450 SecondaryNameNode 13133 DataNode 32001 ThriftServer 15817 HQuorumPeer 16283 HRegionServer我們看到ThriftServer已被啟動(dòng)，后面我們就可以使用多種語(yǔ)言，通過(guò)Thrift來(lái)訪問(wèn)HBase了，這樣就完成了HBase的安裝。
轉(zhuǎn)載請(qǐng)注明出處：
http://blog.fens.me/linux-hbase-install/
總結(jié)
以上是生活随笔為你收集整理的在Ubuntu中安装HBase的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。
- 上一篇： 用Maven构建Hadoop项目
- 下一篇： 用Maven构建Mahout项目