環(huán)境

　　sqoop-1.4.6

?

Sqoop:將關系數(shù)據(jù)庫（oracle、mysql、postgresql等）數(shù)據(jù)與hadoop數(shù)據(jù)進行轉換的工具。

兩個版本：兩個版本完全不兼容，sqoop1使用最多：
sqoop1：1.4.x
sqoop2：1.99.x

同類產(chǎn)品:DataX?阿里頂級數(shù)據(jù)交換工具

一、架構
Sqoop架構非常簡單，是hadoop生態(tài)系統(tǒng)的架構最簡單的框架。sqoop1由client端直接接入hadoop，任務通過解析生成對應的maprecue執(zhí)行。

二、安裝

?

1、上傳解壓

[root@node101 src]# tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /usr/local [root@node101 local]# mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop-1.4.6

?

2、配置sqoop-env.sh
注釋掉未安裝服務的相關內容，比如HBase、HCatalog、Accumulo

[root@node101 conf]# cd /usr/local/sqoop-1.4.6/conf && mv sqoop-env-template.sh sqoop-env.sh

?

#HADOOP安裝目錄 export HADOOP_COMMON_HOME=/usr/local/hadoop-2.6.5#HADOOP安裝目錄 #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/usr/local/hadoop-2.6.5#HBASE安裝目錄 #set the path to where bin/hbase is available #export HBASE_HOME=#HIVE安裝目錄 #Set the path to where bin/hive is available #export HIVE_HOME=#ZOOKEEPER安裝目錄 #Set the path for where zookeper config dir is #export ZOOCFGDIR=

?

3、這里連接關系數(shù)據(jù)庫是mysql 需要依賴mysql jar包

[root@node101 src]# cp /usr/local/src/mysql-connector-java-5.1.26-bin.jar /usr/local/sqoop-1.4.6/lib/

?

4、配置環(huán)境變量

export SQOOP_HOME=/usr/local/sqoop-1.4.6

?

5、驗證是否安裝成功
sqoop-version 或者 sqoop version

[root@node101 conf]# sqoop version Warning: /usr/local/sqoop-1.4.6/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/local/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/local/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/local/sqoop-1.4.6/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 19/07/01 16:26:50 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 Sqoop 1.4.6 git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25 Compiled by root on Mon Apr 27 14:38:36 CST 2015

?

?

參考：

用戶手冊
Sqoop學習之路

轉載于:https://www.cnblogs.com/cac2020/p/11114482.html

總結

以上是生活随笔為你收集整理的【Sqoop学习之一】Sqoop简介的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內容還不錯，歡迎將生活随笔推薦給好友。