文章目錄

• • • • • 一、Oracle同步Mysql
        • • 1. 構建json
          • 2. 執行數據同步
          • 3. 查看同步數據
          • 4. 同步數據正確性和準確性
          • 5. 同步日志分析

一、Oracle同步Mysql

1. 構建json

vim oracle2mysql.json {"job": {"setting": {"speed": {"channel": 3},"errorLimit": {"record": 0,"percentage": 0.02}},"content": [{"reader": {"name": "oraclereader","parameter": {"column": ["IDNO","COL1","COL2","COL3","DT","COL5","COL6","COL7","COL8","COL9","COL10"],"connection": [{"jdbcUrl": ["jdbc:oracle:thin:@192.xxx.xxx.xxx:1521:orcl"],"table": ["TEST.OTBS1"]}],"username": "username","password": "password"}},"writer": {"name": "mysqlwriter","parameter": {"column": ["IDNO","COL1","COL2","COL3","DT","COL5","COL6","COL7","COL8","COL9","COL10"],"connection": [{"jdbcUrl": "jdbc:mysql://127.0.0.1:3306/datax?autoReconnect=true&useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=CONVERT_TO_NULL&useSSL=false&serverTimezone=CTT&nullCatalogMeansCurrent=true","table": ["otbs1"]}],"username": "root","password": "123456"}}}]} }

2. 執行數據同步

python /app/datax/bin/datax.py /app/datax/job/oracle2mysql.json 2021-06-23 12:28:12.390 [job-0] INFO StandAloneJobContainerCommunicator - Total 1048576 records, 69143488 bytes | Speed 1.65MB/s, 26214 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 26.038s | All Task WaitReaderTime 8.483s | Percentage 100.00% 2021-06-23 12:28:12.402 [job-0] INFO JobContainer - 任務啟動時刻 : 2021-06-23 12:27:31 任務結束時刻 : 2021-06-23 12:28:12 任務總計耗時 : 41s 任務平均流量 : 1.65MB/s 記錄寫入速度 : 26214rec/s 讀出記錄總數 : 1048576 讀寫失敗總數 : 0

3. 查看同步數據

登陸mysql

4. 同步數據正確性和準確性

數據量的準確性和數據的正確性


Oracle同步數據量：1048576
Mysql插入數據量：1048576

DT同步日期沒問題
COL5數據也沒有都是精度

5. 同步日志分析

2021-06-23 12:27:32.228 [job-0] INFO JobContainer - jobContainer starts to do prepare ... 2021-06-23 12:27:32.229 [job-0] INFO JobContainer - DataX Reader.Job [oraclereader] do prepare work . 2021-06-23 12:27:32.230 [job-0] INFO JobContainer - DataX Writer.Job [mysqlwriter] do prepare work . 2021-06-23 12:27:32.232 [job-0] INFO JobContainer - jobContainer starts to do split ... 2021-06-23 12:27:32.233 [job-0] INFO JobContainer - Job set Channel-Number to 3 channels. 2021-06-23 12:27:32.242 [job-0] INFO JobContainer - DataX Reader.Job [oraclereader] splits to [1] tasks. 2021-06-23 12:27:32.243 [job-0] INFO JobContainer - DataX Writer.Job [mysqlwriter] splits to [1] tasks. 2021-06-23 12:27:32.276 [job-0] INFO JobContainer - jobContainer starts to do schedule ... 2021-06-23 12:27:32.280 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups. 2021-06-23 12:27:32.283 [job-0] INFO JobContainer - Running by standalone Mode. 2021-06-23 12:27:32.300 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks. 2021-06-23 12:27:32.314 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated. 2021-06-23 12:27:32.315 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated. 2021-06-23 12:27:32.348 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started

結果輸出:
可以看到channel設置為3,限速為-1.
雖然channel設置為3,但是只啟動了一個task.
splitPk如果不填寫，將視作用戶不對單表進行切分，OracleReader
使用單通道同步全量數據。

總結

以上是生活随笔為你收集整理的ETL异构数据源Datax_Oracle同步MySQL(全量)_04的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。