你的位置：首页 > IT圈 > sqoop 从 hive 导到mysql遇到的问题

sqoop 从 hive 导到mysql遇到的问题

IT圈 admin 2024-03-31 73浏览 0评论

2024年3月31日发(作者：卿令梅)

环境

hive 版本hive-0.11.0

sqoop 版本 __hadoop-1.0.0

从hive导到mysql

mysql 表：

mysql> desc cps_activation;

+————+————-+——+—–+———+—————-+

+————+————-+——+—–+———+—————-+

| id | int(11) | NO | PRI | NULL | auto_increment |

| day | date | NO | MUL | NULL | |

| cid | varchar(50) | YES | | NULL | |

| pid | varchar(50) | YES | | NULL | |

| activation | int(11) | YES | | NULL | |

+————+————-+——+—–+———+—————-+

6 rows in set (0.01 sec)

hive表

hive> desc active;

id int None

day string None

pkgname string None

cid string None

pid string None

activation int None

测试链接成功

[hadoop@hs11 ~]sqoop list-databases –connect jdbc:mysql://localhost:3306/ –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:26 WARN oopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:26 INFO anager: Preparing to use a MySQL streaming resultset.

information_schema

easyhadoop

mysql

test

[hadoop@hs11 ~]$ sqoop list-databases –connect jdbc:mysql://localhost:3306/test –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:40 WARN oopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:40 INFO anager: Preparing to use a MySQL streaming resultset.

information_schema

easyhadoop

mysql

test

[hadoop@hs11 ~]$ sqoop list-tables –connect jdbc:mysql://localhost:3306/test –username root –password admin

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:42:54 WARN oopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:42:54 INFO anager: Preparing to use a MySQL streaming resultset.

active

[hadoop@hs11 ~]$ sqoop create-hive-table –connect jdbc:mysql://localhost:3306/test –table active –username root –password admin –h

ive-table test

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

13/08/20 16:57:04 WARN oopTool: Setting your password on the command-line is insecure. Consider using -P instead.

13/08/20 16:57:04 INFO oopTool: Using Hive-specific delimiters for output. You can override

13/08/20 16:57:04 INFO oopTool: delimiters with –fields-terminated-by, etc.

13/08/20 16:57:04 WARN oopTool: It seems that you’ve specified at least one of following:

13/08/20 16:57:04 WARN oopTool: –hive-home

13/08/20 16:57:04 WARN oopTool: –hive-overwrite

13/08/20 16:57:04 WARN oopTool: –create-hive-table

13/08/20 16:57:04 WARN oopTool: –hive-table

13/08/20 16:57:04 WARN oopTool: –hive-partition-key

13/08/20 16:57:04 WARN oopTool: –hive-partition-value

13/08/20 16:57:04 WARN oopTool: –map-column-hive

13/08/20 16:57:04 WARN oopTool: Without specifying parameter –hive-import. Please note that

13/08/20 16:57:04 WARN oopTool: those arguments will not be used in this session. Either

13/08/20 16:57:04 WARN oopTool: specify –hive-import to apply them correctly or remove them

13/08/20 16:57:04 WARN oopTool: from command line to remove this warning.

13/08/20 16:57:04 INFO oopTool: Please note that –hive-home, –hive-partition-key,

13/08/20 16:57:04 INFO oopTool: hive-partition-value and –map-column-hive options are

13/08/20 16:57:04 INFO oopTool: are also valid for HCatalog imports and exports

13/08/20 16:57:04 INFO anager: Preparing to use a MySQL streaming resultset.