Exclude parameter in SQOOP to exclude import tables

Exclude parameter is useful to exclude the tables from sqoop export operation. exclude parameter work with import-all-tables parameter. here import-all-tables will import all tables but exclude the tables in exclude list.

STEP 1. Check the no of tables in mysql database.

mysql> show tables;

+—————-+

| Tables_in_test |

+—————-+

| company |

| student |

| transactions |

+—————-+

3 rows in set (0.00 sec)

STEP 2. Use import-all-tables command to import all tables from test database excluding company table.

sqoop import-all-tables –connect jdbc:mysql://127.0.0.1:3306/test –username root –password cloudera –m 1 -warehouse-dir /user/cloudera/excludetables –exclude-tables company

above command will import all the tables from test database but not company. use , if you are looking for multiple tables to exclude.

STEP 3. Crosscheck the data on hdfs level.

[cloudera@quickstart ~]$ hadoop fs -ls /user/cloudera/excludetables

Found 2 items

drwxr-xr-x – cloudera cloudera 0 2019-01-22 23:00 /user/cloudera/excl udetables/student

drwxr-xr-x – cloudera cloudera 0 2019-01-22 23:01 /user/cloudera/excl udetables/transactions

[cloudera@quickstart ~]$ hadoop fs -ls /user/cloudera/excludetables/student

Found 2 items

-rw-r–r– 1 cloudera cloudera 0 2019-01-22 23:00 /user/cloudera/excl udetables/student/_SUCCESS

-rw-r–r– 1 cloudera cloudera 17 2019-01-22 23:00 /user/cloudera/excl udetables/student/part-m-00000

[cloudera@quickstart ~]$

[cloudera@quickstart ~]$

[cloudera@quickstart ~]$

[cloudera@quickstart ~]$ hadoop fs -cat /user/cloudera/excludetables/student/par t-m-00000

1,2018-04-26,200

[cloudera@quickstart ~]$

#sqoop

#hive

#hadoop

#bigdata

#orcldata

#oracledata

Leave a Reply

Your email address will not be published. Required fields are marked *