Day: March 16, 2023

Command to reset number of mappers to one mapper while importing table sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –autoreset-to-one-mapper Command to reset number of mappers to one mapper while importing table with no primary key sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items_nopk –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir …

190.Sqoop Import – Autoreset to One Mapper Read More »

Command to get list of tables in retail_db database sqoop list-tables –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity Command to get schema information on table order_items_nopk sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –query “DESCRIBE order_items_nopk” Command to import data from table with no primary key sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table …

189.Sqoop Import – Dealing with Tables with out Primary Key Read More »

Command to change the default property during runtime to deal with non-numeric field sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true –connect jdbc:mysql://ms.itversity.com:3306/nyse –username nyse_user –password itversity –table stock_eod –warehouse-dir /user/training/sqoop_import/nyse –delete-target-dir Command to validate imported data hadoop fs -ls /user/training/sqoop_import/nyse/stock_eod hadoop fs -tail /user/training/sqoop_import/nyse/stock_eod/part-m-00003

Command to get list of tables in nyse database sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/nyse –username nyse_user –password itversity -e “SHOW tables” Command to get information on a table sqoop eval –connect jdbc:mysql –username nyse_user –password itversity -e “DESCRIBE stock_eod”

Command to import query results sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –query “SELECT order_date, count(1) order_count FROM orders WHERE \$CONDITIONS GROUP BY order_date” –target-dir /user/training/sqoop_import/retail_db/order_count_by_date –sptit-by order—date –delete-target-dir

Command to import table and divide the data into multiple splits based on specified column. sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –split-by order_item_order_id