Uncategorized

Command to import data by specified delimiters sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table orders –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –fields-terminated-by ‘|’ –lines-terminated-by ‘:’ Command to delete directory if it exists in local system rm -rf orders Command to get files from hdfs to local system hadoop fs -get /user/training/sqoop_import/retail_db/orders . Command to view the …

192.Sqoop Import – Specifying Delimiters – Import NYSE Data with \t as delimiter Read More »

Commands to validate files in directory hadoop fs -ls /user/training/sqoop_import/retail_db/order_items hadoop fs -tail /user/training/sqoop_import/retail_db/order_items/part-m-00003 Command to refer to arguments available on import sqoop help import

Command to reset number of mappers to one mapper while importing table sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –autoreset-to-one-mapper Command to reset number of mappers to one mapper while importing table with no primary key sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items_nopk –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir …

190.Sqoop Import – Autoreset to One Mapper Read More »

Command to get list of tables in retail_db database sqoop list-tables –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity Command to get schema information on table order_items_nopk sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –query “DESCRIBE order_items_nopk” Command to import data from table with no primary key sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table …

189.Sqoop Import – Dealing with Tables with out Primary Key Read More »

Command to change the default property during runtime to deal with non-numeric field sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true –connect jdbc:mysql://ms.itversity.com:3306/nyse –username nyse_user –password itversity –table stock_eod –warehouse-dir /user/training/sqoop_import/nyse –delete-target-dir Command to validate imported data hadoop fs -ls /user/training/sqoop_import/nyse/stock_eod hadoop fs -tail /user/training/sqoop_import/nyse/stock_eod/part-m-00003

Command to get list of tables in nyse database sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/nyse –username nyse_user –password itversity -e “SHOW tables” Command to get information on a table sqoop eval –connect jdbc:mysql –username nyse_user –password itversity -e “DESCRIBE stock_eod”