Uncategorized

Command to merge data using update-mode sqoop export –connect jdbc:mysql://ms.itversity.com:3306/retail_export –username retail_user –password itversity –export-dir /apps/hive/warehouse/training_sqoop_retail.db/daily_revenue –table training_daily_revenue –update-key order_date –update-mode allowinsert Command to connect to mysql mysql -u retail_user -h ms.itversity.com -p Command to connect to a database USE retail_export; To validate table SELECT count(1) FROM training_daily_revenue; SELECT * FROM training_daily_revenue WHERE order_date LIKE …

217.Sqoop Export – Using update-mode – allow-insert Read More »

Command to list the jobs already created Sqoop job –-list Command to get details related to the job sqoop job –show import_orders Command to get list of files in hdfs directory hadoop fs -ls /user/training/sqoop_import/retail_db/orders Command to run the job sqoop job –exec import_orders To display information and usage instructions related to job command sqoop …

224.Running Sqoop Job Read More »

To display information and usage instructions related to job command sqoop help job Command to create a job sqoop job –create import_orders — import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password-file file:///home/training/password_file –table orders –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir Command to list the jobs already created sqoop job –list Command to get details related to the job sqoop job …

223.Creating Sqoop Job Read More »

Command to create a password file nano password_file echo -n “itversity” > password_file Command to evaluate data using password file sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password-file file:///home/training/password_file -e “SELECT count(1) FROM orders”

To display information and usage instructions related to job command sqoop help job Command to list the jobs already created sqoop job –list

Command to launch hive connecting to the specified database hive –database training_sqoop_retail Command to get meta-data on table DESCRIBE FORMATTED daily_revenue; Command to export  sqoop export –connect jdbc:mysql://ms.itversity.com:3306/retail_export –username retail_user –password itversity –export-dir /apps/hive/warehouse/training_sqoop_retail.db/daily_revenue –table training_daily_revenue –update-key order_date Command to connect to mysql mysql -u retail_user -h ms.itversity.com -p To validate table SELECT * FROM …

216.Sqoop Export – Using update-mode – update-only (default) Read More »