Command to delete directory if exists in local file system: rm -rf order_items To copy data from hdfs to local file system: hadoop fs -get /user/training/sqoop_import/retail_db/order_items . To extract schema from file in json format using avro-tools: avro-tools getschema part-m-00000.avro To print first few records from avro file in json format: avro-tools tojson part-m-00000.avro | …
To save the imported table as avro file format: sqoop import –connect jdbc:mysql –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –as-avrodatafile To validate imported files: hadoop fs -ls /user/training/sqoop_import/retail_db/order_items
To get the list of files in current directory: ls -ltr To view a file: view part-m-00000 To validate order_items table: sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity -e “DESCRIBE order_items”
Command to import table and customize the number of mappers: sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir –num-mappers 8 To validate: hadoop fs -ls /user/training/sqoop_import/retail_db/order_items
To redirect log messages to other files sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir 1>import.out 2>import.err To view the import.out file: view import.out To view the import.err file: view import.err Command to get status of the job: mapred job -status job_id
Command to import order_items table and overwrite the directory if exists: sqoop import –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity –table order_items –warehouse-dir /user/training/sqoop_import/retail_db –delete-target-dir To validate: sqoop eval –connect jdbc:mysql://ms.itversity.com:3306/retail_db –username retail_user –password itversity -e “DESCRIBE order_items”
