Let us see how we can preview the data in HDFS. If we are dealing with files contain text data (files of text file format), we can preview contents of the files using different commands as -tail, -cat etc. -tail can be used to preview last 1 KB of the file -cat can be used …
Managing files and file permissions – previewing data in HDFS Read More »
We can also copy files with in HDFS using commands like cp and mv. hadoop fs -cp to copy files from one HDFS location to another HDFS location hadoop fs -mv to move files from one HDFS location to another HDFS location mv is faster than cp as mv deals with only metadata where as …
We can copy files from local file system and vice versa. We can append data into existing files in HDFS. hadoop fs -copyFromLocal or hadoop fs -put – to copy files from local filesystem and HDFS. Process of copying data is already covered. File will be divided into blocks and will be stored on Datanodes …
Managing files and file permissions – Copying files Local file system and HDFS Read More »
Create a scala program Right click on scala -> new -> scala class Choose scala object and give the object name and hit enter Copy the code which we have seen earlier Pass the arguments as mentioned in the video Run the program Example import scala.io.Source object OrderRevenue { def main(args: Array[String]) = { val …
Development Cycle – Develop Scala application using SBT in IntelliJ Read More »
As part of this topic, we will setup IntelliJ with scala on windows One of the popular IDE for Scala is IntelliJ It is very well integrated with scala and SBT We will see how to develop the application using IntelliJ Setting up IntelliJ with scala Go to the link Scala Download Click on Download IntelliJ …
Now let us get into commands related to managing files in HDFS. It includes deleting files, copying files as well as HDFS File Permissions. Deleting Files from HDFS Let us see how we can delete files from HDFS. As we have already copied data into HDFS, let us start with deleting files using hadoop fs …
