We can also copy files with in HDFS using commands like cp and mv. hadoop fs -cp to copy files from one HDFS location to another HDFS location hadoop fs -mv to move files from one HDFS location to another HDFS location mv is faster than cp as mv deals with only metadata where as …
We can copy files from local file system and vice versa. We can append data into existing files in HDFS. hadoop fs -copyFromLocal or hadoop fs -put – to copy files from local filesystem and HDFS. Process of copying data is already covered. File will be divided into blocks and will be stored on Datanodes …
Managing files and file permissions – Copying files Local file system and HDFS Read More »
Create a scala program Right click on scala -> new -> scala class Choose scala object and give the object name and hit enter Copy the code which we have seen earlier Pass the arguments as mentioned in the video Run the program Example import scala.io.Source object OrderRevenue { def main(args: Array[String]) = { val …
Development Cycle – Develop Scala application using SBT in IntelliJ Read More »
As part of this topic, we will setup IntelliJ with scala on windows One of the popular IDE for Scala is IntelliJ It is very well integrated with scala and SBT We will see how to develop the application using IntelliJ Setting up IntelliJ with scala Go to the link Scala Download Click on Download IntelliJ …
Now let us get into commands related to managing files in HDFS. It includes deleting files, copying files as well as HDFS File Permissions. Deleting Files from HDFS Let us see how we can delete files from HDFS. As we have already copied data into HDFS, let us start with deleting files using hadoop fs …
As part of this topic, we will explore another way of compiling and running the jar file with arguments through SBT We can make the changes to the code Recompile the jar file Run the jar file How to pass the arguments to the program Example import scala.io.Source object OrderRevenue { def main(args: Array[String]) = …
Development Cycle – Compile changes and run jar with arguments Read More »
