Benchmarking is the process of stress testing the resources of the cluster to understand the performance of a cluster. Let us see more details about it. Hadoop installation provides a jar file called as hadoop-mapreduce-examples.jar. As part of packages, it will be under path starts with /var/lib and with parcels it will be under path …
ACLs stands for Access Control Lists. It is primarily to provide finer level access to HDFS. The concept is inherited from Linux. Here we need to understand how to Get or Set ACLs for a file or directory. Click here to revise details about ACLs in HDFS.
Snapshots are primarily used to create backups for the data in HDFS. Click here to revise details about Snapshots in HDFS.
In this section, we will setup Cloudera Manager in one of the nodes in the cluster and install Cloudera Manager Agents in all the nodes of the cluster while installing CDH. Also we will see how to configure Cloudera Management Service on the first node. Setup Pre-requisites Install Cloudera Manager Licensing and Installation Options Install …
Let us see how we copy data within a cluster or between clusters. We can use hadoop fs -cp to copy and hadoop fs -mv to move data within a cluster. mv can also be used for renaming the files. We have seen these examples as part of Copying or Moving files within HDFS in …
Efficiently copy data within a cluster/between clusters Read More »
We need to add HTTPFS Role Instance to the cluster in order to use HDFS commands via http. Let us go ahead and setup on our gateway (bigdataserver-1) To add HTTPFS role Click HDFS -> Add Role Instance -> Select “HTTPFS” Select the host to install HTTPFS gateway daemon Click on Install. Once services are restarted, …
