Durga Gadiraju

Snapshots are primarily used to create backups for the data in HDFS. Click here to revise details about Snapshots in HDFS.

In this section, we will setup Cloudera Manager in one of the nodes in the cluster and install Cloudera Manager Agents in all the nodes of the cluster while installing CDH. Also we will see how to configure Cloudera Management Service on the first node. Setup Pre-requisites Install Cloudera Manager Licensing and Installation Options Install …

Introduction Read More »

Let us see how we copy data within a cluster or between clusters. We can use hadoop fs -cp to copy and hadoop fs -mv to move data within a cluster. mv can also be used for renaming the files. We have seen these examples as part of Copying or Moving files within HDFS in …

Efficiently copy data within a cluster/between clusters Read More »

We need to add HTTPFS Role Instance to the cluster in order to use HDFS commands via http. Let us go ahead and setup on our gateway (bigdataserver-1) To add HTTPFS role Click HDFS -> Add Role Instance -> Select “HTTPFS” Select the host to install HTTPFS gateway daemon Click on Install. Once services are restarted, …

Execute file system commands via HTTPFS Read More »

Benchmark the cluster operational metrics, test system configuration for operation and efficiency. Demonstrate ability to find the root cause of a problem, optimize inefficient execution, and resolve resource contention scenarios. Execute file system commands via HTTPFS Efficiently copy data within a cluster/between clusters Create/Restore a snapshot of an HDFS directory Get/Set ACLs for a file or …

Test and Troubleshoot Read More »

As we have successfully enabled encryption let us go ahead and validate it. As part of the validation we will create encrypted zones as well. Encrypted Zone is nothing but directory in HDFS. These directories can be managed by KEY_ADMIN_USER only (itversity in our case) Now we can click on Validate step which will give …

Encryption Zones and Keys Read More »