Configure HDFS ACLs
ACLs stands for Access Control Lists. It is primarily to provide finer level access to HDFS. Concept is inherited from Linux. We need to understand how to Configure HDFS ACLs using Cloudera Manager.
Secure the Cluster
Enable relevant services and configure the cluster to meet goals defined by security policy; demonstrate knowledge of basic security practices Configure HDFS ACLs Install and configure Sentry Configure Hue user authorization and authentication Enable/configure log and query redaction Create Encrypted Zones in HDFS
Commission/Decommission a node
Let us see the details with respect to commissioning or decommissioning a node. Decommissioning is nothing but stopping roles on the cluster. As discussed earlier, we have 3 types of Nodes. Behavior is a bit different for each type of Node. Gateway Master Maintenance – Decommission -> Complete Maintenance -> Recommission. Removing the hosts – …
YARN Resource Assignment
Revise YARN resource assignment based on user feedback. The processing capacity of the cluster is determined by the capacity given to YARN. In our case, both Map Reduce as well as Spark is running using YARN. Node Manager Capacity Container Capacity Map Reduce – Map Task and Reduce Task Capacity Spark – Executores Capacity As …
Add I/O Compression Library
Install new type of I/O compression library in the cluster Compression brings the following advantages Reduces space in the cluster to store large files Data transfer speed increases across the network while processing the data Hadoop supports following compression techniques or codes and these codecs are installed by default along, so no separate installation is …
Cloudera – Local yum repository
Let us see steps involved in setting up local yum repo server for Cloudera Distribution. Make necessary changes to run in other environments such as AWS. Connect to first server ssh -i ~/.ssh/google_compute_engine itversity@35.196.243.128 Install wget and createrepo Validate by running yum repolist and yum list all commands Create directory /var/www/html Setup repository for both cloudera …