Install new type of I/O compression library in the cluster Compression brings the following advantages Reduces space in the cluster to store large files Data transfer speed increases across the network while processing the data Hadoop supports following compression techniques or codes and these codecs are installed by default along, so no separate installation is …
Let us see steps involved in setting up local yum repo server for Cloudera Distribution. Make necessary changes to run in other environments such as AWS. Connect to first server ssh -i ~/.ssh/google_compute_engine itversity@35.196.243.128 Install wget and createrepo Validate by running yum repolist and yum list all commands Create directory /var/www/html Setup repository for both cloudera …
Let us see how we can configure alerts by taking excessive disk fill as an example. As part of the setup process, we have setup Cloudera Management Service. One of the components that are set up as part of Cloudera Management Service is Alert Publisher. Alert Publisher facilitate the support team to receive alerts in a …
Let us see steps involved in setting up httpd service on the first node as user itversity (on AWS user is centos) We are setting up httpd to setup local yum repository server so that we don’t need to download repositories on to all nodes connecting to Cloudera repositories. If we have local yum repository …
Let us see how we can rebalance the cluster. Rebalancing is typically related to HDFS. We have a component called as balancer as part of HDFS. The balancer can run by itself, however, we can kickoff balancer by ourselves. We can use Cloudera Manager Web Interface to rebalance the cluster. Go to Actions -> Rebalance …
Maintain and modify the cluster to support day-to-day operations in the enterprise Rebalance the cluster Set up alerting for excessive disk fill Define and install a rack topology script Install new type of I/O compression library in cluster Revise YARN resource assignment based on user feedback Commission/Decommission a node
