Month: February 2023

This is the reference material for CCA 175 Spark and Hadoop Developer using Scala. Agenda Introduction Curriculum Required Skills Setup Environment HDFS and YARN Data Sets Windows Environment (labs) Introduction CCA Spark and Hadoop Developer is well recognized certification in the industry Conducted by Cloudera – a major Big Data vendor– ITVersity played key role …

CCA 175 Spark and Hadoop Developer using Scala – Introduction Read More »

Let us discuss some of the common performance problems or errors in cluster operation. We might see performance problems/errors in almost all the services. But most common ones are related to applications. We typically run applications using one of these – Map Reduce, Spark, Impala, HBase etc. Map Reduce and Spark are typically run using …

Resolve performance problems/errors in cluster operation Read More »

Let us see how can resolve errors and warnings we see in Cloudera Manager. Errors will be in red and warning will be in orange. We need to make sure there are no errors or warnings in Cloudera Manager. We will only be able to work on errors related to Hosts and Services using Cloudera …

Resolve errors/warnings in Cloudera Manager   Read More »

Benchmarking is the process of stress testing the resources of the cluster to understand the performance of a cluster. Let us see more details about it. Hadoop installation provides a jar file called as hadoop-mapreduce-examples.jar. As part of packages, it will be under path starts with /var/lib and with parcels it will be under path …

Benchmark the cluster (I/O, CPU, network) Read More »

ACLs stands for Access Control Lists. It is primarily to provide finer level access to HDFS. The concept is inherited from Linux. Here we need to understand how to Get or Set ACLs for a file or directory. Click here to revise details about ACLs in HDFS.

Snapshots are primarily used to create backups for the data in HDFS. Click here to revise details about Snapshots in HDFS.