As part of this section, we will see how to set up Spark components while exploring some of the key concepts of this very important service. There are 2 versions of Spark, we will see how to setup both Spark 1.6 as well as Spark 2.3. However to setup Spark 2.3, we need to use …
It is better to have Linux based environment for setting up Big Data Clusters. It provides us below capabilities. Ability to connect to remote servers using ssh Copy files using scp or rsync Enable proxy such as sshuttle to access web applications running on the servers behind firewall with out opening up ports to the public using ssh authentication. Setup Process …
As part of this section, we will see how to set up Pig and Oozie components and some of the key concepts related to each service. Setup Oozie, Pig, Sqoop and Hue Review Important Properties Schedule an Oozie workflow Run Pig Job Validate Sqoop Overview of Hue Cluster Topology We are setting up the cluster …
Configure Hadoop Ecosystem components – Oozie, Pig, Sqoop and Hue Read More »
As part of this section we will talk about setting up necessary tools to create virtual machines on Google Cloud Platform (GCP) Setup Ubuntu using Windows Subsystem Sign up for GCP Create template for Big Data Server Provision Servers for Big Data Cluster Review Concepts Setting up gcloud Setup ansible on first server Format JBOD …
