At ITVersity we provide a multi-node Spark Cluster. Let us understand some important details with respect to ITVersity labs.
- Click here to visit our labs.
- We have 10+ node cluster in which Spark is integrated with Hadoop ecosystem
- It has a total capacity of 80+ cores and 400+ GB Memory and sEveral terabytes of storage.
- The cluster is built using Hortonworks and the cluster can be reviewed using Ambari.
- It has both Spark 1.6.3 and Spark 2.3.0
- You can launch Spark 2 either by exporting SPARK MAJOR_VERSION to 2 o by using spark2 related commands.
- spark2-shell (Spark 2 with Scala)
- pyspark2 (Spark 2 with Python)
- spark2-submit (Spark 2 submit command to submit Spark 2 jobs)