Using ITVersity labs

At ITVersity we provide a multi-node Spark Cluster. Let us understand some important details with respect to ITVersity labs.

  • Click here to visit our labs.
  • We have 10+ node cluster in which Spark is integrated with Hadoop ecosystem
  • It has a total capacity of 80+ cores and 400+ GB Memory and sEveral terabytes of storage.
  • The cluster is built using Hortonworks and the cluster can be reviewed using Ambari.
  • It has both Spark 1.6.3 and Spark 2.3.0
  • You can launch Spark 2 either by exporting SPARK MAJOR_VERSION to 2 o by using spark2 related commands.
    • spark2-shell (Spark 2 with Scala)
    • pyspark2 (Spark 2 with Python)
    • spark2-submit (Spark 2 submit command to submit Spark 2 jobs)

Share this post