dgadiraju

Case Study – Continuous Applications – Spark Structured Streaming

Leave a Comment / Big Data, Data Engineering / By

Continuous Applications is new buzzword where enterprises can achieve real time reports with the lowest latency possible. Sarath Varma, Data Engineer at GrubHub is going to share his experience using Spark Structured Streaming to achieve Continuous Applications. A quick overview of Apache Spark on Amazon Elastic Map Reduce (EMR) Overview of Spark Structured Streaming Demo – …

Case Study – Continuous Applications – Spark Structured Streaming Read More »

Setup Spark Development Environment – PyCharm and Python

Leave a Comment / Big Data, Data Engineering / By

Introduction – Setup Python, PyCharm and Spark on Windows As part of this blog post we will see detailed instructions about setting up development environment for Spark and Python using PyCharm IDE using Windows. We have used Windows 10 for this demo using 64 bit version on Setup development environment on Windows For each of …

Setup Spark Development Environment – PyCharm and Python Read More »

Setup Spark Development Environment – IntelliJ and Scala

Leave a Comment / Big Data, Data Engineering / By

As part of this blog post we will see detailed instructions about setting up development environment for Spark and Hadoop application development using Windows. We have used Windows 10 for this demo using 64 bit version Setup development environment on Windows For each of the section we will see Why we need to perform the …

Setup Spark Development Environment – IntelliJ and Scala Read More »