Continuous Applications is new buzzword where enterprises can achieve real time reports with the lowest latency possible. Sarath Varma, Data Engineer at GrubHub is going to share his experience using Spark Structured Streaming to achieve Continuous Applications. A quick overview of Apache Spark on Amazon Elastic Map Reduce (EMR) Overview of Spark Structured Streaming Demo – …

Case Study – Continuous Applications – Spark Structured Streaming Read More »

After the success of Linux Fundamentals live series, we are planning to adapt that model and have a 5+ month live series on Data Engineering which covers 90% of the skills to be successful Data Engineer. Please fill both the surveys so that we can understand where the traffic is coming from as well what you …

Data Engineering Immersion Bootcamp (Premium Live Series and Self-Paced) – Survey Read More »

Introduction – Setup Python, PyCharm and Spark on Windows As part of this blog post we will see detailed instructions about setting up development environment for Spark and Python using PyCharm IDE using Windows. We have used Windows 10 for this demo using 64 bit version on Setup development environment on Windows For each of …

Setup Spark Development Environment – PyCharm and Python Read More »

As part of this blog post we will see detailed instructions about setting up development environment for Spark and Hadoop application development using Windows. We have used Windows 10 for this demo using 64 bit version Setup development environment on Windows For each of the section we will see Why we need to perform the …

Setup Spark Development Environment – IntelliJ and Scala Read More »