Data Engineering

Database Operations – Overview

Database Operations – Overview¶ Let us get an overview of Database Operations we typically perform on regular basis. They are broadly categorized into the following: DDL – Data Definition Language CREATE/ALTER/DROP Tables CREATE/ALTER/DROP Indexes Add constraints to tables CREATE/ALTER/DROP Views CREATE/ALTER/DROP Sequences DML – Data Manipulation Language Inserting new data into the table Updating existing …

Database Operations – Overview Read More »

Tables as Relations

Tables as Relations¶ Let us understand details about relations and different types of relationships we typically use. In RDBMS – R stands for Relational. In the transactional systems, tables are created using normalization principles. There will be relations or tables created based on relationships among them. Here are the typical relationships among the tables. 1 …

Tables as Relations Read More »

DML or CRUD Operations

DML or CRUD Operations¶ Let us understand how to perform CRUD operations using Postgresql. Normalization Principles Tables as Relations Database Operations – Overview CRUD Operations Creating Table Inserting Data Updating Data Deleting Data Overview of Transactions Exercise – Database Operations Here are the key objectives of this section. What are the different types of Database …

DML or CRUD Operations Read More »

Case Study – Continuous Applications – Spark Structured Streaming

Continuous Applications is new buzzword where enterprises can achieve real time reports with the lowest latency possible. Sarath Varma, Data Engineer at GrubHub is going to share his experience using Spark Structured Streaming to achieve Continuous Applications. A quick overview of Apache Spark on Amazon Elastic Map Reduce (EMR) Overview of Spark Structured Streaming Demo – …

Case Study – Continuous Applications – Spark Structured Streaming Read More »