[ Preparing Database¶ Let us setup the database along with tables to see how the batch loads are typically done. We have scripts and data set available in our GitHub repository. If you are using our environment the repository is already cloned under /data/retail_db. It have scripts to create tables with primary keys. Those scripts …
[ Recap of Insert¶ Let us recap about INSERT statement as we are going to use it for batch loading. We can either insert one record at a time or multiple records. Inserting one record at a time is typically used to insert records for transaction based use cases. For batch processing we typically try …
[ Database Programming – Batch Operations¶ As part of this section, we will talk about how we can perform batch operations against database tables using Python. Recap of Insert Preparing Database Reading Data from File Processing Data using Pandas Writing Data to Table Batch Loading of Data Best Practices – Batch Loading Read Process Write …
[ Recap – CRUD Operations¶ Let us recap the process of performing CRUD operations Establish connection to the database – Create Connection Object Create Cursor Object using Connection Object – connection.cursor. Build Statement to perform one or more INSERT, UPDATE or DELETE. When we perform DML operations as part of transactions such as INSERT, UPDATE …
[ Querying Data From Table¶ Let us understand how to build queries to get the data from the table. We should leverage database capacity to filter as much data as possible (rather than fetching data into application layer and then filtering). To follow the above pattern one need to have decent skills related to Databases …
[ Deleting Data from Table¶ Let us understand how data is deleted using DELETE statement. Delete statement will have optional WHERE clause where we specify the condition to filter for rows that need to be deleted. As part of the WHERE clause generally we will have indexed column for performance reasons. One need to have …
