[ Apply Schema on the dataframe from files¶ Let us understand how to apply schema while creating the data frame. In many cases, data files might not contain the metadata such as column names, data types, etc. We might get the data metadata in the form of separate files. Also, it is common that metadata …
[ Joining Data Frames¶ Let us understand how to join Data Frames using Pandas. In [1]: %run 06_csv_to_pandas_data_frame.ipynb In [2]: orders Out[2]: order_id order_date order_customer_id order_status 0 1 2013-07-25 00:00:00.0 11599 CLOSED 1 2 2013-07-25 00:00:00.0 256 PENDING_PAYMENT 2 3 2013-07-25 00:00:00.0 12111 COMPLETE 3 4 2013-07-25 00:00:00.0 8827 CLOSED 4 5 2013-07-25 00:00:00.0 11318 COMPLETE … …
[ Writing Data Frames to Files¶ Pandas also provides simple APIs to write the data back to files. Let us write the revenue per order along with order_id to a file. Here are the steps which you need to follow before writing Data Frame to a file. Make sure you have the Data Frame that …
[ Performing Grouped Aggregations¶ Let us understand how to perform grouped or by key aggregations using Pandas. Here are the steps we need to follow: Make sure data is read into Data Frame. Identify the key on which data should be aggregated. If the data has to be aggregated on derived field which is not …
[ Performing Total Aggregations¶ Let us understand how to perform total or global aggregations using Pandas. In [1]: %run 06_csv_to_pandas_data_frame.ipynb Getting number of records in the Data Frame. In [2]: orders.shape Out[2]: (68883, 4) In [3]: orders.shape[0] Out[3]: 68883 In [4]: order_items.shape[0] Out[4]: 172198 Getting number of non np.NaN values in each attribute in a Data Frame In [5]: orders.count() …
[ Projecting and Filtering¶ Let us understand how to project as well as filter data in Data Frames. In [1]: %run 06_csv_to_pandas_data_frame.ipynb In [2]: orders Out[2]: order_id order_date order_customer_id order_status 0 1 2013-07-25 00:00:00.0 11599 CLOSED 1 2 2013-07-25 00:00:00.0 256 PENDING_PAYMENT 2 3 2013-07-25 00:00:00.0 12111 COMPLETE 3 4 2013-07-25 00:00:00.0 8827 CLOSED 4 5 2013-07-25 …
