File Paths and Names¶ Let us quickly review about file paths and names. Typically, we will have folder and file name associated while processing a file. A file might be part of a directory structure with multiple levels. All the directories in the path are typically separated by /. The last part of path is …
Folders and Files¶ Let us quickly recap details about folders and files, especially using Linux. You need to be comfortable with the following. Differentiating Files and Folders. Keep in mind that Folders and Directories means the samething. Understanding Absolute or Fully Qualified Path. Understanding Relative Path. Understanding File or Folder permissions. In [1]: # Listing files …
Overview of File IO¶ Let us get an overview of File IO. Files are typically saved in hard drive and we need to perform file i/o to read the data from the files into memory as objects. Here are the commom operations related to files. Read the data from the files into objects (typically collections). …
Basics of File I/O using Python¶ As part of this section or module we will go through some of the basics related to File I/O using Python as programming language. Overview of File I/O Folders and Files File Paths and Names Overview of Retail Data Read Text File into String Write String to Text File …
Limitations of Map Reduce Libraries¶ Here are some of the limitations of using Map Reduce Libraries. We cannot refer attributes with names directly. Functions are scattered and lack consistency. Readability and maintainability are addressed using libraries such as Pandas. Libraries such as PySpark takes care of scalability. {note} We use the approach of loops or …
Revenue per order using itertools¶ Get revenue per order using order_items data set. In [1]: %run 02_preparing_data_sets.ipynb In [2]: order_items[:4] Out[2]: [‘1,1,957,1,299.98,299.98’, ‘2,2,1073,1,199.99,199.99’, ‘3,2,502,5,250.0,50.0’, ‘4,2,403,1,129.99,129.99’] In [3]: order_subtotals = map(lambda oi: (int(oi.split(‘,’)[1]), float(oi.split(‘,’)[4])), order_items) In [4]: list(order_subtotals)[:3] Out[4]: [(1, 299.98), (2, 199.99), (2, 250.0)] In [5]: order_subtotals = map(lambda oi: (int(oi.split(‘,’)[1]), float(oi.split(‘,’)[4])), order_items) order_subtotals_sorted = sorted(order_subtotals) In [6]: order_subtotals_sorted[:3] Out[6]: [(1, …
