Develop myReduce Function¶ Develop a function by name myReduce which takes a collection and a function as arguments. The myReduce function should do the following: Iterate through elements Perform aggregation operation using the argument passed. Argument should have necessary arithmetic logic. Return the aggregated result. In [1]: l = [1, 4, 6, 2, 5] In [2]: l[1:] …
Validate myMap Function¶ Let us validate the function by using some realistic examples. In [1]: %run 04_develop_myMap_function.ipynb In [2]: myMap Out[2]: <function __main__.myMap(c, f)> Use orders and extract order_dates. Also apply set and get only unique dates. In [3]: orders_path = “/data/retail_db/orders/part-00000” orders = open(orders_path). \ read(). \ splitlines() In [4]: orders[:10] Out[4]: [‘1,2013-07-25 00:00:00.0,11599,CLOSED’, ‘2,2013-07-25 00:00:00.0,256,PENDING_PAYMENT’, ‘3,2013-07-25 00:00:00.0,12111,COMPLETE’, …
Develop myMap Function¶ Develop a function by name myMap which takes a collection and a function as arguments. myMap function should do the following: Iterate through elements. Apply the transformation logic using the argument passed. Append the transformed record to the list. Return the collection with all the elements which are transformed based on the …
Validate myFilter Function¶ Use the same examples which were used before as part of Processing Collections using loops. In [1]: %run 02_develop_myFilter_function.ipynb In [2]: myFilter Out[2]: <function __main__.myFilter(c, f)> Read orders data In [3]: orders_path = “/data/retail_db/orders/part-00000” orders = open(orders_path). \ read(). \ splitlines() In [4]: orders[:10] Out[4]: [‘1,2013-07-25 00:00:00.0,11599,CLOSED’, ‘2,2013-07-25 00:00:00.0,256,PENDING_PAYMENT’, ‘3,2013-07-25 00:00:00.0,12111,COMPLETE’, ‘4,2013-07-25 00:00:00.0,8827,CLOSED’, ‘5,2013-07-25 00:00:00.0,11318,COMPLETE’, ‘6,2013-07-25 …
Develop myFilter Function¶ Develop a function by name myFilter which takes a collection and a function as arguments. Function should do the following: Iterate through elements Apply the condition using the argument passed. We might pass named function or lambda function. Return the collection with all the elements satisfying the condition. {note} This is how …
Development of Map Reduce APIs¶ Develop myFilter Validate myFilter Function Develop myMap Validate myMap Function Develop myReduce Validate myReduce Develop myReduceByKey Validate myReduceByKey Exercises
