Question

Problem:

Perform the word counter problem through MapReduce.

Tools: Anaconda/Python 3/Jupyter Notebook

Example word: “One a penny, two a penny, hot cross buns.”

Submitted files: (1) Source code (adding comments); (2) Supporting document for code comments

Answer 1

Code-

def Map(line):

    # varibale to store mapped values

    ret = []

    # remove leading and trailing whitespace

    line = line.strip()

    # split the line into words

    words = line.split()

    # increase counters

    for word in words:

        # add the words with count = 1 to ret list

        ret.append(( f'{word}\t1'))

    return ret

def reduce(lst:list):

    words={}

    for line in lst:

        # remove leading and trailing whitespace

        line = line.strip()

        # parse the input

        word, count = line.split('\t', 1)

        # reduce by adding word to the dictionary

        words[word] = words.get(word, 0) + int(count)

    # print the result to console

    for word, count in words.items():

        print(f'{word} = {count}')

#apply the algorithm by first mapping then reducing

reduce(Map("One a penny, two a penny, hot cross buns."))

OUTPUT-

I hope it helps. For any doubt, feel free to ask in comments, and give upvote if u get the answer.