What is MapReduce? Learn how MapReduce works in three minutes.

Python has built-in map() and reduce() functions.

If you have read Google's famous paper "MapReduce: Simplified Data Processing on Large Clusters", you can roughly understand the concept of map/reduce.

Let’s look at the map first. The map() function receives two parameters, one is a function and the other is an Iterable. map applies the passed function to each element of the sequence in turn and returns the result as a new Iterator.

For example, for example, we have a function f(x)=x2, and we want to apply this function to a list [1, 2, 3, 4, 5, 6, 7, 8, 9]. You can use map() to implement it as follows:

Now, we use Python code to implement it:

>>> def f(x):...     return x * x
...>>> r = map(f, [1, 2, 3, 4, 5, 6, 7, 8, 9])>>> list(r)
[1, 4, 9, 16, 25, 36, 49, 64, 81]

map() Pass in The first parameter is f, which is the function object itself. Since the result r is a Iterator and Iterator is a lazy sequence, we use the list() function to let it calculate the entire sequence. and returns a list.

You may think that you don’t need the map() function. You can also calculate the result by writing a loop:

L = []for n in [1, 2, 3, 4, 5, 6, 7, 8, 9]:
    L.append(f(n))
print(L)

It is indeed possible, but, from the above loop Can you understand the code at a glance "apply f(x) to each element of the list and generate a new list as a result"?

So, map(), as a high-order function, actually abstracts the operation rules. Therefore, we can not only calculate simple f(x)=x2, but also calculate any Complex functions, for example, convert all numbers in this list to strings:

>>> list(map(str, [1, 2, 3, 4, 5, 6, 7, 8, 9]))
['1', '2', '3', '4', '5', '6', '7', '8', '9']

The above is the detailed content of What is MapReduce? Learn how MapReduce works in three minutes.. For more information, please follow other related articles on the PHP Chinese website!