What is MapReduce? Learn how MapReduce works in three minutes.

Python has built-inmap() andreduce() functions.

If you have read Google's famous paper "MapReduce: Simplified Data Processing on Large Clusters", you can roughly understand the concept of map/reduce.

Let’s look at the map first. The map() function receives two parameters, one is a function and the other is an Iterable. map applies the passed function to each element of the sequence in turn and returns the result as a new Iterator.

For example, for example, we have a function f(x)=x2, and we want to apply this function to a list [1, 2, 3, 4, 5, 6, 7, 8, 9]. You can use map() to implement it as follows:

Now, we use Python code to implement it:

>>> def f(x):... return x * x ...>>> r = map(f, [1, 2, 3, 4, 5, 6, 7, 8, 9])>>> list(r) [1, 4, 9, 16, 25, 36, 49, 64, 81]

map()Pass in The first parameter isf, which is the function object itself. Since the resultris aIteratorandIteratoris a lazy sequence, we use thelist()function to let it calculate the entire sequence. and returns alist.

You may think that you don’t need themap()function. You can also calculate the result by writing a loop:

L = []for n in [1, 2, 3, 4, 5, 6, 7, 8, 9]: L.append(f(n)) print(L)

It is indeed possible, but, from the above loop Can you understand the code at a glance "apply f(x) to each element of the list and generate a new list as a result"?

So,map(), as a high-order function, actually abstracts the operation rules. Therefore, we can not only calculate simple f(x)=x2, but also calculate any Complex functions, for example, convert all numbers in this list to strings:

>>> list(map(str, [1, 2, 3, 4, 5, 6, 7, 8, 9])) ['1', '2', '3', '4', '5', '6', '7', '8', '9']

The above is the detailed content of What is MapReduce? Learn how MapReduce works in three minutes.. For more information, please follow other related articles on the PHP Chinese website!