How Can I Iterate Through an Iterator in Chunks Using Python?

Iterating an Iterator by Chunks with Python

Iterating over an iterator by chunks of a specific size is a common task in Python. To achieve this, consider using the following approaches:

Using the itertools.grouper() Function:

The itertools.grouper() function provides a versatile method for grouping an iterable into chunks. However, it requires additional handling to accommodate incomplete final chunks, which can be achieved with the incomplete parameter.

from itertools import grouper

it = iter([1, 2, 3, 4, 5, 6, 7])
chunk_size = 3
chunks = list(grouper(it, chunk_size, incomplete='ignore'))
print(chunks)  # [[1, 2, 3], [4, 5, 6], [7]]

Using the itertools.batched() Function (Python 3.12 ):

Introduces in Python 3.12, the itertools.batched() function explicitly handles chunking and preserves the original sequence type.

from itertools import batched

it = [1, 2, 3, 4, 5, 6, 7]
chunk_size = 3
chunks = list(batched(it, chunk_size))
print(chunks)  # [[1, 2, 3], [4, 5, 6], [7]]

Alternative Solution for Sequence Iterators:

For sequences, a less general but convenient solution is to use list slicing with a step size equal to the chunk size.

it = [1, 2, 3, 4, 5, 6, 7]
chunk_size = 3
chunks = [it[i:i + chunk_size] for i in range(0, len(it), chunk_size)]
print(chunks)  # [[1, 2, 3], [4, 5, 6], [7]]

These methods provide efficient ways to iterate over an iterator by chunks, allowing for flexible handling of incomplete final chunks and preservation of the original sequence type when necessary.

The above is the detailed content of How Can I Iterate Through an Iterator in Chunks Using Python?. For more information, please follow other related articles on the PHP Chinese website!