List processing / data chunking

To split a list into equal-sized chunks for batch processing.

Data processing: processing a large list in batches (e.g., sending API requests in chunks) Parallel computing: grouping items for parallel processing Machine learning: preparing data for mini-batches

Data processing scripts Web scraping pipelines Machine learning batching Any scenario requiring fixed-size blocks of a sequence

[seq[i:i+chunk_size] for i in range(0, len(seq), chunk_size)]

[...[...:...] for ... in range(0, len(...), ...)]

seq: iterable to chunk (e.g., list, tuple); chunk_size: positive integer size of each chunk

• itertools.islice numpy.array_split pandas.DataFrame.iloc

• numpy.array_split(seq
• range(0
• len(seq)
• chunk_size)) more-itertools.chunked(seq
• chunk_size) a simple for loop with slicing

using chunk_size <= 0 leading to infinite or empty chunks forgetting that the final chunk may be smaller if length not a multiple of chunk_size confusing this with stepping slicing seq[::chunk_size]

seq[::chunk_size] takes every chunk_size-th element, not blocks a generator function using yield for lazy chunking using textwrap.wrap for strings

Coming from C/Java: may write manual index increment loops instead of the idiomatic list comprehension Coming from functional languages: may prefer recursion or library chunkers

• list comprehension
• slicing
• batching

The final chunk will be smaller than chunk_size when len(seq) is not a multiple of chunk_size; chunk_size must be > 0; works on any sequence supporting slicing and len.

Enables efficient batch processing without external libraries.

Chop the list into chunks with a stepping range.

Creates a list of consecutive slices (chunks) of a list, each of length 'size', using a list comprehension with range stepping.

(seq[i:i+chunk_size] for i in range(0, len(seq), chunk_size)) # lazy generator version