File input/output

Enables memory-efficient incremental file processing without loading entire contents into memory.

File processing: reading large binary files in fixed-size blocks Data pipelines: processing files too large to fit in memory Network programming: receiving data from sockets in fixed-size chunks

Any Python code that reads from file-like objects (files, sockets, stdin) where chunked processing is needed; common in data pipelines, logging, network receivers.

for chunk in iter(lambda: f.read(size), sentinel):

for ... in iter(lambda: ... .read(...), ...):

chunk: variable name, f: file-like object with read method, size: int > 0, sentinel: str or bytes matching file mode

• open file with 'with' statement
• processing each chunk (e.g.
• hashing
• writing to another file)
• breaking when condition met.

• while True: data = f.read(size)
• if not data: break
• process(data)

Using wrong sentinel (e.g., None instead of empty string) leading to infinite loop; forgetting to open file in appropriate mode (text vs binary); using mutable default arguments incorrectly.

for line in f: (iterates lines); using readlines() to load all lines; using chunksize parameter in pandas read_csv.

Coming from C: may use while loop with explicit EOF check → Python's iter-with-sentinel replaces this concisely Coming from Java: may use InputStream.read(byte[]) in a while loop → iter(lambda: f.read(size), b'') achieves the same with less boilerplate

• file reading idioms
• lazy iteration
• sentinel-based loops

Do not use when line-by-line processing suffices (for line in f is simpler and safer for text). Performance is comparable to a while-read loop; the lambda call overhead is negligible for I/O-bound reads. An empty read result before true EOF (rare with some streams) will prematurely terminate the loop.

Enables memory-efficient processing of large streams; reduces boilerplate code.

Like a conveyor belt that stops automatically when it detects the end marker — no manual shutdown check needed.

Ensure the sentinel matches the return type of read ('' for text, b'' for binary); size must be >0 to avoid infinite loop; an empty read before actual EOF will break the loop prematurely.

Using io.BufferedReader with readinto for zero-copy, or using async iterators with async for chunk in aiter(lambda: await f.read(size), b'')