Meaning
This pattern removes duplicate elements from a list while keeping the order of their first appearance. It works by converting the list to a dictionary (which cannot have duplicate keys) and then back to a list. Available in Python 3.7+ where dict preserves insertion order.
Primary Function
Data deduplication
Communicative Purpose
Remove duplicate items from a sequence while maintaining order.
Pattern
list(dict.fromkeys(input_list))
Core Structure
list(dict.fromkeys(...))
Função primária
Data deduplication
Propósito comunicativo
Remove duplicate items from a sequence while maintaining order.
Situações de gatilho
Data cleaning: removing duplicate entries from a list of user IDs; ETL pipeline: ensuring ordered unique records before aggregation
Contextos
General Python scripting, data processing pipelines, ETL, competitive programming, any code needing ordered unique collections.
Padrão
list(dict.fromkeys(input_list))
Estrutura central
list(dict.fromkeys(...))
Slots de substituição
input_list: iterable (e.g., list of hashable items)
Colocados típicos
- set() for unordered deduplication
- sorted() for ordered unique sorted list
- itertools.groupby for consecutive duplicates
Substituições comuns
- Using a loop with a seen set: [x for x in input_list if not (x in seen or seen.add(x))]
- using pandas.unique() for Series
Erros comuns
Assuming it works for unhashable items (e.g., lists, dicts) causing TypeError; forgetting that dict.fromkeys preserves order only in Python 3.7+; using it on large lists causing memory overhead.
Similar / contraste
set(input_list) – removes duplicates but loses order; collections.OrderedDict.fromkeys(input_list) – explicit order-preserving dedup for older Python versions.
Interferências
Coming from Ruby: you might expect set() to keep order → in Python, set does not guarantee order before 3.7 (implementation detail) but not reliable.
Família do chunk
- deduplication
- ordered set
- unique filter
Nuance
Only works with hashable elements; performance O(n) but creates an intermediate dict; not suitable for very large lists if memory is a concern; preserves order of first occurrence only.
Efeito pragmático
Provides a concise, readable way to deduplicate while preserving order, reducing boilerplate code.
Dica de memória
Think 'dict.fromkeys makes keys unique, then list() brings them back'.
Nota
Requires hashable elements and Python 3.7+ for guaranteed insertion‑order preservation; creates an intermediate dict, so may use extra memory for large lists.
Upgrade path
from collections import OrderedDict list(OrderedDict.fromkeys(input_list)) # works in older Python # or import more_itertools list(more_itertools.unique_everseen(input_list))
Log in to save chunks.