Data transformation

Enables pattern-based text substitution where literal string replacement is insufficient.

Data cleaning: normalizing inconsistent phone number formats in a dataset. Log parsing: redacting sensitive information like emails or IPs from log files. Web scraping: extracting and reformatting date strings from raw HTML text.

Text processing, data cleaning, log parsing, web scraping, configuration templating, and any scenario requiring pattern-based string replacement.

re.sub(pattern, repl, string, count=max_replacements, flags=regex_flags)

re.sub(..., ..., ...)

pattern: str (regex), repl: str or callable, string: str (input text), count: int ≥ 0, flags: int (regex flags)

• re
• compile
• raw string r'...'
• flags
• lambda
• match object
• str.replace
• re.subn

• re.subn (returns count)
• str.replace (literal)
• re.sub with lambda repl
• using a compiled regex object via .sub

Forgetting raw string prefix for pattern (cause: misunderstanding escape sequences; consequence: regex fails or throws invalid escape error). Using str.replace for regex patterns (cause: confusing literal and pattern matching; consequence: dynamic patterns fail to match). Forgetting to import re (cause: oversight; consequence: NameError at runtime).

str.replace (literal replace), re.subn (returns (new_string, count)), re.sub with a compiled pattern, re.sub with a lambda or function as repl.

Coming from JavaScript: expecting replacement string to use $1 for group references → Python uses \1 or \g<1> for backreferences. Coming from sed: expecting the replacement to modify the string in-place → re.sub returns a new string and does not mutate the original.

• Regex substitution functions

1. Do not use when replacing a fixed literal substring; str.replace is faster and avoids regex overhead. 2. Compiling the regex first with re.compile is more efficient if the same pattern is used repeatedly in a loop. 3. Overlapping matches are not replaced; only the leftmost match of any overlapping set is handled.

Enables dynamic, pattern‑based text transformation, facilitating data cleaning, templating, and format conversion.

Think ‘substitute’ – replace matches with a replacement.

Remember to import re and consider using raw strings (r'...') for patterns to avoid unintended escape sequences.

Learn re.subn for getting the replacement count, then re.compile for performance, and finally explore advanced regex features like lookarounds and conditional patterns.