Regular Expressions

Regular expressions (regex) are a syntax for describing string-matching rules. They are commonly used in form validation, web-scraping data extraction, log analysis, and more. Python’s re module provides complete support.

Metacharacter Quick Reference

Predefined Character Classes

Metacharacter	Matches
`.`	Any character except newline `\n` (includes `\n` when `re.S` is set)
`\d`	Digit `[0-9]`
`\D`	Non-digit
`\w`	Letter, digit, or underscore `[a-zA-Z0-9_]` (Unicode-aware)
`\W`	Non-`\w`
`\s`	Whitespace (space, `\t`, `\n`, `\r`)
`\S`	Non-whitespace
`\n`	Newline
`\t`	Tab
`[abc]`	Character class: matches any one of the listed characters
`[^abc]`	Negated character class: matches any character not listed
`[a-z]`	Range: matches any lowercase letter from a to z

Quantifiers

Quantifier	Meaning
`?`	0 or 1 time
`+`	1 or more times
`*`	0 or more times
`{n}`	Exactly n times
`{n,}`	At least n times
`{n,m}`	Between n and m times

Quantifiers are greedy by default (match as much as possible). Adding ? after a quantifier makes it non-greedy (match as little as possible).

Anchors and Groups

Syntax	Meaning
`^`	Start of string (start of each line in multiline mode)
`$`	End of string (end of each line in multiline mode)
`\b`	Word boundary
`a\|b`	Match a or b
`(abc)`	Capturing group; extractable via `group(n)`
`(?:abc)`	Non-capturing group; does not consume a group number
`(?P<name>abc)`	Named group
`(?P=name)`	Back-reference to a named group’s matched content

Common re Module Functions

import re

text = "2024-06-01, Contact: 13812345678, Alternate: 010-87654321"

# findall: return a list of all matches
phones = re.findall(r"1[3-9]\d{9}", text)
print(phones)   # ['13812345678']

# search: return the first match object (None if no match)
m = re.search(r"(\d{4})-(\d{2})-(\d{2})", text)
if m:
    print(m.group())    # 2024-06-01 (the entire match)
    print(m.group(1))   # 2024 (first group)
    print(m.group(2))   # 06
    print(m.span())     # (0, 10) (start and end positions of the match)

# match: only matches at the beginning of the string (otherwise same as search)
m = re.match(r"\d{4}", text)   # check if the string starts with 4 digits

# fullmatch: the entire string must match
m = re.fullmatch(r"\d{11}", "13812345678")  # full 11-digit number

# finditer: return an iterator; each element is a Match object
for m in re.finditer(r"\d+", text):
    print(m.group(), m.start())

# sub: replace matched content
clean = re.sub(r"\d{11}", "***", text)
print(clean)   # 2024-06-01, Contact: ***, Alternate: 010-87654321

# subn: replace and return the substitution count
result, count = re.subn(r"\d+", "N", text)
print(result, count)

# split: split the string on matches
parts = re.split(r"[,\s]+", "a, b,  c,d")
print(parts)   # ['a', 'b', 'c', 'd']

compile — Pre-compiling Patterns

When the same pattern is used multiple times, pre-compiling improves performance:

import re

# Pre-compile
phone_pat = re.compile(r"1[3-9]\d{9}")
date_pat = re.compile(r"(\d{4})-(\d{2})-(\d{2})")

# Compiled objects have the same API as the re module functions
print(phone_pat.findall("Mobile: 13912345678, Landline: 010-88888888"))
m = date_pat.search("Date: 2024-06-01")
if m:
    year, month, day = m.groups()
    print(year, month, day)  # 2024 06 01

Flags (Modifiers)

import re

# re.I (IGNORECASE): case-insensitive matching
re.findall(r"python", "Python PYTHON python", re.I)
# ['Python', 'PYTHON', 'python']

# re.M (MULTILINE): ^ and $ match the start/end of each line
text = "first\nsecond\nthird"
re.findall(r"^\w+", text, re.M)   # ['first', 'second', 'third']

# re.S (DOTALL): . matches all characters including newline
re.search(r"a.+b", "a\nb", re.S)  # can match across lines

# re.X (VERBOSE): allows comments and whitespace in the pattern (improves readability)
email_pat = re.compile(r"""
    [\w.\-]+    # username part
    @           # @ symbol
    [\w.\-]+    # domain part
    \.\w{2,6}   # top-level domain
""", re.X)

Greedy vs. Non-Greedy

import re

html = "<b>bold</b> and <i>italic</i>"

# Greedy (default): match the longest possible string
re.findall(r"<.+>", html)
# ['<b>bold</b> and <i>italic</i>'] (from the first < to the last >)

# Non-greedy (add ? after the quantifier): match the shortest possible string
re.findall(r"<.+?>", html)
# ['<b>', '</b>', '<i>', '</i>']

Named Groups

Named groups are clearer than numbered groups. Define them with (?P<name>...) and extract with m.group("name"):

import re

log = "2024-06-01 14:30:00 ERROR Database connection failed"

pat = re.compile(
    r"(?P<date>\d{4}-\d{2}-\d{2})"
    r" (?P<time>\d{2}:\d{2}:\d{2})"
    r" (?P<level>\w+)"
    r" (?P<message>.+)"
)

m = pat.match(log)
if m:
    print(m.group("date"))     # 2024-06-01
    print(m.group("level"))    # ERROR
    print(m.group("message"))  # Database connection failed
    print(m.groupdict())       # dict of all named groups

Practical Regex Examples

import re

# Mainland China mobile number
re.fullmatch(r"1[3-9]\d{9}", "13812345678")

# Email address
re.fullmatch(r"[\w.\-]+@[\w.\-]+\.\w{2,6}", "[email protected]")

# Date (YYYY-MM-DD)
re.fullmatch(r"[1-9]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])", "2024-06-01")

# QQ number (5–11 digits, not starting with 0)
re.fullmatch(r"[1-9]\d{4,10}", "123456789")

# IPv4 address
re.fullmatch(r"(?:(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(?:25[0-5]|2[0-4]\d|[01]?\d\d?)", "192.168.1.1")

# Extract HTML tag content (simple cases; use BeautifulSoup for complex ones)
html = "<h1>Title</h1><p>Paragraph</p>"
tags = re.findall(r"<(\w+)>(.*?)</\1>", html)
# [('h1', 'Title'), ('p', 'Paragraph')]

Regular expressions are well-suited for simple string pattern matching, but they are not appropriate for parsing HTML (nested structures) or JSON. Use dedicated parsing libraries for those cases (BeautifulSoup, the json module, etc.).

Last updated on 2026-06-23

Python Programming Paradigms Exception Handling