`RegexStore`: Readable, Optimized Regular Expressions#

class my.regex.RegexStore.RegexStore(*, definitions: dict[str, str]={}, patterns: dict[str, ~typing.Annotated[~_regex.Pattern, ~pydantic.types.GetPydanticSchema(get_pydantic_core_schema=~my.utils.SyntaxUtils.SyntaxUtils.pyd_schemify.<locals>.<lambda>, get_pydantic_json_schema=None)]] = {}, parsers: dict[str, ~my.regex.RegexStore.RegexParser]={}, options: Options = <factory>)#

A powerful regex pattern management system with composition and parsing capabilities.

RegexStore provides a comprehensive framework for defining, composing, and applying complex regex patterns. To do this, it also contains code for analyzing existing patterns, breaking them down into their component parts.

Features:

Hierarchical pattern composition from simple building blocks.
Recursive pattern references via named groups.
Ergonomic group management and subroutine invocation.
Automatic parsing of match results into a more ergonomic form (see MatchData).
Pattern optimization applied by default.
Construction of “Router trees” for efficient matching of long patterns to long texts.
A REGEX_TIMEOUT deadline (10s) guarding every public engine call, including substitutions, so runaway backtracking cannot hang unattended processing.

The DSL used to specify patterns can combine a variety of input types into one, including:

String literals and pre-compiled patterns.
Tuples for group creation with custom separators.
Lists for sequential composition of groups.

class my.regex.RegexStore.RegexStore.Options(*, init_formatter: Callable[[...], str] | None = None, formatter: Callable[[...], str] | None = None, separator: str = ' *', force_named_groups: bool = False, force_reinvocations: bool = True, lazy_load: bool = True, autostrip_spaces: bool = True, autostrip_brackets: bool = False, autostrip_commas: bool = False)#

Configuration options for RegexStore behavior – must be set at initialization time.

model_config = {}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

init_formatter: Callable[..., str] | None#: Function to format *only* definitions provided at initialization time.

formatter: Callable[..., str] | None#: Function to format definitions always, whenever they’re created.

separator: str#: The default expression inserted between elements of a list when processing definitions.

force_named_groups: bool#: Whether to convert (...) groups to (?:...), allowing for cleaner definitions.

force_reinvocations: bool#: Whether to convert (?P=name) group substitutions to (?P>name) group invocations, used primarily for text editors that don’t recognize the latter syntax. To write substitutions while this is enabled, use the alternate g<name> syntax.

lazy_load: bool#: Whether to delay compilation of patterns until first use. Even very large stores compile in milliseconds, but this is enabled by default so that invalid definitions don’t raise exceptions before logging and/or error handling is setup.

autostrip_spaces: bool#: Whether to automatically strip spaces from matched group values (` text ` -> text).

autostrip_brackets: bool#: Whether to automatically strip brackets from matched group values (e.g. ([func(params)]){} -> func(params)).

autostrip_commas: bool#: Whether to automatically strip commas from matched group values (e.g. a,b, -> a,b).

`I` Initial Methods#

classmethod RegexStore.new(options: dict[str, Any] | Options | None = None, imports: list[tuple[Self, Iterable[str]]] | None = None, **definitions: RegexDef | Any) → Self#

Create a new store, likely specifying almost all your patterns upfront.

Parameters:

options – A dictionary of store-level options to apply as member variables.
imports – References to patterns contained in existing stores to be included in this one.
**definitions – A dictionary of named regular expressions – see RegexDef.

Returns:

A new RegexStore instance with the given patterns compiled into execution-ready objects.

Examples

Define patterns upfront, invoking earlier definitions as subroutines:

>>> from my import RegexStore
>>> store = RegexStore.new(
...     year=r'\d{4}',
...     month=r'0?[1-9]|1[0-2]',
...     date=r'(?P<y>(?P>year))-(?P<m>(?P>month))',
... )
>>> store.fullmatch('date', '2026-07').flat
{'year': '2026', 'month': '07', 'y': '2026', 'm': '07'}

Compose groups, alternations, and flags with the tuple DSL:

>>> store2 = RegexStore.new(
...     options=dict(separator=''),
...     feeling=('|:i', r'\b', ['happy', 'sad'], r'\b'),
... )
>>> store2.get_def('feeling')
'(?i:\\b(?:happy|sad)\\b)'

RegexStore.initial_load(imports: list[tuple[Self, Iterable[str]]] | None, definitions: dict[str, RegexDef | Any]) → None#: Initial loading of patterns into the store, including imports and formatting.

property RegexStore.load: bool#: Load all lazy definitions into the store now.

property RegexStore.routers: dict[str, list[str]]#

Router-tree names mapped to their ordered category lists.

Triggers a full lazy load first (see .load), so a store whose define_router_tree call was itself queued behind the lazy-load queue still exposes a fully-populated dict rather than the empty one it would start with.

`II` Primary Methods#

RegexStore.parse(match: Match[str] | None, pattern_name: str = '') → MatchData#

Parse a match object, applying registered parsers and cleaning results.

Reads the match, applies any parsers registered for captured groups, removes hidden fields (those starting with ‘_’), and returns a clean MatchData object.

Parameters:

match – Match object to parse, or None for empty MatchData.
pattern_name – Optional name of the pattern that produced this match.

Returns:

MatchData object with parsed captures and optional match reference.

RegexStore.compose(data: RegexVal, sep: str | None = None) → str#

RegexStore.compose(data: str, sep: str | None = None) → str

RegexStore.compose(data: Pattern, sep: str | None = None) → str

RegexStore.compose(data: list, sep: str | None = None) → str

RegexStore.compose(data: tuple, sep: str | None = None) → str

RegexStore.compose(data: dict, sep: str | None = None) → str

Recursively transform a DSL-compliant definition into a valid regular expression.

Parameters:

data – The regex value to compose.
sep – Optional separator string for lists (defaults to store’s separator).

Returns:

The composed expression.

Examples

Tuples become groups, lists become sequences, and the two nest freely:

>>> from my import RegexStore
>>> store = RegexStore.new(options=dict(separator=''))
>>> store.compose(('|:', ['cat', 'dog']))
'(?:cat|dog)'
>>> store.compose(['a', ('|:?', ['b', 'c'])])
'a(?:b|c)?'

RegexStore.clean(name: str, text: Buffer) → set[str]#

Clean and validate a regex pattern, identifying all group dependencies.

Normalizes group invocation syntax and validates that local group names don’t conflict with predefined subroutine names.

Parameters:

name – Name of the pattern being cleaned.
text – Buffer containing the pattern text.

Returns:

Set of all group names invoked by this pattern.

Raises:

AssertionError – If local group names conflict with predefined groups.
ValueError – If a dependency relies on compile-time flags that cannot travel through the rendered definition block.

RegexStore.define(name: str, val: RegexVal, parser: RegexParser | None = None, flags: RegexFlag | None = None) → None#

Define a new named regex pattern in the store.

Composes the pattern from the given value, cleans and validates it, compiles it with any necessary definitions, and stores it along with an optional parser.

Parameters:

name – Unique name for this pattern.
val – Regex value to compose (string, list, tuple, or Pattern).
parser – Optional function to parse match results.
flags – Compile-time flags for a raw DSL value. Leave this unset for a compiled pattern, whose exact engine flags are preserved automatically.

Raises:

AssertionError – If name is already defined.
ValueError – If both a compiled pattern and explicit flags are supplied, or a composed dependency relies on out-of-band flags. Use inline scoped flags such as (?i:...) when flags must travel with a reusable DSL definition.
Exception – If pattern composition or compilation fails.

Examples

Item assignment routes here, so direct calls are only needed for parsers or flags:

>>> import regex
>>> from my import RegexStore
>>> store = RegexStore.new()
>>> store.define('word', ('|:', ['cat', 'dog']))
>>> store.get_def('word')
'(?:cat|dog)'
>>> store['casefolded'] = regex.compile(r'hello', regex.I)
>>> bool(store.fullmatch('casefolded', 'HELLO'))
True

RegexStore.autostrip(values: list[str] | str) → list[str]#

Strip configured characters from values and fix broken brackets.

Parameters:: values – Single string or list of strings to strip.
Returns:: List of stripped strings with bracket balancing corrected.

Examples

Strip configured characters, dropping values that strip to nothing:

>>> from my import RegexStore
>>> RegexStore.new().autostrip([' cat ', '', 'dog'])
['cat', 'dog']

RegexStore.parse_invocations(text: str) → set[str]#

Find all group invocations in text and transitively expand dependencies.

Parameters:: text – Regex pattern text to analyze.
Returns:: Set of all group names invoked directly or indirectly.

Examples

One invocation pulls in its transitive dependencies:

>>> from my import RegexStore
>>> store = RegexStore.new(
...     options=dict(lazy_load=False),
...     year=r'\d{4}',
...     month=r'0?[1-9]|1[0-2]',
...     date=r'(?P<y>(?P>year))-(?P<m>(?P>month))',
... )
>>> sorted(store.parse_invocations(r'(?P>date)'))
['date', 'month', 'year']

`III` Overrides#

RegexStore.get(name: str, default: Pattern[str] | None = None) → Pattern[str] | None#: Get a compiled pattern by name, or return a default if not found.

RegexStore.get_def(name: str, default: str | None = None) → str | None#: Get a raw definition by name, or return a default if not found.

RegexStore.keys() → list[str]#: Get a list of all defined pattern names in the store.

RegexStore.values() → list[Pattern[str]]#: Get a list of all compiled patterns in the store.

RegexStore.items() → list[tuple[str, Pattern[str]]]#: Get a list of all (name, pattern) pairs in the store.

`IV` Top-Level Matching Methods#

RegexStore.match(names: str | Iterable[str], text: str | Buffer) → MatchData#

Match one of the named patterns against text from the beginning.

Parameters:

names – Pattern name or list of pattern names to try.
text – Text to match against.

Returns:

MatchData from first successful match, or empty MatchData if none match.

Examples

Matching anchors at the start of the text, unlike search():

>>> from my import RegexStore
>>> store = RegexStore.new(word=r'[a-z]+')
>>> bool(store.match('word', '123 abc'))
False
>>> store.search('word', '123 abc').text
'abc'

RegexStore.fullmatch(names: str | Iterable[str], text: str | Buffer) → MatchData#

Match one of the named patterns against the entire text.

Also available under the shorter alias full.

Parameters:

names – Pattern name or list of pattern names to try.
text – Text to match against.

Returns:

MatchData from first successful fullmatch, or empty MatchData if none match.

Examples

The whole text must match, not just its start:

>>> from my import RegexStore
>>> store = RegexStore.new(word=r'[a-z]+')
>>> store.fullmatch('word', 'abc').text
'abc'
>>> bool(store.fullmatch('word', 'abc!'))
False

RegexStore.search(names: str | Iterable[str], text: str | Buffer) → MatchData#

Search for one of the named patterns anywhere in text.

Parameters:

names – Pattern name or list of pattern names to try.
text – Text to search.

Returns:

MatchData from first successful search, or empty MatchData if none found.

RegexStore.finditer(name: str, text: str | Buffer, **kwargs: Any) → Iterator[MatchData]#

Find all non-overlapping matches of the pattern in text.

Parameters:

name – Pattern name to search for.
text – Text to search (string or Buffer).
**kwargs – Optional arguments passed to Buffer.rgx_iterator().

Yields:

MatchData objects for each match found.

RegexStore.findall(name: str, text: str | Buffer, **kwargs: Any) → list[MatchData]#

Find all non-overlapping matches of the pattern in text.

Parameters:

name – Pattern name to search for.
text – Text to search (string or Buffer).
**kwargs – Optional arguments passed to Buffer.rgx_iterator().

Returns:

List of MatchData objects for all matches found.

Examples

Collect every match in one list (see finditer() for the lazy equivalent):

>>> from my import RegexStore
>>> store = RegexStore.new(word=r'[a-z]+')
>>> [m.text for m in store.findall('word', 'ab cd')]
['ab', 'cd']

RegexStore.fullsplit(name: str, text: str | Buffer, collapse: bool = False) → tuple[list[str], list[str]]#

Split text by matches, returning both delimiters and sections.

Also available under the shorter alias split.

Parameters:

name – Pattern name to split on.
text – Text to split.
collapse – Whether to collapse empty sections into adjacent delimiters.

Returns:

Tuple of (delimiters, sections) where delimiters[0] is always empty and the lists interleave: section[0], delim[1], section[1], delim[2], etc.

Examples

Keep the delimiters alongside the sections they separate:

>>> from my import RegexStore
>>> store = RegexStore.new(comma=r' *, *')
>>> store.fullsplit('comma', 'a, b , c')
(['', ', ', ' , '], ['a', 'b', 'c'])

RegexStore.polymatch(name: str, text: str | Buffer) → MatchData#

Find all matches and merge their captures into a single MatchData.

Unlike findall which returns separate MatchData objects, this merges all captures from all matches into one result, preserving order by start position.

This is the only such method that doesn’t have an analogue in the `re` standard library.

Also available under the shorter alias poly.

Parameters:

name – Pattern name to search for.
text – Text to search (automatically converted to Buffer).

Returns:

Single MatchData with all captures from all matches merged.

Examples

Merge every match’s captures into one MatchData:

>>> from my import RegexStore
>>> store = RegexStore.new(word=r'(?P<w>[a-z]+)')
>>> store.polymatch('word', 'one two three')
MatchData({'w': ['one', 'two', 'three']})

`V` Functional Utilities#

RegexStore.partial(name: str, func: MatchFunction = 'match') → Callable[[str | Buffer], MatchData]#

Create a partially applied matching function for a pattern.

Parameters:

name – Pattern name to use.
func – Matching function name (‘match’, ‘fullmatch’, ‘search’, or ‘polymatch’).

Returns:

Function that takes text and returns MatchData using the specified pattern.

Examples

Freeze a pattern and function into a reusable predicate:

>>> from my import RegexStore
>>> store = RegexStore.new(num=r'\d+')
>>> is_num = store.partial('num', 'fullmatch')
>>> (bool(is_num('123')), bool(is_num('12x')))
(True, False)

RegexStore.apply(name: str, texts: Iterable[str], func: MatchFunction = 'match') → Iterable[MatchData]#

Apply a pattern to multiple texts.

Parameters:

name – Pattern name to use.
texts – Iterable of text strings to match against.
func – Matching function to use (‘match’, ‘fullmatch’, ‘search’, or ‘polymatch’).

Yields:

MatchData objects for each text in order.

Examples

Run the same pattern over a batch of texts:

>>> from my import RegexStore
>>> store = RegexStore.new(num=r'\d+')
>>> [bool(m) for m in store.apply('num', ['1', 'a', '22'])]
[True, False, True]

RegexStore.filter(name: str, texts: Iterable[str], func: MatchFunction = 'match') → Iterable[str]#

Filter texts by whether they match a pattern.

Parameters:

name – Pattern name to test against.
texts – Iterable of text strings to filter.
func – Matching function to use (‘match’, ‘fullmatch’, ‘search’, or ‘polymatch’).

Yields:

Only those texts that successfully match the pattern.

Examples

Keep only the texts the pattern accepts:

>>> from my import RegexStore
>>> store = RegexStore.new(num=r'\d+')
>>> list(store.filter('num', ['1', 'a', '22']))
['1', '22']

`VI` Optimization Functions#

RegexStore.define_router_tree(router: str, items: Mapping[str, RegexVal], **kwargs: str) → None#

Define a router pattern that classifies text into named categories.

Creates two patterns: one matching router (<router>) and one route-tracking router (<router>_router) that captures which category matched. Both use ordinary ordered alternation because atomic tree condensation can change lazy-quantifier fullmatch semantics; use the <|> DSL mark directly when optimization is explicitly desired.

Parameters:

router – Base name for the router patterns.
items – Mapping of category names to their regex patterns.
**kwargs – Optional ‘prefix’/’suffix’ or ‘p0’/’p1’/’s0’/’s1’ for wrapping patterns.

Raises:

AssertionError – If router name is already defined.

Examples

Route texts into named categories, then ask which one matched:

>>> from my import RegexStore
>>> store = RegexStore.new(options=dict(separator=''))
>>> store.define_router_tree('kind', dict(number=r'\d+', word=r'[a-z]+'))
>>> store.routers
{'kind': ['number', 'word']}
>>> (store.route_match('kind', 'hello'), store.route_match('kind', '42'))
('word', 'number')

RegexStore.route_match(router: str, text: str | MatchData) → str#

Determine which category a text matches in a router tree.

Parameters:

router – Name of router pattern to use.
text – Text to classify, or MatchData from previous match.

Returns:

Name of the matching category, or empty string if no match.

Raises:

AssertionError – If router name is not found.

RegexStore.expand_match(router: str, text: str | MatchData) → str#

Match text against a router and expand using the matched category’s format.

Parameters:

router – Name of router pattern to use.
text – Text to match and expand, or MatchData from previous match.

Returns:

Expanded string using the matched category name as format string.

Raises:

AssertionError – If router name is not found or match object is invalid.

Examples

Category names double as format strings for the matched groups:

>>> from my import RegexStore
>>> store = RegexStore.new(options=dict(separator=''))
>>> store.define_router_tree('swap', {'{b} {a}': r'(?P<a>\w+) (?P<b>\w+)'})
>>> store.expand_match('swap', 'hello world')
'world hello'

`VII` Expression Construction#

static RegexStore.format_url(target: str) → str#

Removes archive.org prefixes, URL fragments, and trailing punctuation from URLs.

Parameters:: target – The URL string to format.
Returns:: The cleaned URL string with detritus removed and trimmed.

Examples

Reduce a matched URL to its stable core:

>>> from my import RegexStore
>>> RegexStore.format_url('https://example.com/page#frag')
'example.com/page'

static RegexStore.atom(*contents: RegexVal) → RegexVal#

Wraps regex content in word-boundary assertions for atomic matching.

This function takes one or more regex values and wraps them in word start ((?P>_ws)) and word end ((?P>_we)) boundary assertions. This ensures the pattern matches complete words or atomic units rather than partial matches.

Parameters:

*contents – One or more regex values (strings, lists, or tuples) to wrap.

Returns:

Multiple contents: tuple with all wrapped together
Single string: string with boundaries
Single list: list with boundaries prepended/appended
Single tuple: tuple with boundaries integrated based on mark type

Return type:

A regex value wrapped with word boundary assertions. Format depends on input

Raises:

ValueError – If no content provided or tuple has invalid length.

Examples

Strings gain the assertions inline; multiple values become a DSL tuple:

>>> from my import RegexStore
>>> RegexStore.atom(r'cat')
'(?P>_ws)cat(?P>_we)'
>>> RegexStore.atom(r'cat', r'dog')
('[]:', '(?P>_ws)', ['cat', 'dog'], '(?P>_we)')

`VIII` Development Utilities#

RegexStore.sanitize(pattern: str | Pattern[str] | Buffer | Regex | Atom) → str#

Sanitize a pattern by normalizing inline flag syntax.

Parameters:: pattern – Either a known pattern’s name, a compiled pattern, or a raw expression.
Returns:: Sanitized pattern string with normalized flag syntax.

Examples

Scoped inline flags become plain groups with a leading flag atom:

>>> from my import RegexStore
>>> RegexStore.new().sanitize(r'(?i:abc)')
'(?:(?i)abc)'

RegexStore.pretty_print(pattern: str | Pattern[str] | Buffer | Regex | Atom, print_head: bool = True, depth: int = 0, maxdepth: int = 6, threshold: int = 48) → str#

Pretty-print a regex pattern as an indented multiline tree structure.

Parameters:

pattern – The name of an existing pattern, a compiled pattern, or a raw expression.
print_head – Whether to include the pattern header (i.e. (?(DEFINE)...)) in output.
depth – Current recursion depth.
maxdepth – Maximum depth to print before truncating branches.
threshold – Maximum length of a branch before truncating.

Returns:

Multi-line string representation with indentation showing nesting.

Examples

Render a stored pattern with its DEFINE header:

>>> from my import RegexStore
>>> store = RegexStore.new(
...     year=r'\d{4}',
...     month=r'0?[1-9]|1[0-2]',
...     date=r'(?P<y>(?P>year))-(?P<m>(?P>month))',
... )
>>> print(store.pretty_print('date'))
(?(DEFINE)
(?P<year>\d{4})
(?P<month>0?[1-9]|1[0-2])
)
(?P<y>(?P>year))-(?P<m>(?P>month))

RegexStore: Readable, Optimized Regular Expressions#

I Initial Methods#

II Primary Methods#

III Overrides#

IV Top-Level Matching Methods#

V Functional Utilities#

VI Optimization Functions#

VII Expression Construction#

VIII Development Utilities#

`RegexStore`: Readable, Optimized Regular Expressions#

`I` Initial Methods#

`II` Primary Methods#

`III` Overrides#

`IV` Top-Level Matching Methods#

`V` Functional Utilities#

`VI` Optimization Functions#

`VII` Expression Construction#

`VIII` Development Utilities#