Patterns Reference

This page provides detailed documentation for all built-in anonymization patterns and guidance on creating custom patterns.

Built-in Patterns

pgEdge Anonymizer includes over 180 built-in patterns covering common PII types, with extensive support for country-specific data formats across 19 countries: Australia, Canada, Finland, France, Germany, India, Ireland, Italy, Japan, Mexico, New Zealand, Norway, Pakistan, Singapore, South Korea, Spain, Sweden, United Kingdom, and United States.

Person Names

PERSON_NAME

Generates realistic full names (first and last name combined).

Input/Output Examples:

Format Preservation:

• Detects comma-separated format (Last, First)
• Preserves all-uppercase input
• Preserves all-lowercase input

PERSON_FIRST_NAME

Generates first names only.

Input/Output Examples:

Format Preservation:

• Preserves all-uppercase input
• Preserves all-lowercase input

PERSON_LAST_NAME

Generates last names only.

Input/Output Examples:

Format Preservation:

• Preserves all-uppercase input
• Preserves all-lowercase input

Date of Birth

DOB

Generates dates of birth for any age (0-100 years).

Input/Output Examples:

Format Detection:

• ISO format: YYYY-MM-DD
• US format: MM/DD/YYYY
• US short: MM/DD/YY
• Long format: Month DD, YYYY

DOB_OVER_13

Generates dates of birth ensuring the person is at least 13 years old.

DOB_OVER_16

Generates dates of birth ensuring the person is at least 16 years old.

DOB_OVER_18

Generates dates of birth ensuring the person is at least 18 years old. Commonly used for adult-only services.

DOB_OVER_21

Generates dates of birth ensuring the person is at least 21 years old. Commonly used for alcohol-related services in the US.

Contact Information

EMAIL

Generates realistic email addresses using combinations of first names, last names, and common domain names. Each generated email includes a unique hash-based suffix derived from the input to ensure uniqueness even with large datasets and unique constraints.

Input/Output Examples:

Features:

• Unique suffix ensures no collisions with unique database constraints
• Same input always produces same output (deterministic)
• Multiple format variations (first.last, flast, firstl, first_last)

US_PHONE

Generates US-format phone numbers.

Input/Output Examples:

Format Preservation:

• Detects and preserves separator style (dashes, dots, spaces)
• Detects and preserves parentheses around area code
• Preserves digit-only format

UK_PHONE

Generates UK-format phone numbers with country code.

Input/Output Examples:

INTERNATIONAL_PHONE

Generates international phone numbers with country codes.

Input/Output Examples:

WORLDWIDE_PHONE

Generates generic phone numbers (digits only, most permissive).

Input/Output Examples:

ADDRESS

Generates street addresses from diverse worldwide data. This pattern randomly selects a country format (US, UK, German, Japanese, etc.) and generates an address in that country's style.

Input/Output Examples:

Generated Format:

• Randomly selects from 19 country-specific formats
• Street number, street name, city, and postal code
• Country-appropriate address structure

For country-specific addresses, use patterns like US_ADDRESS, UK_ADDRESS, DE_ADDRESS, etc.

WORLDWIDE_ADDRESS

Alias for ADDRESS. Generates street addresses from diverse worldwide data.

CITY

Generates city names from diverse worldwide data. Combines city names from all 19 supported countries.

Input/Output Examples:

Format Preservation:

• Preserves all-uppercase input
• Preserves all-lowercase input

For country-specific cities, use patterns like US_CITY, UK_CITY, DE_CITY, etc.

WORLDWIDE_CITY

Alias for CITY. Generates city names from diverse worldwide data.

Postal Codes

US_ZIP

Generates US ZIP codes in 5-digit or ZIP+4 format.

Input/Output Examples:

Format Preservation:

• Detects 5-digit vs ZIP+4 format
• Preserves hyphen separator for ZIP+4

UK_POSTCODE

Generates UK postcodes in valid format.

Input/Output Examples:

Features:

• Generates valid UK postcode patterns (A9 9AA, A99 9AA, AA9 9AA, AA99 9AA)
• Preserves space separator
• Uses valid postcode letter combinations

CA_POSTCODE

Generates Canadian postcodes in valid format.

Input/Output Examples:

Features:

• Generates valid Canadian format (A9A 9A9)
• Preserves space separator
• Uses valid postal code letters (excludes D, F, I, O, Q, U in certain positions)

WORLDWIDE_POSTCODE

Generates postcodes in various international formats. Auto-detects the input format and generates matching output.

Input/Output Examples:

Features:

• Auto-detects US, UK, or Canadian format from input
• Falls back to random format selection for ambiguous input
• Useful when postal code format varies within a column

Financial Information

CREDIT_CARD

Generates valid credit card numbers that pass Luhn checksum validation.

Input/Output Examples:

Features:

• Generates valid Luhn checksum
• Preserves card type prefix (Visa starts with 4, etc.)
• Preserves separator style (dashes, spaces, none)

CREDIT_CARD_EXPIRY

Generates credit card expiration dates (always in the future).

Input/Output Examples:

CREDIT_CARD_CVV

Generates 3 or 4 digit CVV codes.

Input/Output Examples:

Format Preservation:

• 3-digit input produces 3-digit output
• 4-digit input produces 4-digit output (Amex style)

Government Identifiers

US_SSN

Generates US Social Security Numbers in valid format.

Input/Output Examples:

Features:

• Avoids invalid SSN ranges (000, 666, 900-999 area numbers)
• Preserves separator style

UK_NI

Generates UK National Insurance numbers in valid format.

Input/Output Examples:

Format:

• Two prefix letters (excluding D, F, I, Q, U, V)
• Six digits
• One suffix letter (A, B, C, or D)

UK_NHS

Generates UK NHS numbers with valid check digit.

Input/Output Examples:

Features:

• 10 digits total
• Valid modulus 11 check digit
• Preserves space formatting

PASSPORT

Generates passport-style document numbers.

Input/Output Examples:

Format:

• 9 alphanumeric characters
• Mix of letters and numbers

Text Content

LOREMIPSUM

Generates lorem ipsum placeholder text matching the approximate length of the original content.

Input/Output Examples:

Features:

• Matches approximate word count of input
• Uses standard lorem ipsum vocabulary
• Suitable for notes, comments, and free-text fields

Network Identifiers

IPV4_ADDRESS

Generates IPv4 addresses.

Input/Output Examples:

Features:

• Generates valid IPv4 format (x.x.x.x)
• Mixes private and public-looking ranges
• Avoids reserved ranges (0.x.x.x, 127.x.x.x, multicast)

IPV6_ADDRESS

Generates IPv6 addresses.

Input/Output Examples:

Features:

• Supports full and compressed (::) formats
• Preserves uppercase/lowercase preference
• Generates valid hex groups

HOSTNAME

Generates hostnames and fully qualified domain names (FQDNs).

Input/Output Examples:

Features:

• Detects FQDN vs simple hostname
• Preserves numeric suffixes (e.g., web01 → node42)
• Uses realistic server naming conventions

Country-Specific Patterns

pgEdge Anonymizer provides extensive country-specific patterns for names, phone numbers, addresses, postal codes, and ID numbers. This allows you to generate data that matches the format and style expected for a specific country.

Country-Specific Phone Numbers

Country-Specific Postcodes

Country-Specific Addresses

Worldwide Name Generators

For data that may come from any country, use the worldwide generators:

Country-Specific Names

For each supported country, generators are available for first names, last names, full names, and cities. The pattern names follow the format {COUNTRY_CODE}_{TYPE}:

Examples:

• US_FIRST_NAME - American first names (John, Mary, Michael)
• JP_LAST_NAME - Japanese surnames (Tanaka, Suzuki, Sato)
• DE_NAME - German full names (Hans Mueller, Anna Schmidt)
• UK_CITY - UK cities with counties (London, Greater London)

Country-Specific ID Numbers

Custom Patterns

You can define custom patterns in a separate YAML file. Custom patterns can either reference built-in generators or define format-based patterns that generate data based on format strings.

Pattern File Format

patterns:
  - name: PATTERN_NAME
    replacement: "generator name or format hint"
    note: "Human-readable description"

Format-Based Patterns

Format-based patterns allow you to define custom data formats using format codes. This is useful when you need to generate data in a specific format that isn't covered by the built-in patterns.

There are three types of format patterns:

• date - Uses strftime format codes for date/time generation
• number - Uses printf format codes for number generation
• mask - Uses character placeholders for pattern-based generation

Date Format Patterns

Date patterns use strftime-style format codes:

Example date format patterns:

patterns:
  - name: DATE_US_FORMAT
    format: "%m/%d/%Y"
    type: date
    note: US-style date (MM/DD/YYYY)

  - name: DATE_ISO
    format: "%Y-%m-%d"
    type: date
    min_year: 1990
    max_year: 2024
    note: ISO date with year range

  - name: DATETIME_FULL
    format: "%Y-%m-%d %H:%M:%S"
    type: date
    note: Full datetime

  - name: DATE_LONG
    format: "%B %d, %Y"
    type: date
    note: Long format (January 15, 2024)

Number Format Patterns

Number patterns use printf-style format codes:

Example number format patterns:

patterns:
  - name: ORDER_NUMBER
    format: "ORD-%08d"
    type: number
    min: 1
    max: 99999999
    note: Order number (ORD-00000001)

  - name: INVOICE_NUMBER
    format: "INV-%06d"
    type: number
    min: 100000
    max: 999999
    note: Invoice number (INV-100000)

  - name: ACCOUNT_ID
    format: "%010d"
    type: number
    min: 1000000000
    max: 9999999999
    note: 10-digit account ID

Mask Format Patterns

Mask patterns use character placeholders:

Example mask format patterns:

patterns:
  - name: PRODUCT_SKU
    format: "SKU-AA-####"
    type: mask
    note: Product SKU (SKU-AB-1234)

  - name: LICENSE_PLATE
    format: "AAA-####"
    type: mask
    note: License plate (ABC-1234)

  - name: EMPLOYEE_ID
    format: "EMP-AAA-###"
    type: mask
    note: Employee ID (EMP-ABC-123)

  - name: SERIAL_NUMBER
    format: "SN\\-AAAA\\-####\\-XX"
    type: mask
    note: Serial number with escaped hyphens

Auto-Detection

The type field is optional. If omitted, the type is auto-detected based on the format string:

• Formats containing strftime codes (%Y, %m, etc.) are detected as date
• Formats containing printf codes (%d, %08d, etc.) are detected as number
• All other formats are treated as mask

patterns:
  - name: AUTO_DATE
    format: "%Y-%m-%d"
    note: Auto-detected as date type

  - name: AUTO_NUMBER
    format: "%05d"
    min: 1
    max: 99999
    note: Auto-detected as number type

  - name: AUTO_MASK
    format: "##-AAA-##"
    note: Auto-detected as mask type

Example Custom Patterns File

See examples/custom_patterns.yaml for a comprehensive example of custom format patterns.

Using Custom Patterns

Reference your custom patterns file in the configuration:

patterns:
  user_path: ./custom-patterns.yaml

columns:
  - column: public.employees.employee_id
    pattern: EMPLOYEE_ID

Or via command line:

pgedge-anonymizer run --config config.yaml --patterns custom-patterns.yaml

Pattern Selection Guide

Choose patterns based on your data type:

Generic Patterns

Country-Specific Patterns

For data that should match a specific country's format, use the country-specific patterns. The country code prefix indicates the country:

Example: Anonymizing a UK customer database:

columns:
  - column: customers.full_name
    pattern: UK_NAME
  - column: customers.phone
    pattern: UK_PHONE
  - column: customers.address
    pattern: UK_ADDRESS
  - column: customers.postcode
    pattern: UK_POSTCODE
  - column: customers.ni_number
    pattern: UK_NI

Consistency

Within a single anonymization run, the same input value always produces the same output value. This ensures:

• Referential integrity is maintained
• Related records stay linked
• Reports and analytics remain meaningful

For example, if "[email protected]" is anonymized to "[email protected]", every occurrence of that email in the database will be replaced with the same anonymized value.