Sensitive Data Filtering
DD has built-in automatic sensitive-data filtering that replaces passwords, API keys, credit card numbers, and other sensitive information with [REDACTED] before logs are written, preventing leakage into logs.
Quick Enable
cfg := dd.DefaultConfig()
cfg.Security = dd.DefaultSecurityConfig() // Basic filtering (already the DefaultConfig default; made explicit here)
logger, err := dd.New(cfg)
if err != nil {
log.Fatal(err)
}
defer logger.Close()
// The password field is redacted automatically
logger.InfoWith("user login",
dd.String("username", "alice"),
dd.String("password", "s3cr3t123"), // Output: password=[REDACTED]
)Built-in Filter Patterns
DefaultSecurityConfig() enables basic filtering, covering the following sensitive information:
| Category | Matched Targets |
|---|---|
| Passwords | password, passwd, pwd, etc. fields |
| API Keys | api_key, apikey, access_token, etc. |
| Credit cards | Visa, MasterCard, and other card-number formats |
| SSN | US Social Security Number format |
| Phone numbers | Global phone-number formats (including international) |
SecurityConfigForLevel(dd.SecurityLevelStandard) uses the full pattern set (all of basic plus the following common categories; the former DefaultSecureConfig() is deprecated and returns an equivalent config):
| Category | Matched Targets |
|---|---|
| Emails | email address format |
| IP addresses | IPv4/IPv6 addresses |
| JWT tokens | JWT format starting with eyJ |
| Connection strings | passwords inside database connection strings |
Pattern Coverage
The table above only lists common-category examples. DefaultSecurityConfig() (the basic set) actually ships 38 patterns, covering AWS keys, Stripe/GitHub/Slack tokens, Log4Shell, and more; industry presets go higher: HealthcareConfig() 73, FinancialConfig() 74, GovernmentConfig() 75. Note that IBAN and multi-country tax IDs (ABN/NZ-IRD/RUT/CPF/RFC) belong to the full set and industry presets only, not the basic set. See the source internal/patterns.go for the full pattern set.
Custom Filter Patterns
Adding Custom Patterns
filter := dd.NewEmptySensitiveDataFilter()
// Add custom regex patterns (with built-in ReDoS protection)
_ = filter.AddPattern(`(?i)credit_card\s*[:=]\s*\d+`)
_ = filter.AddPattern(`(?i)phone\s*[:=]\s*\d{11}`)
cfg := dd.DefaultConfig()
cfg.Security = &dd.SecurityConfig{
SensitiveFilter: filter,
}
logger, err := dd.New(cfg)
if err != nil {
log.Fatal(err)
}
defer logger.Close()Creating a Custom Filter From Scratch
filter, err := dd.NewCustomSensitiveDataFilter(
`(?i)secret_key\s*[:=]\s*\S+`,
`(?i)private_token\s*[:=]\s*\S+`,
)
if err != nil {
log.Fatal(err) // Regex safety validation failed
}ReDoS Protection
NewCustomSensitiveDataFilter has built-in ReDoS validation. If a regex carries catastrophic backtracking risk, it returns an error. Use non-greedy matching and anchored patterns to avoid this.
Security Levels
DD provides five security levels, each with a corresponding preset configuration:
// Get config by level
cfg := dd.SecurityConfigForLevel(dd.SecurityLevelStandard)| Level | Description | Use Case |
|---|---|---|
SecurityLevelDevelopment | Most permissive | Local development |
SecurityLevelBasic | Basic filtering | Test environments |
SecurityLevelStandard | Standard filtering | General production |
SecurityLevelStrict | Strict filtering | High-security requirements |
SecurityLevelParanoid | Most strict | Sensitive industries (finance/medical) |
Industry-Compliance Presets
DD provides three industry-compliance configurations:
HIPAA (Medical)
cfg := dd.HealthcareConfig()Extra filtering: ICD-10 codes, medical record numbers (MRN), health insurance claim numbers (HICN), patient identifiers.
PCI-DSS (Financial Payments)
cfg := dd.FinancialConfig()Extra filtering: SWIFT codes, IBAN account numbers, CVV/CVC, bank routing numbers.
Government Standard
cfg := dd.GovernmentConfig()Extra filtering: passport numbers, driver's license numbers, tax IDs, SSN variants.
Complete Example
// Medical system: HIPAA-compliant config
cfg := dd.DefaultConfig()
cfg.Format = dd.FormatJSON
cfg.Security = dd.HealthcareConfig()
cfg.Targets = []dd.OutputTarget{
dd.FileOutput("logs/hipaa-audit.json"),
}
logger, err := dd.New(cfg)
if err != nil {
log.Fatal(err)
}
defer logger.Close()
// Log messages with sensitive info are auto-redacted
logger.Info("patient record mrn=MRN123456 diagnosis=J18.9 updated")
// MRN and ICD-10 codes in the message match patterns and are redacted
// Structured fields are filtered by key-name sensitivity
logger.InfoWith("user login",
dd.String("password", "s3cr3t123"), // -> [REDACTED] (key name is sensitive)
dd.String("department", "internal"), // Normal output
)Filter Statistics
Monitor filter runtime state:
filter := dd.NewSensitiveDataFilter()
stats := filter.GetFilterStats()
fmt.Printf("active goroutines: %d\n", stats.ActiveGoroutines)
fmt.Printf("filter pattern count: %d\n", stats.PatternCount)
fmt.Printf("total redactions: %d\n", stats.TotalRedactions)
fmt.Printf("timeout count: %d\n", stats.TotalTimeouts)Disabling Filtering
// Use the Development security level (no sensitive-data filtering)
cfg := dd.DefaultConfig()
cfg.Security = dd.SecurityConfigForLevel(dd.SecurityLevelDevelopment)
logger, err := dd.New(cfg)
if err != nil {
log.Fatal(err)
}
// Or manually set an empty SensitiveFilter
cfg = dd.DefaultConfig()
cfg.Security = &dd.SecurityConfig{
SensitiveFilter: nil,
}
logger, err = dd.New(cfg)
if err != nil {
log.Fatal(err)
}
defer logger.Close()Filtering Is Enabled by Default
DefaultConfig() enables basic sensitive-data filtering by default (DefaultSecurityConfig()). Even when the Security field is not set, the default security config is used. To disable filtering, you must explicitly set SensitiveFilter to nil.
Next Steps
- Field Validation -- Key-name conventions and injection protection
- Integrity -- Tamper-proofing with HMAC
- Industry Compliance Configuration -- HIPAA/PCI-DSS in depth
- API Reference - Security -- Complete security API documentation
- Production Checklist -- Pre-launch checks