feat(validate): email+URL validators with DNS/IDN, redirect and reputation checks by hyp3rd · Pull Request #31 · hyp3rd/sectools

hyp3rd · 2026-01-10T14:35:44Z

Introduce pkg/validate with email and URL validation helpers (docs/tests included)
Email: syntax checks, optional IDN support, TLD requirements, IP-literal policy, and domain verification via DNS (MX/A); clear error types (e.g., ErrEmailDomainInvalid, ErrEmailDomainLookupFailed, ErrEmailDomainUnverified, ErrEmailIPLiteralNotAllowed)
URL: scheme allowlist, host allow/block rules, optional IP-literal support, redirect policy (depth/allow), reputation checker hook; error types (e.g., ErrURLSchemeNotAllowed, ErrURLHostMissing, ErrURLRedirectNotAllowed, ErrURLRedirectLoop, ErrURLReputationBlocked)
Docs: add usage examples and security checklist guidance for input validation
Chore: extend cspell dictionary with IDN/DNS-related terms
Deps: add/upgrade golang.org/x/net v0.48.0 and golang.org/x/text v0.32.0

…ation checks - Introduce pkg/validate with email and URL validation helpers (docs/tests included) - Email: syntax checks, optional IDN support, TLD requirements, IP-literal policy, and domain verification via DNS (MX/A); clear error types (e.g., ErrEmailDomainInvalid, ErrEmailDomainLookupFailed, ErrEmailDomainUnverified, ErrEmailIPLiteralNotAllowed) - URL: scheme allowlist, host allow/block rules, optional IP-literal support, redirect policy (depth/allow), reputation checker hook; error types (e.g., ErrURLSchemeNotAllowed, ErrURLHostMissing, ErrURLRedirectNotAllowed, ErrURLRedirectLoop, ErrURLReputationBlocked) - Docs: add usage examples and security checklist guidance for input validation - Chore: extend cspell dictionary with IDN/DNS-related terms - Deps: add/upgrade golang.org/x/net v0.48.0 and golang.org/x/text v0.32.0

Copilot

Pull request overview

This PR introduces a comprehensive email and URL validation package (pkg/validate) with security-focused features including DNS verification, IDN support, redirect tracking, and reputation checking. The implementation follows secure-by-default principles with HTTPS-only enforcement for URLs and strict validation rules for email addresses.

Changes:

New pkg/validate package with EmailValidator and URLValidator types supporting configurable validation rules
Email validation with optional DNS MX/A record verification and IDN domain normalization
URL validation with HTTPS-only enforcement, redirect following (with loop detection), and pluggable reputation checking
Comprehensive error types for granular validation failure reporting
Documentation updates including usage examples and security checklist guidance
Dependencies: added golang.org/x/net v0.48.0 and golang.org/x/text v0.32.0

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 20 comments.

Show a summary per file

File	Description
pkg/validate/email.go	Core email validation logic with DNS verification, IDN support, and comprehensive syntax checking
pkg/validate/email_test.go	Test coverage for email validation including DNS verification, IDN handling, and IP literal scenarios
pkg/validate/url.go	URL validation with HTTPS enforcement, redirect following, reputation checking, and security controls
pkg/validate/url_test.go	Test coverage for URL validation including redirects, reputation blocking, and private IP rejection
pkg/validate/errors.go	Comprehensive error definitions for granular validation failure reporting
pkg/validate/doc.go	Package documentation
docs/usage.md	Usage documentation for email and URL validators
docs/security-checklist.md	Security guidance for input validation
README.md	Example code demonstrating validator usage
go.mod	Added golang.org/x/net dependency
go.sum	Checksums for new dependencies
cspell.json	Dictionary additions for IDN/DNS-related terms

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-10T14:44:25Z

pkg/validate/url_test.go

+	if err != ErrInvalidURLConfig {
+		t.Fatalf("expected ErrInvalidURLConfig, got %v", err)
+	}
+}


Missing test coverage for WithURLMaxLength option. URL length limits are important for preventing DoS attacks, and this should be tested.

Copilot · 2026-01-10T14:44:25Z

pkg/validate/url_test.go

+func TestURLRedirectLoop(t *testing.T) {
+	client := &http.Client{
+		Transport: &fakeRoundTripper{
+			responses: map[string]*http.Response{
+				"https://example.com/loop": {
+					StatusCode: http.StatusFound,
+					Header:     http.Header{"Location": []string{"/loop"}},
+					Body:       io.NopCloser(strings.NewReader("")),
+				},
+			},
+		},
+	}
+
+	validator, err := NewURLValidator(
+		WithURLCheckRedirects(2),
+		WithURLHTTPClient(client),
+	)
+	if err != nil {
+		t.Fatalf("expected validator, got %v", err)
+	}
+
+	_, err = validator.Validate(context.Background(), "https://example.com/loop")
+	if err != ErrURLRedirectLoop {
+		t.Fatalf("expected ErrURLRedirectLoop, got %v", err)
+	}
+}


The test expects ErrURLRedirectLimit when the redirect limit is exceeded, but the code only returns this error after the loop completes (line 598). Consider adding a test case that explicitly exceeds the redirect limit to verify this behavior.

Copilot · 2026-01-10T14:44:26Z

pkg/validate/url_test.go

+	if err != ErrInvalidURLConfig {
+		t.Fatalf("expected ErrInvalidURLConfig, got %v", err)
+	}
+}


Missing test coverage for the WithURLAllowIDN option. IDN (Internationalized Domain Names) support should be tested to ensure proper Unicode domain handling and ASCII conversion.

Copilot · 2026-01-10T14:44:26Z

pkg/validate/url_test.go

+	if err != ErrInvalidURLConfig {
+		t.Fatalf("expected ErrInvalidURLConfig, got %v", err)
+	}
+}


Missing test coverage for WithURLAllowLocalhost option. Localhost URLs are commonly used in development, and this option should be tested to ensure proper validation.

Copilot · 2026-01-10T14:44:26Z

pkg/validate/email.go

+	return true
+}
+
+func isDotAtom(local string) bool {


The bound check is missing before accessing local[0] and local[len(local)-1]. If local is empty, this will panic with an index out of range error.

Suggested change

func isDotAtom(local string) bool {

func isDotAtom(local string) bool {

if len(local) == 0 {

return false

}

Copilot · 2026-01-10T14:44:29Z

pkg/validate/url_test.go

+	if err != ErrInvalidURLConfig {
+		t.Fatalf("expected ErrInvalidURLConfig, got %v", err)
+	}
+}


Missing test coverage for WithURLRedirectMethod option. The redirect method can be HEAD or GET, and both should be tested to ensure proper behavior.

Copilot · 2026-01-10T14:44:29Z

docs/usage.md

+
+Behavior:
+
+- Enforces `https` only; non-https schemes are rejected (including if configured).


The documentation states "non-https schemes are rejected (including if configured)" which is accurate based on the code. However, consider clarifying that attempting to configure non-HTTPS schemes will result in an error during validator construction, rather than during URL validation.

Suggested change

- Enforces `https` only; non-https schemes are rejected (including if configured).

- Enforces `https` only; attempting to configure non-https schemes results in an error during validator construction.

Copilot · 2026-01-10T14:44:29Z

pkg/validate/url.go

+func isRedirectStatus(code int) bool {
+	switch code {
+	case redirectStatusMultipleChoices,
+		redirectStatusMovedPermanently,
+		redirectStatusFound,
+		redirectStatusSeeOther,
+		redirectStatusTemporaryRedirect,
+		redirectStatusPermanentRedirect:
+		return true


The redirect status check includes status code 300 (Multiple Choices), which is typically not automatically followed by clients. Consider documenting this behavior or reconsidering whether 300 should be treated as a redirect that the validator will follow.

Copilot · 2026-01-10T14:44:29Z

pkg/validate/email.go

+	labels := strings.SplitSeq(domain, ".")
+	for label := range labels {
+		err := validateDomainLabel(label)
+		if err != nil {
+			return err
+		}
+	}


Using strings.SplitSeq which was introduced in Go 1.23. Also using for label := range labels pattern with iterators. While the go.mod specifies Go 1.25.5 (which appears to be invalid as Go 1.25 doesn't exist), ensure these iterator-based functions are compatible with the actual Go version being used.

Copilot · 2026-01-10T14:44:30Z

pkg/validate/url.go

+func WithURLCheckRedirects(maxRedirects int) URLOption {
+	return func(cfg *urlOptions) error {
+		if maxRedirects <= 0 {
+			return ErrInvalidURLConfig


The redirect limit validation requires maxRedirects > 0 when redirects are enabled, but the default value is 10. If a user sets WithURLCheckRedirects(0), this will return an error. Consider clarifying in the error message or documentation that the value must be positive when redirects are enabled.

Copilot AI review requested due to automatic review settings January 10, 2026 14:35

Copilot started reviewing on behalf of hyp3rd January 10, 2026 14:36 View session

Copilot AI reviewed Jan 10, 2026

View reviewed changes

hyp3rd merged commit 13035b1 into main Jan 10, 2026
19 checks passed

hyp3rd deleted the feat/validate branch January 12, 2026 13:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(validate): email+URL validators with DNS/IDN, redirect and reputation checks#31

feat(validate): email+URL validators with DNS/IDN, redirect and reputation checks#31
hyp3rd merged 1 commit intomainfrom
feat/validate

hyp3rd commented Jan 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant


		Behavior:

		- Enforces `https` only; non-https schemes are rejected (including if configured).

	- Enforces `https` only; non-https schemes are rejected (including if configured).
	- Enforces `https` only; attempting to configure non-https schemes results in an error during validator construction.

Uh oh!

Conversation

hyp3rd commented Jan 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant