Skip to content

Package Functions

Package-level functions are suited for one-off call scenarios; they reuse Processor instances internally via sync.Pool and require no manual lifecycle management. Note: pooled Processors have cache and audit retention disabled; for caching/statistics/audit, use html.New to create a dedicated Processor.

Internal Mechanism

INFO

Package functions maintain a sync.Pool underneath that reuses Processor instances between calls to avoid repeated allocation. Key implementation details:

  • Pooled config disables the cache: the pooled config (poolCfg) is based on DefaultConfig() but explicitly zeroes three cache-related fields — MaxCacheEntries=0, CacheTTL=0, CacheCleanup=0. As a result, package functions cannot leverage the cache; every call is a full processing pass. This is by design, because the pooled Processor clears its cache on every return; enabling the cache would only pay hashing and map-write overhead that could never hit.
  • State reset on return: before returning the Processor at the end of each call, ResetStatistics, audit.Wait(), ClearAuditLog, and ClearCache are executed in turn, preventing cross-call leakage of statistics, audit, and cache state.
  • Closed Processors are not returned: if a Processor is closed during use (a misuse), the return logic discards it instead of putting it back in the pool (sync.Pool tolerates a missing Put; the next Get rebuilds via pool.New).
  • Panic fallback: pool.New only panics when a library invariant is broken (poolCfg is derived from DefaultConfig() and is valid by construction); this panic is caught by getPooledProcessorSafe, wrapped into ErrInternalPanic, and returned — it never escapes to the public API.

Config Parameter Resolution

The cfg ...Config of all package functions is an optional variadic parameter, parsed by the internal resolveConfig:

ArgumentBehaviorPooled?
NoneUses DefaultConfig()Yes (pooled=true)
OneUses that ConfigNo (pooled=false)
Two or moreReturns ErrMultipleConfigs

WARNING

When a custom Config is passed, the call does not go through sync.Pool — the pool only stores Processors based on DefaultConfig() and cannot safely reuse instances with different configs. In this case each call News a temporary Processor and Closes it when done. If you need to reuse a custom config across high-frequency calls, create a Processor directly.

Content Extraction

Extract

Extract content from HTML bytes, returning the full Result.

go
func Extract(htmlBytes []byte, cfg ...Config) (*Result, error)

Parameters:

ParameterTypeDescription
htmlBytes[]byteHTML content
cfg...Configoptional config, at most one

Example:

go
result, err := html.Extract(data)
if err != nil {
    log.Fatal(err)
}
fmt.Println(result.Title, result.Text)

Full runnable example (demonstrates field access and error handling):

go
package main

import (
	"fmt"
	"log"

	"github.com/cybergodev/html"
)

func main() {
	data := []byte(`<html><head><title>Example Page</title></head>
<body><h1>Welcome</h1><p>Body content <a href="https://example.com">link</a>.</p></body></html>`)

	// No Config passed — goes through the pooled path
	result, err := html.Extract(data)
	if err != nil {
		log.Fatalf("extraction failed: %v", err)
	}

	fmt.Println("Title:", result.Title)
	fmt.Println("Words:", result.WordCount)
	fmt.Println("Links:", len(result.Links))
	// Output:
	// Title: Example Page
	// Words: 4
	// Links: 1
}

Error returns: Extract returns the same errors as Processor.Extract and may additionally return:

ErrorCondition
ErrMultipleConfigsTwo or more Config arguments passed
ErrInvalidConfig (wrapped in *ConfigError)The passed Config fails validation (e.g. MaxInputSize<=0)

ExtractFromFile

Extract content from an HTML file.

go
func ExtractFromFile(filePath string, cfg ...Config) (*Result, error)

Error returns: in addition to Extract's errors, file access may return a *FileError wrapping ErrFileNotFound, ErrInvalidFilePath, or a path-traversal denial (see Security for AllowedBaseDir).

Text Extraction

ExtractText

Extract only the plain text content.

go
func ExtractText(htmlBytes []byte, cfg ...Config) (string, error)

ExtractTextFromFile

Extract plain text from a file.

go
func ExtractTextFromFile(filePath string, cfg ...Config) (string, error)

Context Variants

All functions have variants that take a context.Context for cancellation and timeout control:

FunctionSignature
ExtractWithContext(ctx context.Context, htmlBytes []byte, cfg ...Config) (*Result, error)
ExtractFromFileWithContext(ctx context.Context, filePath string, cfg ...Config) (*Result, error)
ExtractTextWithContext(ctx context.Context, htmlBytes []byte, cfg ...Config) (string, error)
ExtractTextFromFileWithContext(ctx context.Context, filePath string, cfg ...Config) (string, error)
go
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

result, err := html.ExtractWithContext(ctx, data)

Output Formats

FunctionSignatureDescription
ExtractToMarkdown(htmlBytes []byte, cfg ...Config) (string, error)HTML → Markdown
ExtractToMarkdownFromFile(filePath string, cfg ...Config) (string, error)File → Markdown
ExtractToMarkdownWithContext(ctx context.Context, htmlBytes []byte, cfg ...Config) (string, error)With context
ExtractToMarkdownFromFileWithContext(ctx context.Context, filePath string, cfg ...Config) (string, error)File + context
ExtractToJSON(htmlBytes []byte, cfg ...Config) ([]byte, error)HTML → JSON
ExtractToJSONFromFile(filePath string, cfg ...Config) ([]byte, error)File → JSON
ExtractToJSONWithContext(ctx context.Context, htmlBytes []byte, cfg ...Config) ([]byte, error)With context
ExtractToJSONFromFileWithContext(ctx context.Context, filePath string, cfg ...Config) ([]byte, error)File + context

For detailed usage and examples see Output Formats.

FunctionSignatureDescription
ExtractAllLinks(htmlBytes []byte, cfg ...Config) ([]LinkResource, error)Extract all links
ExtractAllLinksFromFile(filePath string, cfg ...Config) ([]LinkResource, error)Extract links from a file
ExtractAllLinksWithContext(ctx context.Context, htmlBytes []byte, cfg ...Config) ([]LinkResource, error)With context
ExtractAllLinksFromFileWithContext(ctx context.Context, filePath string, cfg ...Config) ([]LinkResource, error)File + context

For detailed usage and examples see Link Extraction.

Batch Processing

FunctionSignatureDescription
ExtractBatch(htmlContents [][]byte, cfg ...Config) *BatchResultBatch extraction
ExtractBatchWithContext(ctx context.Context, htmlContents [][]byte, cfg ...Config) *BatchResultWith context
ExtractBatchFiles(filePaths []string, cfg ...Config) *BatchResultBatch file extraction
ExtractBatchFilesWithContext(ctx context.Context, filePaths []string, cfg ...Config) *BatchResultFiles + context

For detailed usage and examples see Batch Processing.

Package Functions vs Processor

Both ultimately call Processor underneath, but they differ markedly in resource reuse and state retention:

DimensionPackage functionsProcessor
CacheNone (pooled config MaxCacheEntries=0)Yes (returns a deep copy on hit)
StatisticsReset each call (ResetStatistics on return)Accumulated; GetStatistics anytime
Audit logCleared each call (ClearAuditLog on return)Accumulated; queryable via GetAuditLog
Custom ConfigCreates + destroys a temporary Processor each callReuses the same instance
LifecycleAuto-managed (pool / temporary instance)Requires manual defer Close()
Suited forOne-off calls, scripts, low-frequency requestsHigh-frequency calls, long-running services, cache-needing

TIP

For a single extraction or an occasional call, package functions are the easiest. If you need to extract repeatedly inside loops, HTTP handlers, or batch jobs, create a long-lived Processor and reuse it; the cache can then substantially reduce overhead.