# English PDF-to-Word benchmark methodology

Generated: 2026-08-13T13:27:11Z

## Scope

This benchmark contains 24 manifest-declared English PDF cases. Each case is keyed by a stable ID and points to a source PDF plus a UTF-8 ground-truth text file. The recorded conversion target is `https://pdf-to-word.tools/wp-json/pdf-to-word/v2` and the run started at `2026-08-13T13:17:58Z`. This analyzer is offline and performs no HTTP or other network requests.

## Conversion and timing data

Job status, observations, errors, and timing values come only from `run-log.json`; the analyzer does not infer a successful conversion from an HTTP response. A job counts as ready only when the log status is `ready`. `ready_in_ms` should run from the start of the per-case workflow to the first observed ready state, so it may include health-check latency, upload, queue, conversion, network, and polling delay. `timings_ms.total` may additionally include ticket and download time. These values describe this run and are not a service-level guarantee.

## DOCX validation and OOXML inspection

For every present output, the analyzer verifies that it is a non-corrupt ZIP package containing `[Content_Types].xml` and `word/document.xml`, then parses the main story plus headers, footers, footnotes, endnotes, and comments. It reports:

- total and non-empty paragraphs;
- Word tables, rows, cells, horizontal spans, and vertical merge tags;
- media files, DrawingML/VML objects, and image references;
- explicit page breaks, last-rendered page breaks, page-break-before settings, and sections;
- external package relationships and macro indicators.

`security_clean` means the package has zero external relationships and no detected VBA/macro-enabled parts. It is a narrow package-content check, not a malware certification. OOXML counts show semantic structure but do not prove visual fidelity; layout still requires rendering and human review.

## Text extraction and normalization

Visible `w:t` text is extracted in XML order. Tabs, line breaks, and no-break hyphens are represented; tracked deletions and field instructions are excluded. The extracted text is saved under `extracted-text/` for audit.

Both ground truth and output use normalization profile `english_benchmark_v1`:

1. Unicode NFKC normalization and Unicode casefold.
2. Soft hyphen, zero-width formatting characters, BOM/word joiner, and directional controls are removed.
3. Typographic quotes/apostrophes, dash variants, the minus sign, and ellipsis map to ASCII equivalents.
4. Unicode space variants become ASCII spaces and whitespace collapses to one space.
5. Tokens are word-character runs. Common internal connectors are retained for contractions, URLs, emails, paths, percentages, and technical identifiers.

No language-specific stemming, spelling correction, synonym matching, or OCR substitution is applied.

## Scores

- **Character recall** is multiset overlap of normalized non-whitespace characters divided by the ground-truth character count.
- **Token recall** is multiset overlap of normalized tokens divided by the ground-truth token count.
- Precision and F1 use the corresponding extracted-output denominator.
- **Word error rate (WER)** is token-level Levenshtein edit distance divided by the ground-truth token count; it is order-sensitive and can exceed 1.0.

Scores are rounded to six decimals. `high` means both recall values are at least 0.98, `moderate` means both are at least 0.90, and otherwise the label is `low`. This label describes text retention only, not overall conversion quality.

## Reproduction

```powershell
$python = '<path-to-python>'
& $python .\analyze_benchmark.py
```

Outputs are `results.json`, UTF-8-BOM `results.csv`, `methodology.md`, and per-case files under `extracted-text/`.

## Interpretation limits

The corpus is a controlled engineering sample, not a representative estimate for all English PDFs. Separate digital-text and image-only/OCR-required cases. Do not infer layout fidelity from text scores, and do not claim an image or table was preserved merely because its labels survived as text. Publish per-file failures and category sample sizes alongside averages.