PDF extraction / a reproducible experiment
Finding the words
between the lines.
Line numbers are not sentences. Columns are not neighbours. We built a geometry-first extractor for the legal appendix shared on Hacker News—and kept the evidence, the code, and the failures.
place. There was no operational software, but in my head, I was able to do that on a one-house-by-one-house basis.
Same page. Different reading.
Real output from the same input PDF. Switch the baseline, inspect the full page, or try a phrase. No hand-repaired “after” text.
A sentence, interrupted by its line numbers
The transcript’s left gutter and right timestamps interrupt ordinary text search. The geometry version separates those lanes and keeps the speech together. This excerpt is taken directly from the saved output.
place. 2 head, I was able to do that on a 14:21:55 3 one-house-by-one-house basis. 14:21:57 4 BY MS. WANG: 14:22:09 5 Q There was no operational software, but in my You mentioned you made modifications to the 14:21:52 14:22:10 6 off-the-shelf hardware to collect data; is that 14:22:14 7 correct?
place. There was no operational software, but in my head, I was able to do that on a one-house-by-one-house basis. BY MS. WANG: Q You mentioned you made modifications to the off-the-shelf hardware to collect data; is that correct?
This demo searches saved outputs. Both sides use the same case, punctuation and whitespace normalization; the new method searches only within its own block-local variants. It does not run PDF extraction or OCR in your browser. A missing phrase can mean an extraction error or an absent text layer. See the warnings and the original page.
Keep the geometry.
Then decide what belongs together.
PDFium decodes the PDF. Our code supplies the reading rules. The final extraction path makes no LLM calls.
Schematic, not a segmentation screenshot. Amber: classified marginal material. Green: separate reading regions. The downloadable output contains the actual boxes and decisions.
Read characters, not a flattened page
PDFium supplies Unicode, character boxes, font and rotation information. The code rebuilds spacing from positions rather than treating a plain-text dump as its input.
Find the lanes before joining the prose
Repeated whitespace suggests a column boundary. A vertical run of numerals or timestamps needs sequence and geometry evidence before it becomes a gutter. Short Q/A labels stay with their lines.
Classify the margins; keep the evidence
Recurring page-edge strings identify headers and footers. Classified text stays in structured metadata with its original coordinates and the reason—not simply deleted from the record.
Order blocks, then search within them
Spatial cuts order the inferred blocks. Search tries literal and conservative dehyphenated variants inside a block, returning page coordinates for each hit. A wrong block boundary can still cause a miss or a bad join.
The useful distinction is not PDF versus text. It is characters with evidence versus text without its layout. A missing or inaccurate text layer is a different problem; this implementation does no OCR.
Useful improvements.
Not a universal fix.
The main score keeps missing-body and noisy-OCR phrases in the denominator. The illustrations are selected examples; the evaluation is a separately declared positional sample.
Visually checked phrases found
Strict matching: Unicode NFC and collapsed whitespace, case-sensitive, literal punctuation. Only declared line-wrap alternatives are allowed. These are selected-phrase results, not a percentage of pages correctly reconstructed.
17 of 42 checked phrases still fail: 10 absent/partial body and 7 noisy existing text/case/spacing. This is not a general accuracy rate.
Whole-file extraction time
| Method | Seconds | Pages / s |
|---|---|---|
| pdftotext default | 0.371 | 1005.4 |
| pdftotext -layout | 0.379 | 984.4 |
| PDFium plaintext | 0.605 | 616.5 |
| pdfgeo r23 | 7.503 | 49.7 |
Three serial warm-cache subprocess trials per method; table uses median wall time including interpreter startup and output writing. Ranges: pdftotext default 0.368–0.394 s; pdftotext -layout 0.377–0.386 s; PDFium plaintext 0.582–0.697 s; pdfgeo r23 7.425–7.635 s. Not a cold-cache or performance-improvement claim.
Read the evaluation method and every individual result
The fixed sample contains physical pages 1, 20, 39, 58, 77, 96, 115, 134, 153, 172, 191, 210, 229, 248, 267, 286, 305, 324, 343, 362 and 373. Two phrases per page were selected from predetermined spatial positions; sparse figures and notices have recorded adjustments. No sampled page was dropped.
An independent reader inspected every sampled page and a second model checked the reference wording against rendered images. The reference was frozen before the first implementation score. The builder then received the failures. Later scores are development rechecks on disclosed examples, not held-out validation.
All methods received the same PDF bytes. The 42-check score includes eight checks on absent-body pages, two on a partial-body page, eight on noisy existing text layers, and 24 on pages with the sampled native text present. The code also reports seven specified false-interleaving probes, 21 header/footer-cleanup checks and 57 page-label checks. An unassessable probe is not a pass. Metadata roles require the accompanying adjudication, not merely matching a numeral somewhere on a page.
The browser search demonstration is deliberately separate: it uses the CLI’s case/punctuation-normalized, block-local search streams. It does not reproduce the stricter text-fidelity score above.
What geometry cannot fix.
Making missing text obvious is part of the result. These cases remain in the record, not quietly outside the denominator.
Pixels still need recognition
Page 75 visibly contains a jury instruction, while its native text layer contains stamps and labels rather than the body. The extractor flags risk; it does not OCR. Its classifier signals are heuristic, not an exhaustive coverage guarantee.
A coordinate cannot restore a dropped digit
On page 300 the source image reads 11.54 kWh; all compared native-text outputs retain 1.54 kWh. The saved output preserves that error rather than repairing it. Compare the crop with the original appendix.
Source crop from physical page 300; output remains the existing native layer.
Not every layout is solved
Tight table columns, last-column rows, and nonrecurring headings remain difficult. Image warnings are useful signals, not coverage proof; wrong inferred blocks can still produce a bad join.
About the original discussion
Rayiner explicitly says his tool uses glyph positions. This is not a benchmark against his implementation: we do not have it. Our comparisons are against the named baselines above.
About “lower level”
Plain-text output discards layout information. But pdftotext also offers bounding-box modes. The experiment is about using character geometry to reconstruct reading order—not discovering that PDF coordinates exist.
Take it apart.
The implementation, exact input identity, reference phrases, and complete outputs are available below. Start with the source bundle.
python3 -m pip install -r requirements.txt python3 pdfgeo.py extract appendix.pdf out python3 pdfgeo.py search out "There was no operational software, but in my head" --json python3 pdfgeo.py overlay appendix.pdf out 210 overlay.png
Input identity, environment, and build disclosure
SHA-256 b11e78413fee46e151eb441556afca793ad3378ea76debd7bc3811bc9f988d49
- Original Hacker News discussion — the challenge, not a performance baseline.
- EcoFactor, Inc. v. Google LLC · document 15 — the shared appendix.
- Read the extractor source directly · Usage and limitations · Baseline version and modes.