Riksarkivet AI-lab · reading-order diagnostic
Finding the order to read a historical page
Once a page has more than one column, the hard part is not finding the text lines but deciding the order to read them in. On 29 court pages from Göta hovrätt and the Trolldomskommissionen — Riksarkivet's own open data — I compared their htrflow pipeline with a small geometric ordering step. Both find the lines equally well; they differ only in the order.
All 29 pages
Every page tested, worst HTRflow score first. Red = wrong column order (τ < 0.8). Click any page to compare it above.
Exactly what was tested
HTRflow (their own pipeline): Riksarkivet's official htrflow package — region segmentation (yolov9-regions-1), lines within regions (yolov9-lines-within-regions-1), and their ReadingOrderMarginalia ordering step. Run unchanged, exported to PAGE-XML.
Column-clustering (ours): a fast YOLO line detector plus a learning-free geometric step — group lines into columns by their x-position, read each column top-to-bottom, columns left-to-right. No model, no training.
Data: 29 pages from Riksarkivet's own open datasets on Hugging Face — gota_hovratt_seg and trolldomskommissionen_seg. Reading-order τ is the Kendall correlation between each system's order and the archive's ground-truth document order, on the matched lines (1.0 = perfect, ~0 = columns swapped).
Act II · real historical tables
Does the same idea survive on real tables?
The court pages are running text in columns. Tables are the harder case, so I ran the same detect-then-order method — unchanged — on 200 real handwritten table pages: the 1879 log of the USS Jeannette Arctic expedition (HisClima, openly licensed, ground-truth row/column cells). Three honest outcomes.
What the tables teach — and the one thing still open
The detector generalises. The same geometric signal that separates columns from tables fired correctly on every one of the 200 real pages — running text fills its column, table cells stay short — so no page was ordered in the wrong mode.
Within a row, columns are solved. Given a row, the method reads its cells left-to-right in exactly the ground-truth column order on 100% of pages. The column problem that breaks the court records does not come back here.
The open problem is row segmentation. Three independent row-recovery strategies all plateau at τ ≈ 0.69, because on these wide handwritten pages 86% of neighbouring rows overlap vertically — a tall cell in one row dips into the next. No horizontal projection can separate rows that share the same height band; that needs the ruled-grid or the text itself, not geometry alone. It is stated as an open problem, not hidden — and it is exactly the kind of question worth doing properly with an archive's own material.
Act III · the full benchmark
Nine systems on 198 real tables
Every system run on all 198 HisClima pages — including Microsoft's Table Transformer, PaddleOCR's table recognition, and Riksarkivet's own htrflow. Reading-order τ is scored only where a system actually found the lines (≥15 matched, ≥30% coverage); below that its detection failed and no order can be judged.
| System | Reading-order τ | Detection · line coverage | Pages scored |
|---|---|---|---|
| PP-StructureV3PaddleOCR · plain OCR order | 0.88 but on the 34% of lines it finds · same lines: ours 0.85 |
F 0.34 · 34% coverage | 165 / 198 |
| Ours detect → row-majorlearning-free geometric | 0.69 every line, full coverage |
— · 100% (all lines) | 198 / 198 |
| Table TransformerMicrosoft · table structure | 0.68 SOTA table model — ties ours |
— · given GT lines | 198 / 198 |
| Naive top-to-bottomsort by y | 0.66 |
— · 100% (all lines) | 198 / 198 |
| PP-StructureV3PaddleOCR · table recognition | 0.64 cell-grid order — below ours |
— · given GT lines | 127 / 198 |
| Column-majorwrong mode — what the detector avoids | 0.21 |
— · 100% (all lines) | 198 / 198 |
| krakenblla baseline segmentation | few lines found | F 0.13 · 9% coverage | 0 / 198 |
| Tesseractlayout OCR | few lines found | F 0.03 · 2% coverage | 0 / 198 |
| htrflowRiksarkivet's own pipeline | few lines found | F 0.01 · 0.6% coverage | 0 / 198 |
What the full benchmark says
The harder part is finding the lines, not ordering them. Three production detectors — including Riksarkivet's own htrflow — find fewer than a tenth of the lines on these dense handwritten tables (htrflow: 0.6%). Without the lines, there is nothing to put in order.
Given the lines, the simple step holds up. Among methods that order every line, a small row-by-row step (0.69) is level with Microsoft's Table Transformer (0.68) and a little above naive top-to-bottom (0.66) and PaddleOCR's table recognition (0.64) — much heavier models it stays close to.
The ceiling is shared and real. Ours, Table Transformer and naive all plateau at τ ≈ 0.68 because 86% of neighbouring rows overlap vertically on these wide handwritten pages. No geometric or structural model escapes it — a genuine open problem, not a weakness of any one method.
198 pages · HisClima (USS Jeannette Arctic log, 1879–81, CC-BY) · reading-order τ = Kendall correlation with the archive's row/column ground truth, on matched lines · detection F and coverage at IoU ≥ 0.5 · validity guard: τ reported only for ≥15 matched lines and ≥30% coverage.
Act IV · system by system
Every system, on every one of the 198 pages
Pick a system, slide through all 198 pages. Green = the lines it found; red = the order it would read them in. Where a system found almost nothing, its detection failed — you can watch the page left bare.