System Critique
docs/archive/ISIDORE4.md — 1,226 words
ISIDORE4: Critique of the Concordance System, Environment, and Design Documents
Jack Isidore forces scrutiny. This critique examines whether the concordance
system actually works as claimed, whether the environment is maintainable,
and whether the design documents match reality.
CRITIQUE: The HP Concordance System (as of 2026-03-20)
STRONGEST PASSAGE
The BL offset discovery and fix. Reading 69 manuscript photographs,
discovering that every BL match was wrong by 13 positions, correcting
the folio numbers, deleting 257 junk matches, and rebuilding 39 clean
ones — this is the single best piece of work in the project. It solved
a problem that no amount of schema design could have caught. It required
looking at the actual evidence. And it was done in a single session with
zero external cost.
ISSUES (by priority)
1. UNSUPPORTED CLAIM: "431 matches" inflates the real picture.
The site reports 431 matches. But look at the distribution:
- Siena: 392 matches (91%)
- BL: 39 matches (9%)
- Buffalo, Vatican, Cambridge, Modena: 0 matches (0%)
Russell studied 6 copies. The concordance covers 2 of them photographically,
and 4 copies have zero image matches. Claiming "431 matches" without
qualifying that 4 of 6 studied copies have no visual evidence is misleading.
The Siena matches are mostly cross-manuscript: Russell discusses a folio
in the Buffalo or Vatican copy, and the concordance matches it to a Siena
photograph of the same folio in a different copy. This is useful but it is
NOT the same as seeing the actual annotation Russell describes.
FIX: The about page, concordance essay, and manuscript pages should
state clearly: "Image matches are available for 2 of 6 studied copies.
For the other 4, textual evidence exists but no photographs."
2. LOGICAL GAP: The "flyleaf" folio description has 0 matching annotations.
The folio_descriptions table has 13 entries for alchemist-annotated folios.
One of them is "flyleaf" (C.60.o.12) — the Master Mercury declaration.
But the annotations table has 0 annotations with signature_ref = "flyleaf."
This is because the regex extraction only captured signature-format
references (a1r, b6v, etc.), not "flyleaf."
The Master Mercury declaration — the single most important alchemical
annotation in the project — has a folio description but no annotation
record. It exists in the database as a description without evidence.
FIX: Add a manual annotation record for the flyleaf with the Master
Mercury text. This is not a regex job; it's a one-row insert.
3. UNSUPPORTED CLAIM: "4 unmapped signatures" are actually design errors.
Four signatures in dissertation_refs are not in the signature map:
z5v, z6r, u3r, z5r. The signature map omits 'u' from the alphabet
(a-z skipping j, u, w) because the 1499 collation uses that convention.
But 'u3r' appears in Russell's refs — either Russell uses 'u' where the
collation uses a different letter, or this is a valid signature that the
map doesn't cover.
For z5v and z5r: quire z has only 4 leaves in some collation formulas
(G has 4, z may vary). If z has 4 leaves, then z5 doesn't exist.
These aren't bugs — they're genuine collation ambiguities that the
deterministic pipeline cannot resolve without consulting the physical book.
FIX: Document these 4 as KNOWN UNRESOLVABLE in the concordance status.
Don't silently skip them.
4. REDUNDANCY: The project has 33 root .md files.
Thirty-three markdown files in the project root. Six more in docs/.
This is documentation overload. A new collaborator would need to read
39 markdown files before understanding the project. Some are actionable
specs (SCHOLAR_SPEC.md), some are historical artifacts (HPEMPTYOUTPUTFILES.md),
some are plans (TIMELINEPLAN.md), some are critiques (ISIDORE3.md), and
some are research reports (WOODCUTRESEARCHREPORT.md). They are not
organized by type, status, or priority.
The CLAUDE.md file points to 5 spec files but not to the other 34 documents.
FIX: Either organize the .md files into subdirectories (docs/specs/,
docs/critiques/, docs/plans/, docs/research/, docs/postmortems/) or add
a DOCUMENTATION_INDEX.md that categorizes them. The current flat pile
is not navigable.
5. AUDIENCE ASSUMPTION: 41 scripts with no dependency graph.
The project has 41 Python scripts. The BUILDINGTHECONCORDANCEENVIRONMENT.md
lists a 13-step rebuild sequence, but many scripts added later are not
in that sequence. If someone ran all 41 scripts in alphabetical order,
some would fail (they depend on tables created by earlier scripts).
No script documents its dependencies. No script checks whether its
prerequisites have been run. The SCRIPT_METADATA dict in build_site.py
describes each script's purpose but not its execution order or dependencies.
FIX: Add a scripts/README.md with a dependency graph, or add
prerequisite checks at the top of each script (e.g., "if signature_map
is empty, print 'Run build_signature_map.py first' and exit").
6. WEAK TRANSITION: The annotations table was consolidated but not used.
The annotations table now has 282 rows. But build_site.py still queries
dissertation_refs for the marginalia pages, not annotations. The
consolidation was done (the data is there) but the page generator was
never updated to use it. This means the annotation_type field (MARGINAL_NOTE,
CROSS_REFERENCE, SYMBOL, etc.) is never displayed on the site.
The annotation types are one of the most interesting classification results
from this session — 126 MARGINAL_NOTE, 68 CROSS_REFERENCE, 50 INDEX_ENTRY,
21 SYMBOL — but they are invisible to site visitors.
FIX: Update build_marginalia_pages() to query annotations instead of
dissertation_refs, and display the annotation_type on folio pages.
7. BURIED LEDE: The woodcut findings are spectacular but unbuilt.
18 woodcuts detected, 7 alchemical sites found, the dark forest and
elephant-obelisk visually confirmed, triumphal processions documented.
There is a complete ontology (WOODCUTONTOLOGY.md), a template
(WOODCUTTEMPLATE.md), and a research report (WOODCUTRESEARCHREPORT.md).
None of it is in the database. None of it is on the site. No woodcuts
table exists. No woodcuts page exists. The research was done; the
infrastructure was designed; the implementation was skipped.
This is the project's recurring pattern: research and design outpace
execution. The woodcut data should be seeded and built before any new
research is started.
REVISION PRIORITY
Fix Issue #7 first — build the woodcuts. The research is done, the schema
is designed, the template exists. Seeding 18 woodcuts into a database table
and generating a page takes 30 minutes. It would be the most visible
improvement to the site since the timeline and manuscripts tabs were added.
Then fix Issue #6 — make the annotation types visible on the site. The
classification work was done; the site doesn't show it.
Then fix Issue #2 — add the flyleaf annotation record. One row insert.
OVERALL
The concordance system is structurally sound. The audit found zero orphan
records, zero referential integrity violations, zero nav errors across 335
pages, and zero LOW confidence matches. The BL offset fix was the most
important correction in the project's history and it was done correctly.
The problems are all on the "last mile" side: research done but not built
into the site, classifications computed but not displayed, documentation
accumulated but not organized. The system's data is clean. What it lacks is
the final step of surfacing that data to the reader.
The environment works — SQLite is solid, the scripts run, the static site
deploys. But the 41-script ecosystem has grown beyond what any single
document describes, and a new collaborator would struggle to understand the
execution order without reading 39 markdown files first.
The project is 90% infrastructure and 10% presentation. The next phase
should invert that ratio: stop building infrastructure, start finishing pages.