What is being done, as opposed to what is known
The work list
30 items open, 9 of them waiting on somebody outside this atlas. Every other page here answers what the map holds; this one answers what is being done to it, including the parts that are stuck.
1. Running — a machine or a person is at it now this atlas
Walk every FamilySearch collection's waypoint tree
Running now. 3,504 collections; the tree descends to the individual book. 707,471 volumes so far against 4,903 when the day started. The catalogue is gated but the records API is not, which an earlier note in `import-fs-catalogue.py` had written off without trying — the same mistake as declaring Scotland's parishes closed when data.gov.uk had them. Publishes every two hours while it runs. US collections are half the total and come last, so the rest of the world lands first.\nSupervised now: scripts/supervise.sh restarts it if it dies, bounded to five attempts.
2. Running — a machine or a person is at it now this atlas
Place the harvested volumes on the places they belong to
347,272 of 775,686 books land on a place. The rest name somewhere this atlas has never drawn, and they are written to `data/fs-volumes-unplaced.json` ranked by how many books are waiting — the map's next places, not a silent loss.
3. Done — finished and published this atlas
Cross-border collections: a register moves when a jurisdiction moves
Done 20 September. Of 1,165 Croatian places, none was reached by a collection filed under another country — the stated thesis of this atlas was dead. Two causes: `placeIds` were resolved through the wrong endpoint, which answered an unrelated place with HTTP 200 (Italy, Pola and Trieste came back as Monomie, New South Wales); and a title disagreeing with its places was treated as an error and the places discarded. Now ten explicit filing groups — Habsburg, Yugoslav, Prussian, Russian, Ottoman, Nordic, Low Countries, Iberia, these islands, the Alps — because records cross for historical reasons, not geographic ones. Verteneglio finds Brtonigla, Isola d'Istria finds Izola. Kept honest by `scripts/test-cross-border.py`.
4. Done — finished and published this atlas
A surname timeline for every country, not only France
Done 20 September. Where a name is found today and its variations were already global; WHEN it was borne existed for France alone. 363,574 surnames now carry a birth timeline from Wikidata, reaching the 1400s. Aggregated over the variant family and folded, so a rare spelling inherits its family's history — Defranceski has no notable namesake, De Franceschi has eleven, Italy from the 1550s and Croatia from the 1800s. The bias is stated on the page: these are people notable enough for an encyclopaedia, not a census.
5. Done — finished and published this atlas
Stop a name inheriting every country its archive touches
Done 20 September. Testing on David's own name found `De Franceschi` attested in 29 countries including Albania, Cuba, Paraguay and Zimbabwe — all from one Friulian family's archive. The fallback gave any unplaced name the archive's whole geography. Archives that place names individually do it for most of them, so silence in that list is information. The fallback survives only where an archive has one country and cannot mislead.
6. Done — finished and published this atlas
Antenati: the Ministry's own registers, on the map
Done 20 September. 5,591 registers harvested from the SAN open-data feed in September and read by nothing since — the file sat in `data/` while Italian places showed only FamilySearch. 4,159 are now on 72 comuni, 68 of which this atlas drew because a register pointed at them.
7. Done — finished and published this atlas
Scotland's civil parishes, searchable
Done 20 September. 871 parishes from data.gov.uk, harvested and then read by nothing. The civil parish is the unit every Scottish record is filed by — the OPRs, the statutory registers, every census — and typing one returned nothing at all.
8. Blocked — waiting on a person or an institution the Ministry
Resume the Antenati harvest at volume 6,000 of 464,770
Blocked on THEIR side, not ours. `www.san.beniculturali.it` is unreachable — port 80 refuses and the parent site times out. Stopped cleanly with its resumption token intact. Test with `curl "http://www.san.beniculturali.it/oaicat/OAIHandler?verb=Identify"` before retrying.
9. Done — finished and published this atlas
index.json is over its 800 KB budget
Done 21 September. Resolved by measuring the right unit. The budget counted bytes on disk; GitHub Pages gzips JSON about four to one, so an index failing at 1,084 KB is 271 KB over the wire — the number nobody has ever downloaded was the one failing the audit. Budgeting disk bytes creates pressure to delete real places to satisfy a figure no reader experiences. The budget is 400 KB transferred. A field `f` that nothing read came out too: 59 KB on 10,046 rows.
10. Blocked — waiting on a person or an institution David
Point recordatlas.org at GitHub Pages
Bought 17 September; the A records still point at the registrar's landing page. `scripts/cutover-domain.sh` does the whole switch and refuses to run until at least two of GitHub's four addresses appear, because a CNAME file and a base of `/` are not independently safe. Replace the A records with 185.199.108–111.153.
11. Blocked — waiting on a person or an institution David
AdSense answers unfilled
Everything this atlas controls is verified correct, live, on 21 September. The slot renders as <ins class="adsbygoogle" style="display:block"> with data-ad-client=ca-pub-4156851882993001 and data-ad-slot=1255898967, format horizontal, full-width-responsive; the adsbygoogle script loads; the push call is present; and ads.txt at daviddef.github.io answers 200 with the matching publisher line. The earlier faults are both gone — the slot was a flex child with zero width, which is what «No slot size» meant, and it is display:block now. So an unfilled slot from here is AdSense's own decision about approval or inventory, not a mistake in the page. Only David can see the AdSense dashboard.
12. Next — ready to start David to send
Ask the Meertens Instituut about the Familienamenbank
DRAFTED, not sent — docs/letter-meertens.md. It asks for surname distributions at surname/region/count grain, offers credit and whatever licence they set, says plainly that the site carries advertising in case that makes the use commercial in their terms, and says a clear no is itself useful because the atlas records why each country is missing. Only David can send it.
13. Blocked — waiting on a person or an institution David
Waiting on NARA and Trove
NARA's API key needs a human to request it. Trove replied with ticket RSref189042 and nothing since.
14. Done — finished and published this atlas
The site is 844 MB of GitHub Pages' 1,024 MB
Done 21 September. The surname dumps were a sixth of the site and are gzipped now, 165 MB to 17. places-full.json was 143 MB imported by nine pages, of which 135 MB duplicated the per-place detail files — slimmed to 6.9 MB, which also fixed a build that had started dying out of memory. 815 MB of 1,024 and tracked by scripts/audit-site.py.
15. Done — finished and published this atlas
Run the whole build before pushing, not one stage of it
Done 20 September. Three CI failures in one afternoon, all from running `npm run data` and calling it tested. One of them — `row is not defined` — was sitting in my own local log unread. The gates work; skipping them does not.
16. Done — finished and published this atlas
Put every harvested dataset onto a page
Done 21 September. Three datasets had been harvested and read by nothing. Antenati: 5,591 registers from the Ministry's open data, 4,159 now on 72 comuni. Scotland: 871 civil parishes, the unit every Scottish record is filed by, and typing one returned nothing at all. And this work list, which the estate keeps one of per archive and this atlas had none of.
17. Done — finished and published this atlas
David's own surname returned 404
Done 21 September. A surname got a page only with a CURATED variant, so 564 names a family archive had actually placed in a country had none — Defranceski among them, attested in nine countries with every variant phonetic. And the birth timeline was null on every page: the module resolved its data file relative to import.meta.url, Vite bundles it elsewhere, the read threw and a bare catch swallowed it. A silent catch around a path is how a whole feature disappears with nothing failing.
18. Done — finished and published this atlas
A hidden tab never wrote the URL
Done 21 September. Reported by the archive-kit session. URL writes coalesced on requestAnimationFrame alone, and a browser runs no frames for a hidden tab — so somebody who switched away, changed the year and came back had an address bar describing the map they left. I had seen this myself while testing the record-kind filter and written it off as an automation artefact.
19. Running — a machine or a person is at it now this atlas
Place the remaining unplaced books — 799,897 across 46,345 names
Measured 21 September: 1,571,627 volumes are placed, across 7,778 places — L'Aquila 72,724, Avellino 62,558, Bergamo 55,242. What is still unplaced is 870,018 volumes under 31,530 names. The largest single cause found so far is that an unplaced name carries its COLLECTION's country and not its own: 168 Friulian villages sat under AT holding 32,579 books, because the collection is the Austrian Küstenland. The exonym harvest now searches the historical filing groups, and the unplaced list is no longer cut at 4,000 names, which had been hiding 200,641 books from the only thing that looks for them.\n21 September, measured: 2,187,438 volumes placed on 9,910 places; 799,897 volumes across 46,345 names still unplaced, and the list is now written out in full rather than cut at 4,000. Three systematic causes found in the top of that list and fixed in the matcher: «不明» is Japanese for unknown and held 2,516 volumes that no gazetteer will ever match; «Zona de reclutamiento de Madrid» is 5,718 volumes of Madrid behind an administrative wrapper; «Auckland Court» and «Christchurch Court» are a thousand volumes each standing in cities this atlas has drawn. Unwrapped readings are tried last, never first, because put first they made «Hampton Court» resolve to Hampton.
20. Running — a machine or a person is at it now the holders, and David for the Netherlands
Close the countries with no surname register — was eight, now four and a half
Italy, Denmark and Belgium closed; Croatia, Sweden and Finland rewritten — all on 22 September, and all by a reader rather than this project. David sent the Croatian statistics office's own name application, which answers. Checking the rest the same way found the identical mistake under Denmark, whose entry proudly recorded that all 2,308 tables in the statistics bank had been searched for «navn» — they had been; Denmark publishes on its website. Then Belgium, which had been rechecked and confirmed that same morning by reading the open-data category directly: Statbel keeps its surnames on the family-names theme page. FOUR ENTRIES, ONE MISTAKE: search a machine-readable index exhaustively, then make a claim about an institution. The pattern is the finding. Belgium is also the best surname data here — per municipality, 565 communes, 106,869 names, CC BY 4.0, which is surname to place to count and nothing else in the directory has that grain. Left: Czechia (possible and refused — the ministry's own record says both, so it needs a formal request, not a script), the Netherlands (Meertens, needs the letter), Finland (one manual download from a portal that 403s us), Sweden (a lookup and a top-100 table, no list published anywhere). Croatia is half closed: 141 surnames the atlas already attests, asked one at a time.
21. Done — finished and published this atlas
The Kosina test: a Slovene parish, a German exonym, and an s/z spelling
David's Blazevic research has a Martin Kozina born about 1841 in «Kranjska župa sv. Martin kod Krainburg» — Carniola, the parish of St Martin near Kranj, whose German name is Krainburg. Driven against the live map on 21 September: Kranj, Krainburg, Kranjska and Carniola all frame Carniola at 46.348,14.067; Kozina lands at 45.607,13.943 and «Kosina» lands on it too, through the spelling tier; Šmartno and Smartno both reach 46.189,14.939. One name failed and is now fixed: «Krain» flew to Ukraine, because "ukraine" contains "krain" — a country is matched on whole words now. What the map cannot do is show his register: FamilySearch publishes 2 collections for Slovenia and the waypoint walk finds Kranj zero times. Those books are on Matricula and the Slovenian archive portals, which this atlas does not scrape, so Kranj's page names the archives that hold them instead of pretending to the books.
22. Done — finished and published this atlas
Make surname pages lean enough to afford 16,138 of them
Done. 16,135 surnames earn a page and all 16,135 are live and in the sitemap. The row was blocked for weeks on two numbers that were both wrong: 24.6 KB a page (measured off Defranceschi, which carries eleven country blocks when the median surname carries one) and «985 MB of 1,024» (the RAW build tree, when the limit applies to the artifact GitHub deploys). Admitting the class took the artifact from 158 MB to 164 MB. Twelve thousand pages cost six megabytes of what actually ships. Two cuts paid for part of it — nearest-archive listing and class-based style scoping — and CI now prints the weight on every build so no decision here is ever made against a guess again.
23. Done — finished and published this atlas
A showcase of the family archives at the foot of the map
Live at the foot of the home page as «Eight families, the same method», all eight archives including Luwinski. Two errors found afterwards by checking the cards against the data behind them and both fixed: Booyzen printed the same name twice and omitted Booyens, which is the form its archive actually records, and Luwinski listed Sauerbaum among the places. audit-site.py now checks the cards every run.
24. Done — finished and published this atlas
The Otočac question: can the map see a place indexed across two countries?
David asked whether the atlas would pick up Otočac being indexed by two collections across a border — Church Books 1839-46 and Austria Küstenland 1848-58. Traced end to end on 21 September. Half of it works and the other half cannot. Otočac's page does carry its FamilySearch collections and the cross-border matcher does cross borders where there is something to cross with — IT to HR 9 places, IT to SI 16, AT to DE 2, with no false positives. But the Austrian side of David's example is not reachable by this method at all: «Austria, Catholic Church Records, 1565-2007» (collection 5000002) has been walked and is recorded in the harvest's own indexOnly list. It publishes no browsable volumes through the open waypoint API, so its tree names no places, Croatian or Austrian. The Küstenland attribution David saw is in the FamilySearch CATALOGUE, which is gated, and this atlas does not climb gates. So the honest position is: the map shows Otočac's Croatian books and names the archives for the rest, and no amount of better matching will reach the Austrian volumes while they are only in the catalogue.
25. Done — finished and published this atlas
Stop measuring the site against the wrong limit
audit-site.py reported «the deployable site is 1418 MB and GitHub Pages publishes at most 1,024 — this will not deploy» about a build that deployed successfully minutes later in the same CI run. It was adding up raw bytes in dist/ and comparing them to a limit that applies to the COMPRESSED artifact, which GitHub stored at 189 MB. A 7.5:1 ratio, on a site that is almost all HTML and JSON. That one wrong comparison is why 16,138 surname pages were refused as unaffordable for weeks. The check now estimates the compressed size, sampled per file extension with a seeded sample so builds stay comparable, and prints the raw figure beside it so a local `du` does not look like a contradiction.
26. Done — finished and published this atlas
The map waited on a fetch a quarter of which said «nothing here»
Done 22 September. index.json is the one file the first paint waits on and it had reached 533 KB over the wire. There is no fat in the rows — 28 bytes each, and dropping every derivable id saves 3% because gzip already collapses that repetition. The honest saving was not in the rows but in which rows: 4,932 of the places carry no volume and no collection. They are real ground, correctly placed, nobody has walked them, and the difference between «not surveyed» and «nothing here» is one of this atlas's promises — so they stay on the map. They just stop being something the map waits for. 409 KB draws the atlas; 124 KB fills in the ground behind it a moment later. The split's real cost was everywhere else: ten static pages imported index.json and would have silently dropped those places — no `/place/<id>/` page for any of them — so they now import the index whole from `lib/atlas.js`. The audit's integrity checks, the deployed-site comparison and the unread-field gate were all reading half a file and believing it was the whole one.
27. Next — ready to start this atlas
Draw the migration a surname made, not just where it ended up
David's, 22 September. The surname page says where a name is borne and where a record names it, as two lists. It does not say that the one became the other. Blažević on this map is a column of countries; what a reader wants to see is Croatia to Germany, Croatia to the United States, with the weight of each line. The data for it is partly here already — archive-surnames.json carries each archive's declared country against every country its research reaches, which is a claim about movement — and partly not: nothing in this atlas yet holds «N families of this name left HERE for THERE». Acta Croatica has exactly that for Croatia and cannot be ingested (no licence, unstated derivation — see data/_declined.json), so the honest version is built from emigration sources this atlas can cite, not from arrows we felt like drawing. An arrow is a strong claim. Drawn from a guess it is the most confident-looking lie a map can tell.
28. Next — ready to start this atlas
Make every surname a page worth citing: spellings, meaning, history
David's, 22 September, and the largest thing on this list. The aim is for Record Atlas to become a source people cite for names, by drawing pictures no other site can — which it can only earn by being right, one name at a time. Three parts, and they are not equally hard. SPELLINGS is nearly solvable now: the atlas already links s/z and c/k alternations between two independently attested names, and Acta Croatica's Horvat page (Horvath, Horwath, Thorvat, Horváth, Horwat, Horvát, Thorwat, Horvatt, Horvaht) shows the standard to reach. MEANING and HISTORY are research per name, not a harvest — and the moment a page asserts an etymology it cannot cite, it stops being citable itself. So: no generated etymologies, no plausible guesses. Each one carries its source or the page says nothing. 16,138 surname pages at that standard is years of work, which is the right answer rather than a reason not to start: begin with the names the family archives actually turn on.
PROGRESS, 23 Sep: the surname page now draws the one picture no other surname site can. Every such site can tell you a name is common in Croatia; this one also holds 40 record regions with an era table naming the polity and the archive its paper went to, and the two are halves of one question — my people were called this, they were there, so who has the paper, and for which century. A Blažević page now lists the seven pieces of Croatian ground this atlas has drawn, the Military Frontier among them, each linking to its own timeline. It is careful about what it is not: a register counting a name in a country says nothing about which village, and a region may cover only a corner of a large country — Acadia reaches into Maine and is not a statement about the United States. Spellings, meaning and history are still the rest of this row.
29. Done — finished and published this atlas
Denmark's 294,496 surnames are harvested and will not build
Done 22 September. 294,496 Danish surnames, CC BY 4.0, now in the corpus — verified by a full local build before pushing: 41,263 pages, 19,779 surname pages, peak RSS 4.6 GB against a 10 GB ceiling. It took three wrong diagnoses. First GitHub Pages' 1,024 MB, which the deploy's own report already answered — 385 MB stored. Then the heap ceiling, raised twice. The stack trace named the real one: «Zone Allocation failed» inside FastJsonStringifier, 55 seconds in, heap at 1.4 GB. Four build entrypoints imported surnames.json, so Vite inlined 152 MB into a module's source text, in a compiler arena --max-old-space-size does not govern. Reading it from disk fixed it in one change. Every wrong guess here was reasoning where a measurement was available: the size was in the build log, and the cause was in the stack trace.
30. Done — finished and published this atlas
France had four providers, and one of them was a link to Geneanet
Done 22 September. David asked whether the atlas had mapped Geneanet's collection catalogue. It had not, and the catalogue cannot be read: it sits behind a Cloudflare managed challenge whose own page carries «noindex, nofollow», and this project does not climb bot gates. But the question exposed the gap behind it. The atlas held four providers covering France — Bas-Rhin, Haut-Rhin, Moselle and a link to Geneanet itself — for a country with one of the deepest register series in Europe. Geneanet is largely an aggregator OF the archives départementales, so the answer was to go to them. 96 of them now, from Wikidata, each with a checked https address and coordinates a median 1.3 km from its own town. Every one is access «unsurveyed», because nobody here has opened 96 archive sites to see what is free and what is not, and marking them «free» because departmental archives usually are would be the map asserting 96 things nobody verified. What was refused is the useful part: two hosts that no longer resolve at all were dropped rather than shipped, which is the ARHiNET lesson from the same morning; two abolished archives — Seine-et-Oise and the Archives de la Seine, départements dissolved in 1968 — were caught by P576, the filter the US county harvest already needed; and six archives alive only on http are held back, because this project's data gate requires https and relaxing a rule to admit one's own data is the wrong way round. Those six need their current address found by hand. Next: 555 archives communales are on Wikidata too, and are a larger and much more variable question.
31. Done — finished and published this atlas
Belgium hands us surname-by-town, and now the map draws it
Done 22 September. All 565 Statbel communes matched to coordinates through Wikidata's P1567 — an exact code join, not a name join, which matters when every Belgian commune is spelled twice and there are a dozen Sint-Martens- to pick between. Wikidata holds 2,981 Belgian NIS codes and most are deelgemeenten with a letter suffix; the five-digit municipal ones match Statbel exactly, 565 of 565. A surname page for a Belgian name now draws the communes: Peeters is 29,888 people across 411 of them, 2,060 in Antwerpen, and the picture shows it clustering in Flanders. The caption says it is the only country that can be drawn this way, because a reader seeing Belgium detailed and France blank would otherwise read it as a fact about the name rather than about what states publish.
32. Done — finished and published this atlas
The sitemap 404s that were never there
Not a defect, and recorded because believing it was cost real time. The audit printed «45,829 urls listed, 45,782 pages built» for weeks. I read the gap as 47 urls pointing at nothing, wrote it into this list as «handed to search engines that answer 404», and reported it to David as a defect. `listed` is every url in every sitemap; `built` counted only the five kinds that check knows how to walk, so /about/, /surnames/, /coverage/ and the paginated /places/N/ were on one side and not the other. Two definitions, not two numbers. Verified independently: 45,409 listed, 0 without a page. The note now compares like with like.
33. Blocked — waiting on a person or an institution David
Poland's scanned metrical books are behind a Vaadin app
David's, 22 September. Genealogia w Archiwach — genealogiawarchiwach.pl — was not here, and it should be: scanned civil registration from the Prussian partition, parish duplicates before 1874, population books from 1837, free to download. It has a provider row now. Its holdings are not harvestable as things stand. Every url returns the same Vaadin shell, robots.txt and sitemap.xml included, and the data moves over a stateful UIDL protocol — there is no crawlable surface to read, which is a different thing from the bot gates this project declines to climb. The route, if there is one, is the Polish archives' own catalogue rather than this front end: szukajwarchiwach.gov.pl is already a provider here and its bulk harvest was declined as a bot gate, so both doors are shut for now. Worth asking NDAP directly whether the fonds list is published anywhere machine-readable — that is an email, like Meertens, not a harvester.
23 September: the letter is drafted — docs/letter-ndap.md, in Polish and in English, for David to send. Two things were checked first so that it does not ask for something already public. dane.gov.pl, the national open data portal, has no Naczelna Dyrekcja Archiwów Państwowych institution and no state-archives dataset at all. And szukajwarchiwach.gov.pl serves an Imperva JavaScript challenge for `robots.txt` ITSELF, marked noindex,nofollow — so we cannot read the archives' own crawling policy, never mind crawl. That is worth stating in the letter rather than hiding: we stopped at the door, and said so.
What it asks for is the inventory and not the scans — which fonds exist, which archive holds each, which place and which years. That is the layer that turns «Poland» into «write to the State Archive in Poznań for this parish, these years», and it is what this atlas is made of.
34. Done — finished and published this atlas
Five thousand archives on the map that nobody could click
Done 22 September. data/archives-wikidata.json held 4,929 archives and not one website — every row an empty `sites` list, because the file came from the generic bounding-box harvester written for cemeteries, which selects a label and a coordinate and nothing else. For an atlas whose premise is «here is the paper and here is who holds it», five thousand dots a reader can see and cannot open is the least useful thing it draws. The client had rendered `sites[0]` as a link all along; the field was simply never filled. Re-harvested worldwide with P856 and a transitive class net: 10,786 archives across 156 countries, 5,276 of them with a website. Germany goes from 541 to 1,069. Dissolved institutions excluded on P576, the filter the French archives and the US counties both needed. Still unverified and still drawn apart from the curated providers, because nobody here has opened them.
35. Blocked — waiting on a person or an institution David
Argentina publishes surnames by province and will not serve them to us
The best find of the portal sweep. RENAPER publishes «Distribución de apellidos en Argentina» under CC BY 4.0 — three CSVs, one of them surnames BY PROVINCE, which is the grain that makes Belgium the richest thing in data/frequencies/. Argentina took a great deal of European emigration and this atlas already holds places there. datosabiertos.renaper.gob.ar resolves and accepts a connection on 443, then never completes the TLS handshake — at 60 seconds and at 180. http 301s to the https that does not answer. Nothing said no; it simply does not reply to us, which may be geography or may be a bad afternoon. Two ways through: retry it in a week, or one person with a working connection fetches three files — apellidos_mas_frecuentes_pais.csv, apellidos_mas_frecuentes_provincia.csv and apellidos_cantidad_personas_provincia.csv. The harvester for Belgium is the template.
36. Running — a machine or a person is at it now this atlas
Sweep open-data portals rather than guess a country an afternoon
Eight countries were checked one at a time and four of those verdicts were wrong in a single day. scripts/sweep-surname-portals.py asks nineteen national catalogues at once, each in its own language — the word for «surname», not the English one, because missing that is how a sweep quietly finds nothing. First run: Ireland's VSA110, which closed Ireland, and Argentina's RENAPER distribution, which is blocked on its host. Nine portals answered nothing and five errored — 403 from Mexico, 401 from Brazil, a dead TLS version from Chile, a 404 from Australia's api path. Those five are the next work: an error is not an answer, and «the catalogue found nothing» is the exact sentence that was wrong four times, so the output file says as much in those words. It is a FINDER, not a verdict — every hit still needs somebody to open it.
37. Running — a machine or a person is at it now this atlas
Test the thing end to end, because nothing tests the client
Partly done 22 September — unit layer in, browser layer not. 22 tests on Node's own runner, no dependency to install, wired into `npm run check` so they gate the build. They pin where the real bugs were: the box order read as x,y pairs, France's frame dragged to Réunion, the clamped height that squashed Belgium, the padding that put a reader inside a Spanish town, and «three countries with no count must not earn a page». Writing them found two things immediately: atlas.js could not load outside a bundler, and `node --test test/` dies on a directory in Node 26 — which would have failed on CI's first run. STILL MISSING: the browser layer. Playwright over the built site — search a place, search a surname, click a dot, set a year, a phone viewport — is the half that would have caught every bug David found by looking. It needs a browser download in CI and its own job, not a slot in a build that already takes forty minutes.
38. Done — finished and published this atlas
Germany has 1,069 archives on the map and not one provider
Done 22 September. 1 provider to 35: the Landesarchive, Staatsarchive and Landeskirchliche Archive. German civil registration begins in 1876 and the parish registers before it are largely with the church archives, so which of the two holds a record depends on its date and both are now in. Twelve did not answer and are not in. Done with scripts/promote-archives.py, which is general: a country, a name pattern, every URL opened.
39. Done — finished and published this atlas
Five open-data portals errored rather than answered
Done 22 September. Three of the five were this project's fault, not the portal's: Australia had moved its API under /data/, Colombia is Socrata rather than CKAN and wanted a different path and result shape, and Chile answers curl but not urllib, whose default context refuses the TLS version it offers. All three answer now and none of them publishes surnames — which is a verdict, where before there were three errors being read as one. Chile could have been forced with a lowered SECLEVEL and was not: weakening a TLS client to read a public catalogue is a bad trade, so it was read by hand instead. Mexico still returns 403 to every client and Brazil 401 without an API key. Those two remain UNANSWERED, which is not the same as answered no, and the file says so.
40. Next — ready to start this atlas
Survey the 204 French archives nobody has opened
France went from 4 providers to 204 in a day and every new row says access «unsurveyed», which is honest and is not the end of the job. Most départementales put their registers online free; some charge, some are index-only, a few have nothing. Until somebody looks, the map cannot colour them, and colour is the whole legend. Sample twenty by hand first to find out whether the answer is usually the same, because if it is, the remaining 184 are a much smaller job.
41. Running — a machine or a person is at it now this atlas
Place the 799,897 books still without a place
Long-running, and the number that most changes what this atlas is worth. Every one is a volume that exists, is catalogued, and has no dot — so a reader whose parish is in that pile is told there is nothing. Continues under the matcher; see row 19.
42. Done — finished and published this atlas
The provider `checked` dates will start ageing out
Done 22 September. scripts/recheck-providers.py takes the N oldest, opens them and writes the date back; 120 done in the first run and one moved (Archion). A link that stops resolving is reported and NOT dropped — removing an institution is somebody's decision, not a script's, and ARHiNET is why the discipline exists.
43. Done — finished and published this atlas
Nothing tells a reader when the atlas last changed
Every place page now says when it last changed. The footer's build date could not do this: it moves whenever anything anywhere in the atlas moves, so it tells a reader nothing about THEIR parish.
Fingerprinted on what a reader would notice at that place — its collections, volumes, names, parishes and burials — and deliberately not on the build date or on counts derived elsewhere, or every place would «change» on every build and the line would be noise. Checked in isolation before shipping: a rebuild with identical content does not move the date, adding a field the reader cannot see does not move it, and a volume arriving does.
IT REFUSES TO OVERCLAIM. Until a place has been seen to change, first and last are both the day tracking began, and «not seen to change since 23 September» is a different sentence from «changed on 23 September». The second would be a small lie renewing itself on every build, so the page says the first.
data/_place-history.json is committed on purpose — the whole value is that it remembers across builds, and CI regenerating it from nothing would stamp today onto all 21,473 places every run.
44. Running — a machine or a person is at it now this atlas
Collect more content: depth where the atlas is already deep
David's, 22 September. Breadth is the obvious gap and depth is the quieter one. Croatia, Istria and Carnia are the ground this project knows best and they are still thin below the parish: no notarial records, no land registers, no school or conscription lists, and those are where a family appears when the church books stop. The Antenati harvest (row 8) is 6,000 of 464,770 volumes and blocked on the Ministry's own server. Matricula is NOT an option and the earlier note here was wrong to imply it: it covers Austrian, German and Polish parishes and its images and descriptive metadata are CC BY-NC-ND. This site carries advertising, so republishing its inventory is exactly what that licence refuses. It is linked, correctly, and will stay linked — see data/_declined.json. Depth here has to come from sources that permit it.
45. Running — a machine or a person is at it now this atlas
Collect more content: breadth across the 199 countries with nothing
David's, 22 September. 199 of 237 countries have no curated provider. Mexico, Brazil and the Philippines got their first twelve today, which leaves the rest. The order that matters is places-already-on-the-map: Colombia 143, Guatemala 122, Chile 118, the Dominican Republic 109, Honduras 88, Turkey 81, India 73 — every one of them a country where this atlas holds ground and cannot name an archive to write to. Partly done 22 September, and a line drawn. 42 countries gained their first provider from the national-archive layer, taking the directory to 506 across 84. The obvious next step — bulk-promoting every archive the Wikidata layer holds for a country with none — was considered and REFUSED. Indonesia has 158, Israel 128, Japan 80, Georgia 47, and they are archives in Wikidata's sense: corporate record offices, film archives, museum collections. Promoting them would move them from a layer that says «unchecked, and not all genealogical» to the diamonds that mean somebody looked, and the count of curated providers would stop meaning anything. They are already on the map, already clickable since the websites arrived, and already honestly labelled. What the next country needs is the French and German treatment: a name pattern that picks the institutions that hold civil or parish registers, and every URL opened. scripts/promote-archives.py does that and takes about twenty minutes a country.
46. Done — finished and published this atlas
The Others layer has 5,276 links nobody knows are there
Done 22 September. The label now counts them: «10,786 mapped by others, 5,276 with a website · unchecked, and not all genealogical». A layer worth turning on should say what turning it on gets you.
47. Done — finished and published this atlas
A surname page for a name with nothing to say
Done, and NOT by relaxing the page rule, which is what this row suggested and would have been the wrong answer. 946,575 names against 23,930 pages means the fix as written is 922,645 new pages built out of one register row each — the textbook definition of a doorway page, on a build that already takes twenty-three minutes for 46,000 and ships into a one-gigabyte ceiling.
Instead: ONE page, /surname/, that answers for every name in the corpus. It reads the same sharded register the map's panel reads — s/<first three folded characters>.json — so there is no new data, no build cost, and a name added to the corpus is answerable the moment it lands without a page being generated for it. Exact match first, then names beginning with what was typed, then names containing it; never the whole shard, which for «bla» is 1,853 names and answers nothing.
The map's panel now links there for the thirty-nine names in forty that have no page, where it used to stop dead. And a name no register has counted is told so plainly, with the point that matters: that is a statement about this atlas's registers, not about the name. Several states publish no surname table at all and a few suppress small counts, so absence here is not absence in the world.
48. Done — finished and published this atlas
Measure the first paint on a real connection
Done 22 September, and the answer was not the file size. Measured live: first contentful paint 4,500 ms, and index.json was not REQUESTED until 4,821 ms because the map is built inside window.load, which waits for every image, font and CDN script. The 533→419 KB split saved a hundred kilobytes on a request that had not been made. Fetch now starts at parse time with a preload; locally index.json completes at 1,472 ms, before DOMContentLoaded. A budget is not a measurement.
49. Blocked — waiting on a person or an institution David
The atlas cannot be corrected by the people who would know
The page and the plumbing are done: set `contact.email` in site/src/data/site.json and /add/ grows a plain «or just write» route with no account and no template. Left empty deliberately — publishing a personal address on a public site is the owner's decision, and a contact link that goes nowhere is worse than none. A forwarding address would do; it does not have to be a personal one.
50. Done — finished and published this atlas
Cemeteries and churches are harvested and barely used
Done, and half of it turned out to be done already: the 108,054 churches were wired to place pages as «the nearest parishes», with the distance shown because eight kilometres to Pazin is the useful fact about Gračišće. The 48,482 cemeteries were not, and now are — the nearest five within twelve kilometres, deduplicated between Wikidata and OpenStreetMap on «same name within 300 m», each linking back to the source that knows it. The page says plainly what they are NOT: points on a map, nobody here has visited one, and whether the burial register survives is a question for the parish or the municipality above. Unnamed cemeteries stay out — a shape with no name is not a place anyone can ask about.
51. Running — a machine or a person is at it now this atlas
One accessibility pass, properly
Partly done 22 September. The panel rewrites itself on every answer and said nothing while doing it — aria-live=polite now, and the result count is its own status region. The map is labelled for what it is: 21,302 dots on one canvas cannot be tabbed through, and making them focusable would be 21,302 tab stops, so it says so and points at the search box and /places/, which reach the same answers in text. That is a fallback rather than an ARIA costume over an unusable control. NOT DONE: a real screen-reader run, contrast checks, and the year slider, which is two overlapping range inputs and almost certainly unusable by keyboard.
52. Done — finished and published this atlas
Wire the Croatian and Belgian lookups into a re-runnable job
Half of this was already done and the note had not caught up: harvest-croatia-surnames.py stopped reading a fixed list of 466 some time ago and now asks about every name the corpus believes is Croatian, skipping anything already in its scan cache. What was missing was the cadence — nothing ran it.
.github/workflows/surnames.yml runs it weekly, and two details are the whole design. The scan cache is gitignored, correctly, so a CI run would start blind and re-ask every name it has ever asked, every week, at a government form — actions/cache restores it. And HR_MAX caps one run at forty questions, so even with the cache lost the backlog drains over a few weeks instead of DZS being asked eleven hundred questions in a row. Taking by volume what a service offers by question is the thing this project does not do.
IT DOES NOT COMMIT, which is deliberate and follows the link-rot job next door. These are answers about people's names from national services and they should land in the repository because somebody looked at them, not because a cron job had a quiet Wednesday. Results come back as an artifact and an issue with the commands to fold them in.
53. Done — finished and published this atlas
The atlas speaks English at people researching Croatia
Done for the chrome, which is the first thirty seconds and the part that decides whether somebody stays. Croatian and Italian, 47 strings: the navigation, the search box and its label, the layer switches, the eight access classes, the legend summaries, the opening panel, the loading notice, the phone sheet's way out. The language comes from ?lang=, then a remembered choice, then the browser's own preference, so it works for somebody who never finds the switcher. lang= on the html element follows it.
WHAT IS DELIBERATELY NOT DONE, and the row itself asked for this: the prose. The paragraphs on this site are careful about evidence — what «attested» means, what a grey dot does not mean — and a translated «attested» that drifts is worse than an English one that does not. Those stay in English until somebody who speaks the language reads them, and the footer says so rather than implying a full localisation.
NO HREFLANG, on purpose. There is one URL per page; pointing hreflang at a page that is still mostly English would be a claim this site cannot back. Translated routes are the next step and would triple a 46,000-page build.
54. Next — ready to start this atlas
21,085 place pages share one social card
site/public/og has six images, one per page TYPE, so every place on earth shares a single grey rectangle when it is shared. A card naming the place, its country, its volume count and its shape on a small map would make a shared link legible before it is clicked — and the machinery exists: lib/surname-map.js already composes SVG at build time, and scripts/build-og-images.py already writes PNGs.
55. Done — finished and published this atlas
The data is CC0 and there is no way to use it
Done. /data/ documents every published file — path, size, shape, what it is and, read out of the file itself, its licence. That last part corrected the first draft: three licences are served out of this one directory, and the OpenStreetMap-derived sets are ODbL share-alike, so telling somebody they were CC0 would have put them in breach. The repository still carries no licence file for this atlas's own compilation; the page says so rather than inventing one.
56. Next — ready to start this atlas
An archive cannot show its own holdings here
516 institutions are listed and not one of them can put «our records, mapped» on their own site. A small embeddable widget — one archive, its places, its volumes — turns the directory into something the directory's own subjects want to link to.
57. Blocked — waiting on a person or an institution David
Nobody has told the 516 archives they are listed
Most carry access «unsurveyed», which means this atlas has published a claim about an institution without asking it anything. Writing to them is both the correction path and the reach: an archivist who is told «you are on this map, here is what it says, tell us what is wrong» usually answers, and sometimes links back. Needs the contact address of row 49.
58. Done — finished and published this atlas
There is no way to hear that the atlas changed
Done. /feed.xml is an Atom feed of the changelog, one entry per day rather than per commit — a day here is routinely fifteen commits and fifteen notifications is how a feed gets dropped. Advertised in every page's head and linked in the footer.
59. Done — finished and published this atlas
Surname pages are the front door and are treated as a side room
Done, as far as navigation goes. Surnames are second in Explore and named as the question — «Look up a surname» — instead of fourth under a word that reads like a table of statistics. The opening panel of the map now asks for a surname first as well. The surname pages themselves are still the next thing to deepen.
60. Done — finished and published this atlas
«Records in Bolivia» is a question nobody here answers well
Done. Every country page opens with what a reader came for: how many sources can be opened tonight, who to write to for the rest, what FamilySearch holds and over what years, whether the jurisdiction layer reaches this ground, and how many places have actually been walked. Writing it caught a real trap — 317 of 516 providers are unsurveyed, so the first draft would have read «2 of 19 sources can be opened» for countries whose other seventeen nobody has ever looked at. Unsurveyed is now its own number and never the bottom of a fraction.
61. Done — finished and published this atlas
The map opens by asking the reader to already know the answer
Done. The panel opens on «Start with what you have» — a surname, a village by any name it ever had, or only a country — with three live examples (Peeters, Gallignana, Breslau) that run the search on click, and one sentence saying the colour is what it costs to look.
62. Next — ready to start David
Nothing points at the atlas from where genealogists already are
Cyndi's List, the FamilySearch Wiki's country pages, national genealogical society link lists — these are where people looking for exactly this already go. Being absent from them is a bigger gap than any ranking problem.
63. Done — finished and published this atlas
Play the years, because that is the whole argument
Done. A play button runs the year window forward in ten-year steps from 1500, fifty years at a time, redrawing the jurisdiction shading and refiltering the shelf at every step. `p` starts it; Escape, or touching either slider, stops it.
64. Done — finished and published this atlas
21,302 dots at world zoom is a grey smear
Done. Below zoom 6 the shelf is drawn as one bubble per grid cell — 12° cells at zoom 2 down to 2° at zoom 5 — area by count, tooltip naming places AND volumes so the bubble cannot be mistaken for a book count. The individual markers come off the map entirely while bubbles are up, because a cluster drawn over the dots it summarises is just more ink.
65. Done — finished and published this atlas
A surname and a place cannot be asked about together
Done. «Blazevic Istria» is read as a name and a place: the query is split only when the whole of it matches no place, so nothing that works today changes, and the panel answers both halves at once. It refuses to conflate them — a register counting 11,163 people in Croatia is a fact about Croatia and not about the four villages on screen, and the panel says which places sit in an attested country and leaves the rest unmarked.
66. Done — finished and published this atlas
The panel is a wall of text on a phone
Done. Under 700px a result becomes a sheet over the bottom of the map with its own way out, instead of filling a box below the fold that the reader never sees. Driven off :has() on the body's hidden attribute, so there is no state to keep in sync and a browser without :has() keeps today's stacked panel.
67. Done — finished and published this atlas
A place's sources cannot be taken away from the screen
Done. A print stylesheet in the shared stylesheet — chrome, ad slot and map gone, every link followed by the URL it goes to, tables that do not break across a row — and a «Print this plan» button on every place page. The links mattered most: a printed «FamilySearch» that does not say where it goes is no use to somebody typing it back in.
68. Done — finished and published this atlas
The map is blank while it loads and says nothing
Done. The map says «Loading the atlas…», then «Drawing 21,302 places…» the moment the index arrives, and the notice is removed when the markers are on. A number rather than a spinner: it tells the reader the wait is proportionate to something.
69. Done — finished and published this atlas
The legend lists eight colours and teaches nothing
Done. The legend now states the two things that make it readable: that the colours run from what you can open tonight down to what needs a letter, and that a place takes the colour of the MOST OPEN thing it has — so red means red is the best on offer there, and green says nothing about what is underneath. It also says the swatches are filters, which was undiscoverable.
70. Done — finished and published this atlas
No keyboard shortcuts on a tool people will use for hours
Done. `/` focuses the search, `y` toggles the year, `p` plays the years, Escape clears the box or closes the phone sheet. None of them steal a keystroke from a field.
71. Done — finished and published this atlas
The jurisdiction layer covers 33 regions and is the reason to be here
Seven more record regions, 33 → 40. The Military Frontier is the one that earns its place: from 1746 a Lika or Slavonian village belonged to its REGIMENT, not to a county, so its registers were kept by the regimental chaplain and filed under Ogulin, Otočac, Slunj, Gradiška, Brod or Petrovaradin — and a search of the county archive for anything before 1881 finds nothing, which is not the same as there being nothing. Also the Netherlands (civil registration from 1811, and 1796 in the parts annexed early), Denmark (two parish registers kept separately from 1814, so a gap in one is usually survivable in the other — and Schleswig filed under Prussia from 1864 to 1920), Norway, Sweden (the 1686 church law and the husförhörslängder), Finland (the Karelian parishes' registers were evacuated WITH the people, so they are in Finland and not where the places are) and Portugal (civil registration only from 1911).
AND BELGIUM, added after checking the feature on a real name rather than the one it was built for. «Peeters» is about as Belgian as a surname gets, and its page said «no record region is drawn yet for Belgium» and then filled the screen with Bohemia, Pomerania, Acadia and Alsace — every region in every country where some register had counted a Peeters. Honest, and useless. So: nine eras for Belgium and the Southern Netherlands, whose hinge is 1796 — France annexes the ground in 1795 and civil registration begins the year after, so before that date there is no state record of a birth anywhere in the country, only a baptism. Its trap is that there is no central archive to write to: the State Archives keep the registers in thirteen provincial repositories, and a request to Brussels about a Limburg parish goes to the wrong building.
The list is also capped at five countries now, ordered by how many people a register counted — so the countries a name actually belongs to come first and the tail is a count rather than a wall. 41 regions.
72. Done — finished and published this atlas
The first thirty seconds are never rehearsed
Done, as far as a machine can do it. test/e2e/first-thirty-seconds.spec.js walks the path a stranger takes, in order, and asserts each step gives them something to act on: second one the map says what it is doing, second five the panel asks for something they actually have, second ten the world is bubbles rather than a smear, second fifteen a surname answers with ONE heading, second twenty a name and a place are asked together, second twenty-five the panel is not still answering the previous question, second thirty there is a way out. Plus the phone, where the answer must be a sheet and not a box below the fold. It is not a substitute for watching a real stranger — it is a rehearsal, so the same ground is not lost twice.
73. Blocked — waiting on a person or an institution David
The archive-kit pin is 156 commits behind, and holding it is a decision
Raised by the archive-kit session on 23 September: site/package.json pins TheArchiveKit#8174676 (16 Sep), kit HEAD is 2790d85, and the eight family archives are all on HEAD. It flagged five living-gate commits missing from this pin, c1f9958 above all — «the living check read only HTML, and an atlas serves its people as JSON» — on the grounds that a gate which cannot read JSON will report clean either way.
CHECKED HERE BEFORE DOING ANYTHING, and the answer is that neither half applies to this repository.
No gate runs in this build at all. `npm run check` is check-data.py and the node tests; CI runs the build and audit-site.py. checkliving is not invoked anywhere, so there is no gate to be out of date. The kit is used for exactly one thing here: WorkList.astro, which renders this page.
And there are no people to gate. This atlas has no GEDCOM, no people file and no living flag — the only string «living» in the whole corpus is the surname Living, 280 of them in the American register. archive-surnames.json states the boundary in its own header: surname strings only, no given name, no date, no place, no relationship, no identifier. The one place this could approach an individual is the Belgian commune grain, 510,811 rows of surname-by-municipality — and Statbel suppresses under five, so the smallest number published anywhere on this site is 5.
So the pin is not a privacy risk. It is still drift, and it reads as drift from outside, which is why this row exists rather than a shrug. Repinning is 156 commits of change to a shared dependency, and the kit session's own warning is that eight gate tools did not exist at this pin and should be expected to fail on first run with real work behind them. That is David's call on his own estate, not one to make blind from here, and doubly not with two deploys in flight. If it is repinned: 2790d85 or later, never 337a353, which raised NameError in checkliving for twelve minutes this morning.
The one thing worth lifting regardless of the pin is c1f9958's LESSON rather than its code: if this atlas ever publishes a person as JSON, the gate has to read JSON. Today it publishes none.
74. Done — finished and published this atlas
Attribution was met by memory, and memory failed twice
/data/ shipped in the morning saying «all CC0» over OpenStreetMap data. Corrected, and by the afternoon it said «there are three licences» over seven. Both were caught by a person looking, which is not a system, and the second only because the archive-kit session asked about something else entirely.
So check-data.py now refuses the build on three rules. A data file that names a source must declare a `licence` FIELD — two of them stated theirs in an English sentence inside `source`, where no machine could read it, which is why the page printed «none declared» over CC BY data and was not wrong to. Every licence in data/ must be named on /data/, which is the page that tells a reuser what they are taking on. And every licence asking for attribution must name its holder in the site footer, because CC BY asks for credit wherever the work is used and not only where it is catalogued.
It found twelve on its first run, and one of them mattered more than the licence question that prompted it: every volume title in every p/<id>.json is FamilySearch's catalogue metadata — «Defunciones 1980-1989» is their cataloguing, not our writing — and the page was filing those files under «this atlas». The fact that a collection covers a place is not theirs to own and is the whole point of this project; the strings are theirs. The page says so now.
The rule it cannot enforce is whether the credit is adequate. It checks that a name appears, and does not pretend to judge more.
Its own first version ran AFTER the loop that prints failures, so twelve findings were counted into the exit code and never shown. A gate that stops a build without saying why is worse than no gate.
75. Done — finished and published David
Three filters, three different shapes, and two of them hidden in full screen
David: «could things like the filters rather be a multi select dropdown? its got really messy. similarly the «what it costs» and «what kind of record» dropdowns? maybe same consistency?»
He was right on both counts. The layers were eight loose pills in a row while the other two were tidy disclosures reporting how many were on — so the control a reader touches most looked like scattered chrome beside two that looked designed. All three are the same control now, each stating its own state in its summary, so a closed one still answers «what is on the map».
AND IT SURFACED THE SAME BUG HE HAD JUST REPORTED, two controls further along. «What it costs» and «what kind» lived under the map, outside the element that goes full screen, so full screen hid them exactly as it hid the search box. All three are in the bar now, which full screen carries.
They are popovers rather than fold-outs: an inline disclosure pushed the map down every time somebody looked at a filter, and in full screen it would have changed the height of a bar whose height is measured.
TWO THINGS ONLY MEASURING CAUGHT. The popover rule appeared to work while the dev server was serving a stylesheet frozen at startup — the toggling all behaved, and only reading the computed `position` showed it was still `static` and the map was still moving. And the first attempt at keeping the last dropdown on screen flipped it to right-aligned under 900px, which pushed it 49 pixels off the LEFT edge: the bar WRAPS, so that control is at the right-hand end at one width and the left-hand end at another, and no breakpoint knows which. It is measured and nudged now, and verified on screen at 1000, 820, 640, 480 and 380 pixels.
And the count moved up with them, at David's asking. It reported on those filters and sat nowhere near them — under the map, which is outside the element that goes full screen, so the one line saying whether a filter did anything was missing in the mode where the map is biggest. It sits at the right-hand end of the bar when there is room and takes its own line when there is not, which matters because it is not always three words: set a year nothing can answer for and it becomes a sentence explaining why the map is empty, with a way out. Checked in that state — the bar grows one row, the buttons stay put, the map keeps its 600 pixels.
That emptied .ra-foot entirely, so it is gone.
76. Next — ready to start this atlas
The colour legend promises a spectrum the data does not have
David asked «i dont even see much for eg. costs, almost nothing for most categories? can you check as well», and the count is worse than «not much». Of 21,473 places: unsurveyed 19,416 (90.4%), account 2,029 (9.4%), onsite 19, catalogue 6, free 3 — and index, mixed and paid are ZERO. Eight categories, three of which cannot remove anything and three more that remove almost nothing.
The counts are on the filter now, and the three empty ones are drawn dead and disabled, so the control no longer offers choices that do nothing. That is honesty, not a fix.
THE FIX IS SURVEYING. `unsurveyed` is not a property of the world, it is this atlas not having looked: 317 of 519 providers carry no access class, and a place takes the best class among the collections reaching it. Row 40 (the 204 French archives) is the same debt in one country. Until that is paid the map is effectively two colours — grey for «nobody looked» and green for «FamilySearch, account needed» — and the eight-colour legend describes an atlas that does not exist yet.
Same for record kind: only 1,727 of 15,780 loud places carry any kind bit, so any kind filter hides seven places in eight. The counts say so now.
77. Done — finished and published David
Remember the searches somebody already made
David: «is there a way to remember the last searches someone made? and show them as quick pills?» A researcher works down a list of parishes and comes back to the same three or four names all week, and every visit started from an empty box.
The last eight, most recent first, under the search box. In localStorage and nowhere else — no account, no server, nothing that leaves the machine, and a «Forget» that empties it. Every read and write is wrapped, because private windows and blocked site data throw on access rather than returning empty, and a map that would not draw because it could not remember a search would be a poor trade.
REMEMBERED ONLY WHEN THE SEARCH WORKED, which took two goes. The first guard was «is a panel showing», and a search that finds nothing still shows a panel — saying «nothing here, and here is why» is the honest answer — so «zzzzqq» went straight into the list of things that worked. The second version used a real flag, and then dropped «Blazevic»: a place match is decided 140ms after the last keystroke, but a surname match waits on a shard fetch that lands long after any reasonable timer. So the timer catches the fast answers and the slow ones report themselves when they arrive, as long as the reader has stopped typing.
23 September, later: moved off the bar entirely. It was a standing row of real estate for something wanted at the start of a visit and never again, so it is a dropdown under the search box now — on focus, gone the moment you type, because these are the searches you made and not suggestions for the one you are making.
78. Done — finished and published this atlas
Derive our own surname origins, because Wiktionary answers 2.8% of them
Done 30 September: 2.8% to 98.0%. `scripts/derive-surname-origins.py` reads 940,562 surnames and hands back an origin for 921,288 of them (98.0%) — 86,833 strong, 583,554 fair, 250,901 weak. Wiktionary's 25,972 rows are still Wiktionary's, kept separate and still labelled as its own; these are derived and say so.
IT IS NOT AN ETYMOLOGY AND THE FILE SAYS SO IN ITS FIRST SENTENCE. It does not know what a name means. It knows where the name is now, where it was first recorded, and whether a place carries it too, and every row carries its own `basis` — the working, not just the answer. That was the condition this row set for itself and it is the reason the panel can print «first attested in Croatia in the 1710s, and also a village in Istria» rather than «Croatian».
Three bugs in the derivation were found by David's own examples rather than by me: a three-letter-language short-circuit that answered before the evidence was read, ties broken by dictionary insertion order, and register counts outweighing everything else — which is why Defranceschi came back as anything other than Italy. The origin now also sets where the map opens when a surname is chosen.
Cost: the first cut was a 197 MB JSON file. Structured and gzipped it is 8.3 MB, the same trick and the same reason as the gazetteer.
WHAT THIS ROW DID NOT CLOSE is carried on as row 79: the 52,515 names with a Wiktionary entry and no etymology in it, where the panel still says nothing at all.
79. Next — ready to start this atlas
Say so when Wiktionary has the name and no etymology for it
Two different silences, and only one of them is written. 52,515 surnames have a Wiktionary entry that contains no etymology. «The dictionary holds this name and does not say where it is from» is a finding; «the dictionary has never heard of it» is a different one; the panel currently prints nothing for either, so a reader cannot tell which they are looking at — and for the first group there is a page worth opening.
Cheap and bounded: the entry is already known to exist, so this is a label and a link, not a harvest. It is deliberately NOT an invitation to quote Wiktionary prose — David on the reference itself: «Wiktionary shouldn't be a reference or link, it's far too small; just harvest the origin of the name.» What goes on the page is the absence, stated.
80. Next — ready to start this atlas
Grow the corpus beyond Italy and the USA — and France is the whole answer
David, on the headline «21,754 places · 2,893,805 record volumes · 528 archives in 83 countries · 41 record regions»: «how do we break that and push it larger?» Measured first, because three of those four numbers are less than they look.
THE VOLUME COUNT IS ONE COUNTRY. Italy is 46.4% of all 2,893,805 volumes. Italy and the United States together are 67%. Nine countries hold 1% or more; twenty countries that have a place on the map hold fewer than a hundred volumes. Adding Italian places moves the headline and almost nobody's research, because the shelf there is already deep.
THE PLACE COUNT IS MOSTLY CONTEXT-FREE. 21,754 places is 16,058 drawn plus 5,696 quiet — and only 8,474 of them (39%) sit inside one of the 41 record regions. Parishes, burial grounds, the era table and «who governed this ground» are all region-scoped, so 13,280 places (61%) can never show any of it, whatever is harvested. A place outside a region is a dot with volumes and no story.
THE ARCHIVE COUNT IS MOSTLY UNOPENED. 329 of 547 providers are `unsurveyed` — 60%. We know where they are and nobody has looked. And 66 countries have a provider but under a thousand volumes: Bulgaria has 30 providers and zero volumes, Serbia 6 and two, Bosnia 4 and none, Greece 2 and none, Norway 1 and none.
SO THE ORDER OF WORK, BY RETURN PER UNIT OF EFFORT:
1. More regions. A region is a polygon and a list of eras — days of work, no crawl — and it switches on parishes, burial grounds, the era table and the jurisdiction trap notes for every place inside it at once. This is the only lever that improves places already on the map. 41 regions reach 39% of them.
2. Survey the 329. Each one converts a known address into volumes and links. This is the lever that grows the volume count outside Italy.
3. The 66 countries with archives and no records, Bulgaria first, because 30 providers and zero volumes is either a harvest that was never run or a gate nobody has written down.
4. Not more Italian places. Breadth in the 46% is the one thing that flatters the headline and helps nobody.
Row 79 and the cemetery work of 30 September are the same lesson in miniature: the map looked empty at Senj because the data was filtered, not because the ground was bare.
FOLLOWED UP THE SAME DAY, AND THE ANSWER IS ONE COUNTRY. «Survey the 329» is not a diffuse backlog. It is concentrated: France 200, Germany 32, Bulgaria 30, and everything else in single figures. France alone is 61% of every unsurveyed provider this atlas holds.
Set beside Italy it is embarrassing, and it is the clearest instruction this worklist has ever contained:
| | providers | unsurveyed | places | volumes |
|---|---|---|---|---|
| Italy | 129 | 2 | 2,897 | 1,343,253 |
| France | 205 | 200 | 81 | 1,513 |
| Germany | 37 | 32 | 191 | 4,218 |
| Bulgaria | 30 | 30 | 0 | 0 |
FRANCE HAS MORE ARCHIVES CATALOGUED THAN ITALY AND A THOUSANDTH OF THE VOLUMES. 203 of its 205 are regional — the archives départementales and communales, which are the free, online, état-civil and parish-register backbone of French genealogy and among the most generous public record holdings in Europe. Nothing about France is harder than Italy. The only difference in this atlas is that somebody surveyed Italy's 129 and nobody has opened France's 200.
Bulgaria's 30 are not even triaged — `triage: null` on every one, all state archives, so nobody has so much as looked at whether they answer.
So the order narrows: France first, then Germany's 32, then Bulgaria's 30 — before any further breadth anywhere. And note what this does NOT need: no new region, no gazetteer work, no licence question. The addresses are already in providers.json and have been all along.
81. Running — a machine or a person is at it now this atlas
Survey France's 95 departmental archives — the half of the backlog that holds the records
Row 80 said France was 200 unsurveyed providers and 61% of the whole backlog. Looked at properly, that is two different populations filed together:
- 108 `ac-` rows — archives COMMUNALES. A commune's own municipal papers. Thin online, and mostly not where registers live.
- 95 `ad-` rows — archives DÉPARTEMENTALES. One per department, and these are the ones that matter: French departmental archives put état civil and registres paroissiaux online, free, for most of the country. Two were surveyed — Bas-Rhin and Haut-Rhin.
Surveying France alphabetically would therefore spend most of the effort on the wrong half. Do the 95 first.
A LIVENESS SWEEP OF ALL 95, 5 October: 85 answered a scripted request, 10 refused. The refusals are the interesting part, and they are NOT link rot:
| code | count | what it means |
|---|---|---|
| 403 | 3 | Forbidden to a script |
| 418 | 2 | the teapot code sites use to turn scripts away |
| 404 | 2 | the only ones likely to be genuinely wrong addresses |
| 000 | 3 | no connection completed |
ad-tarn answered 418 and is perfectly alive: in a browser it resolves to the real Archives Départementales du Tarn, with an anti-DDoS flood page standing in front of scripts. It has since been surveyed and promoted to `free` — «consultation en ligne de documents numérisés», no account. So a refusal from a sweep is a gate, not a grave, and every one of those nine rows now says so in its triageNote rather than being written off.
WHAT IS LEFT is the slow part and there is no shortcut: each departmental portal is bespoke software, so there is no pattern to match and no bulk promotion to be had. 93 of the 95 still need a browser, one at a time. Alsace CEA and Mayotte need a corrected URL first.
Nothing in this archive is in that state.
The other work lists
- Blažević Archive
- Booyzen Archive
- D'Arcy Archive
- Defranceschi Archive
- Falco Archive
- Lerena Archive
- Mazza Archive
Each archive keeps its own, in the same shape. Nobody has a list of everybody — that is the point: the work sits with whoever can do it.
The pages that answer what is known are the map, Countries, What's missing and What changed.