49 Commits

Author SHA1 Message Date
Frank Bartolitsch f8a4fcefd2 fix(taxonomies): respect active locale in admin surfaces (#2342)
* fix(taxonomies): respect active locale in admin surfaces

Group translated definitions by logical identity before rendering admin navigation and editor choices. Scope visible term counts and cache entries to the resolved content locale.

* fix(taxonomies): preserve locale in admin term views

* ci: update query-count snapshots

* fix(admin): preserve locale sidebar active state

* fix(taxonomies): preserve exact locale updates

* test(perf): update locale query snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-08-18 07:18:22 +00:00
Matt Kane 5c216c2f92 fix: preserve taxonomy assignments across locale fallbacks (#2508)
* fix(taxonomies): preserve assignments across locale fallback

* fix(admin): clarify taxonomy locale resolution

* perf(taxonomies): join locale fallbacks once

* fix(admin): gate taxonomy creation by permission

* chore: retry external preview build

* fix(admin): show flat taxonomy creation errors
2026-08-17 09:45:55 +00:00
CacheMeOwside 5224b5711a feat(core): configurable titleField and dateField per collection (#1973)
* feat(core): configurable displayField and dateField per collection

* style: format

* ci: update query-count snapshots

* add fixes as per PR comments

* ci: update query-count snapshots

* reject field type changes that break displayField/dateField

* rename displayField to titleField

* match content-list search and suggestions against configured titleField

* ci: update query-count snapshots

* update integration tests

* style: format

* removed reference to PR from comments

* Remove PR reference

Co-authored-by: emdashbot[bot] <273199577+emdashbot[bot]@users.noreply.github.com>

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
Co-authored-by: emdashbot[bot] <273199577+emdashbot[bot]@users.noreply.github.com>
2026-08-16 07:41:13 +01:00
Matt Kane 6602ae05bc Fix taxonomy filter detection across collections (#2459)
* fix(loader): scope taxonomy filters to collections

* ci: update query-count snapshots

* fix(loader): harden taxonomy name cache

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-08-13 17:23:31 +00:00
Matt Kane 598e6fb95f Fix taxonomy assignments across content translations (#2461)
* fix: share taxonomy assignments across translations

* fix: count shared taxonomy assignments once

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-08-13 18:20:40 +01:00
Matt Kane f6385dab2c refactor(cli): deprecate emdash dev (#2405)
* refactor(cli): remove dev command

* refactor(cli): deprecate dev command
2026-08-10 13:41:21 +01:00
MA2153 ea4c39bb18 feat(taxonomies): let terms carry a manual order (#2353)
* feat(taxonomies): let terms carry a manual order

Term listings were ordered by label alone, so the only way to control
how a taxonomy renders on a site was to rename its terms. Add a
`sort_order` column and order every term read by (sort_order, label, id).

Existing rows default to 0, which keeps a taxonomy nobody has reordered
alphabetical -- the behaviour before this change. A group only becomes
manually ordered once it is reordered, at which point it is renumbered
0..n-1 and new terms append to it; groups still at a uniform 0 keep
inserting new terms alphabetically.

Ordering is scoped to one sibling group in one locale so an order set in
one language cannot decide placement in another, and reordering never
reparents -- the endpoint rejects any list that is not the group's exact
membership rather than applying a stale one.

sort_order leads the ORDER BY and no index satisfies it, so the seeked
group is sorted in a temp b-tree. That costs no extra rows read (what D1
bills) and ORDER BY label had the same sort before; the alternative is a
five-column index paid on every term write. The query-plan test pins the
index seek so the #1723 full-locale scan cannot come back unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(taxonomies): tie a term's order to the group it is rendered in

Two places assumed a term's `parent_id` names the group it belongs to.
Neither holds once the tree is hierarchical.

The admin derived the group of a move from the moving term, so a term
whose parent has no row in the current locale -- which the term list
shows at the top level -- addressed its absent parent's group. That group
has no members, so every move on such a row failed with
REORDER_MISMATCH. The group now travels with the siblings it is rendered
from, which is what the server resolves membership against.

Reparenting left `sort_order` alone, so a term carried a position from
the group it left. Beyond landing mid-list, one non-uniform value makes a
group nobody has ordered read as ordered, switching it from alphabetical
insertion to append-at-end. An update that changes `parent_id` now
re-places the term by the same rule a new term uses.

`nextSortOrder` still scopes a group by the raw `parent_id` column while
the reorder endpoint scopes it by the effective parent; they agree except
under an untranslated parent, where a new top-level term can land
mid-list. Closing that costs a parent-existence check plus a correlated
NOT EXISTS on every term create -- documented on the PR rather than paid
for here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(taxonomies): place a translation by the group it lands in

A translation copied its source's `sort_order` unconditionally, so one
created under a different parent than its source -- including with no
parent at all, which makes it a root in the target locale -- carried in a
position belonging to a group it was not joining. It lands mid-list, and
one non-uniform value makes a group nobody has ordered read as ordered,
switching it from alphabetical insertion to append-at-end.

The copy now applies only when the new row joins the group that mirrors
the source's; otherwise the row is placed by the same rule a new or
reparented term uses. A translation of a term whose parent is unchanged
still inherits its source's position, so a translated tree keeps the
order its source was given.

Reachable from `POST /taxonomies/{name}/terms` with `translationOf` and
from a seed whose translated term declares a different parent. The
dedicated translations route passes the source's parent through, so the
admin's Translate button never hit this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(taxonomies): trim comments to what a future reader needs

The manual-order work left comments that argue for the code rather than
explain it: a query-plan test header that justified its own assertions and
told a reviewer which ones to delete, issue references that go stale on
merge, and "rather than X" notes restating decisions the code already makes
plain.

Also drops a filler assertion that only existed to consume a destructured
binding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci: update query-count snapshots

* fix migration number

* feat(taxonomies): give a term one order across its locales

Reworks manual term ordering so a position belongs to a term rather than
to one of its rows, and drops the alphabetical fallback entirely.

`sort_order` is now per translation_group: every row of a group carries
the same value, mirroring `parent_id`, which already stores the parent's
translation_group rather than a row id. Sibling groups key on the raw
`parent_id` column everywhere -- placement, reordering, and the
migration -- so one definition decides what a group is.

That removes the cases the previous model had to reason about. There is
no rule for placing a translation, because a translation is the same
term at the term's position. A half-translated locale can't produce
gaps or duplicate positions. And the reorder endpoint no longer takes a
`locale`, because a position doesn't have one -- which also removes the
cross-locale renumbering that omitting it used to cause.

Migration 056 mints a position for every existing term instead of
defaulting them all to 0. Numbering by (label, id) keeps rendered order
identical the moment it runs; what changes is that a term created
afterwards appends to the end of its group rather than slotting in
alphabetically. Inferring "has this group been ordered?" from whether
its values were uniform is gone with it.

Reorder now permutes the listed terms within the positions they already
occupy instead of demanding the group's exact membership. A locale
renders only the terms translated into it, so a caller often cannot name
every member; a member left out keeps its place, which also makes a
stale list harmless rather than rejected.

A child whose parent has no row in the current locale is rendered at the
top level but belongs to its parent's group. Its carets are disabled and
it is left out of the ids sent for the group it is drawn in, rather than
silently addressing a group it isn't in.

Ordering applies to reads of a sibling group. The terms attached to one
entry are a flat list that can span groups, so those reads are unchanged
and still order by label; ordering terms within an entry is a separate
feature.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci: update query-count snapshots

* fix(taxonomies): move a term's parent with the whole group

A term's `parent_id` holds the parent's translation_group, so it is
locale-agnostic and every row of a group should carry the same value.
`update()` wrote it with `WHERE id = ?`, so reparenting a term left it
nested in the locale it was moved in and a root in all the others.
Migration 045 rewrote parent_id values but scoped its UPDATE to
`parent_id IS NOT NULL`, so it converged groups whose rows all named a
parent and never the ones where a row still held null -- the exact shape
a one-row reparent produces. Reparenting now writes the group.

Sibling positions are keyed the same way, so a divergent group also put
two roots on the same position, and `reorder()` cannot permute positions
that tie: it returned success while the list never moved, leaving the
group unorderable. It now renumbers 0..n-1 when the positions it is given
tie, which honours the request and clears the tie. Renumbering keys on
each member's index in the rendered order rather than on a row id, since
ULIDs minted in the same millisecond have no order and would make the
result arbitrary. `reorder` takes the sibling list in rendered order for
that reason.

Creating a translation into a different parent inherited the source's
position, colliding with whatever already held it. A translation now only
takes the group's position while it stays in that group, and appends
otherwise.

Migration 056 guarded its whole body on the column it adds, so a run that
died during the backfill was never recorded, retried, and immediately
early-returned -- stranding most groups at 0 with no way to re-run.
Only the ALTER is guarded now and both passes run unconditionally,
following 051. The backfill was one UPDATE per translation group, which
spends a D1 subrequest per term and exhausts the 1,000 a Worker gets on
the free plan on any sizeable taxonomy; it is now chunked CASE statements
at 32 groups, three bound parameters each, inside D1's 100-parameter
ceiling. The parent_id repair runs before numbering, because numbering
keys on parent_id and would otherwise place a divergent group by whichever
row it happened to read.

The reorder endpoint's `order` response was never read: the admin client
returned it and the caller discarded it, and its localeCompare tiebreak
could disagree with the next list read. It now answers `{ reordered: true }`
and `repo.reorder` returns void. `ids` caps at 100 to match the comparable
arrays in the same schema, which also bounds the writes in one transaction.

Tests: the runtime-helper test could not fail, because `cachedQuery` runs
its loader when no cache backend is configured, so invalidation is
invisible in that file. It is renamed to say it covers ordering, and
term-list-object-cache.test.ts covers invalidation against a real backend.
Adds coverage for optimistic rendering, for the queued-move refetch guard,
and for the bridge ORDER BY on both sandbox implementations -- the workerd
fixture's rows all sat at the default position, so its order assertion was
purely alphabetical. Drops the half of the findByName test that asserted
`repo.reorder`'s calling convention rather than any behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(taxonomies): write a reorder as one statement per chunk

A reorder applied one UPDATE per translation group inside withTransaction,
which degrades to a bare passthrough on D1. A failure partway left the
sibling group half-permuted, and the tie-repair path spent a subrequest
per member of the group.

Batch the writes into one CASE statement per 32 groups, the same
parameter arithmetic migration 056 uses. A swap is now a single
statement and cannot tear; only a renumbering wide enough to span chunks
can, and that leaves ties the next reorder renumbers away.

The migration keeps its own copy of the helper: a migration has to stay
frozen, and sharing one would let a later edit rewrite history for
anyone mid-upgrade.

Also:

- Say that the migration preserves the *source* locale's order and
  re-sorts the others. One position per translation group means one
  collation wins; the docstring and changeset claimed more than that.
- Stop claiming term listings arrive "in rendered order" — with locales
  interleaved a tie falls to whichever locale's label sorts first.
- Explain the disabled carets on a term whose parent has no translation
  in the current locale, in the accessible name and as a hover title.
- Narrow the changeset: getTerms() honours the manual order, the terms
  attached to an entry are still alphabetical.
- Make reorderTermsBody strict, and document that omitted terms hold
  absolute positions, so a one-step move in a partial list can be a
  two-step move in a fuller one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(taxonomies): type the reorder CASE value for Postgres

Postgres resolves a CASE whose THEN arms are all untyped bind parameters
to text, then refuses to assign that to an integer column:

  column "sort_order" is of type integer but expression is of type text

The WHEN parameters take text from the comparison against the selector,
and nothing pins the THEN parameters, so the whole expression lands on
text. node-postgres sends numbers with an unspecified type OID, so a JS
number does not settle it either. SQLite does not type-check the
assignment, which is why this only showed on the Postgres dialect runs.

Both the batched reorder write and migration 056's backfill emit that
shape, so on Postgres every term reorder failed -- swallowed into a
failed ApiResult rather than surfacing -- and the migration could not
mint positions at all.

CAST the value in each arm. applyByGroup serves parent_id as well as
sort_order, so it takes the cast target from its column argument. The
cast binds no parameters, so the three-per-group budget behind
GROUPS_PER_UPDATE is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(taxonomies): seed a missing translation_group before numbering terms

Reorder and reparent both key sibling groups on `translation_group`, while
callers derive a group as `translationGroup ?? id`. A row whose column is
null therefore matches nothing: the write reports success and changes
nothing.

Migration 036 seeded the column on every row it rebuilt and the repository
has set it on every insert since, so no supported path produces a null. 056
normalises anyway, since it is the migration that starts keying group writes
on the column in SQL. Repairing the row is strictly better than coalescing
at each read site: a null is equally invisible to translation lookups and to
the `content_taxonomies` join, and an expression key would cost the index
seek on every reorder.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-08-09 08:51:04 +01:00
Scott Buscemi 534f23884f fix(ui): move Comments/CommentForm to emdash/ui/comments subpath (#2039) (#2044)
* fix(ui): move Comments/CommentForm to emdash/ui/comments subpath (#2039)

Their <style> blocks were pulled into a shared, render-blocking CSS chunk
on every page that imported anything from the emdash/ui barrel (e.g.
PortableText), because Astro scans the whole barrel module graph. Splitting
them into a dedicated entry point keeps comment CSS off pages that don't
render comments, mirroring the existing emdash/ui/search entry point.

* fix(ui): keep barrel Comments exports as deprecated (#2039)

Address ascorbic review: add emdash/ui/comments without breaking existing
emdash/ui imports. Mark barrel re-exports @deprecated (remove in 1.0),
allowlist the subpath in the public-source guard, and sync template skills.
2026-08-08 11:39:42 +01:00
Eder Sánchez 85dd99d348 fix: skip byline query path when the bylines table is empty (#2304)
* fix: skip byline query path when the bylines table is empty

The folded-byline fast path fell back to the full query path whenever any
entry in the batch had an author_id but no folded credits. On sites that
never use bylines every entry matches that shape, so every content read
paid getBylinesForEntries round trips (chunked content-byline lookups
plus a user-id lookup per locale bucket) that could only return zero rows.

Fold an uncorrelated existence probe on _emdash_bylines into the content
query (evaluated once per statement — no extra round trip). An empty
table makes an empty fold authoritative: no credit can exist in any
locale and the author fallback has no byline to resolve to, so hydration
serves the folded result directly. When the probe is missing (cached
snapshots) or the table has rows, the conservative fallback is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: update query-count snapshots

* docs: carve out same-statement probes from the has-any-probe rule

The rule exists because a separate LIMIT 1 round trip costs every live
request. A probe folded into a query the request already runs has no
such cost, and the byline hydration fast path relies on exactly that
shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: update query-count snapshots

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
Co-authored-by: Matt Kane <m@mk.gg>
2026-08-08 09:07:17 +00:00
Noah (Nguyen Pham) 72660dabdb fix(core): prevent pre-publication scheduled content leaks (#2349)
* fix(core): make scheduled publication atomic

* fix(core): require committed publication for public reads

* ci: update query-count snapshots

* test(core): use valid portable text in publication regression

* test(core): cover batched scheduled publication visibility

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-08-06 11:33:28 +01:00
MA2153 3aabb7bff8 fix(core): drive taxonomy term counts from the pivot (#2238)
* fix(core): drive taxonomy term counts from the pivot

The consolidated term-count query joined `content_taxonomies` to the
content table with an `INNER JOIN`. On stats-blind SQLite/D1 the planner
picked `ec_*` as the outer table and re-ran the whole
`taxonomy_id IN (SELECT ...)` term list as a pivot-primary-key probe for
every visible entry in the collection:

    SEARCH e USING INDEX idx_ec_<collection>_deleted_status (deleted_at=?)
    SEARCH ct USING COVERING INDEX sqlite_autoindex_content_taxonomies_1
        (collection=? AND entry_id=? AND taxonomy_id=?)
    LIST SUBQUERY 1

so the cost was `entries x terms`, not a scan — which is why the
composite indexes on the pivot never helped: the pivot was never the
driving table. On a collection of ~26k entries with a ~1.4k-term
taxonomy one call read ~35.6M rows in ~29s, on every render of a term
list or taxonomy filter.

Switch to `CROSS JOIN` with the join predicate in `WHERE`. In SQLite
that is a join-order hint, not a cartesian product: it pins the pivot as
outer, so the terms are seeked on a `(taxonomy_id, collection)` index
and the content row is touched once per assignment by primary key.
Postgres has statistics and treats it as a plain inner join.

Measured on a production D1 with the dataset above: 35,627,677 rows /
28,892ms -> 63,854 rows / 120ms. The predicates are untouched, so the
counts are identical.

Closes #2237

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci: update query-count snapshots

* fix(core): trim narrative/issue-referencing comments per review

Comments should state the invariant, not the PR story or issue number.

* fix(core): tighten changeset and comment per review

Keep the changeset user-facing (observable slowness, not internal
mechanics); drop the justifying "deliberately".

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-26 17:18:03 +01:00
logelog f81aa6842c fix(core): compute taxonomy counts on demand (#2219)
* fix(core): compute taxonomy counts on demand

* fix(core): version taxonomy term-list cache key

---------

Co-authored-by: logelog <194732487+logelog@users.noreply.github.com>
2026-07-24 15:00:40 +01:00
MA2153 c24b7d3be5 fix(loader): seek taxonomy-filtered listings via a denormalized pivot (#1962)
* fix(loader): seek taxonomy-filtered listings via a denormalized pivot

Denormalize the filter+sort columns onto content_taxonomies, mirror the
ec_* sort indexes onto the pivot, and drive taxonomy-filtered listings
from the pivot with an authoritative ec_* re-check. Fixes full-collection
scans (~75k D1 rows read for a one-row page) on selective terms (#1834).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: update query-count snapshots

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-12 11:36:22 +01:00
marcusbellamyshaw-cell 558e06fc0d fix(ci): allow fork checkout in auto-format-apply workflow (#1904)
* fix(ci): allow fork checkout in auto-format-apply workflow

actions/checkout now refuses to check out fork PR code from a
workflow_run trigger unless allow-unsafe-pr-checkout is set, which broke
the formatted-patch push-back for fork PRs (e.g. run 29045432430). This
job never executes the fork's code -- it only applies an inert diff
artifact and pushes back via an app token, same shape as the already-
merged fix for query-counts-apply.yml (#1816). Added the opt-in flag
with a comment explaining why it's safe.

* ci: update query-count snapshots

---------

Co-authored-by: Matt Kane <mkane@cloudflare.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-10 11:06:12 +00:00
Matt Kane 9c0733f6f5 fix(taxonomies): count only publicly visible entries in term counts (#1892)
* fix(taxonomies): count only publicly visible entries in term counts (#581)

Term usage counts aggregated the content_taxonomies pivot directly, so
drafts, scheduled-future, and trashed entries inflated the counts shown
in the Categories/Tags widgets, on term pages, and in the admin term
list. All three paths now share a single UNION ALL count query that
joins the content tables, filters with buildStatusCondition (published
or scheduled-and-due) and deleted_at IS NULL, and scopes to the
taxonomy's declared collections.

Consolidates the fixes proposed in #1839, #822, and #593.

* ci: update query-count snapshots

* fix(taxonomies): include collection scope in term-count request-cache key

Per-locale rows of the same taxonomy def can drift in their declared
collections, and a future caller may pass a narrower scope; keying the
request cache by taxonomy name alone let the first scope poison the
entry for the rest of the request. Raised by review on #1892.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-09 16:00:06 +01:00
Matt Kane 8107edf8cf docs: document Cloudflare scheduled publishing setup (#1836)
* docs: document Cloudflare scheduled publishing setup

* ci: update query-count snapshots

* Update wording and asides

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-06 15:07:33 +01:00
Matt Kane 92fd412272 fix(skills): correct audit-log example, re-sync template skills, stop .gitignore clobber (#1732)
* fix(skills): correct audit-log plugin example and re-sync template skills

The plugin registration example in building-emdash-site showed a named
auditLogPlugin() factory that has never existed -- the package's only
export is a default plugin descriptor, matching what the template
astro.configs already do. Re-running sync-template-skills.sh also
propagates the ctx.input guidance (#1555) and trailing-slash page
resolution note (#1305) that landed in skills/ without a re-sync.

sync-cloudflare-templates.sh no longer copies .gitignore over the
cloudflare variants, which was erasing their wrangler-specific
.dev.vars entries on every run.

* fix(docs): address Copilot review findings on plugin examples

Cast Block Kit interactions to the real BlockInteraction union from
@emdash-cms/blocks instead of an ad-hoc shape that under-typed
form_submit and omitted page_load's page field. Fix the emdash()
plugins JSDoc example in core, which showed the same nonexistent
auditLogPlugin()/webhookNotifierPlugin() factory API the skills doc
did -- both plugins are default-export descriptors.
2026-07-02 21:02:33 +01:00
Matt Kane 5299d38d9c fix(core): add hreflang alternates for untranslated sitemap entries (#1696)
* fix(core): add hreflang alternates for untranslated sitemap entries

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-01 15:34:44 +00:00
Matt Kane af31994956 chore: fix broken lockfile (#1676)
* Revert "Regenerate lockfile"

This reverts commit ba4ed17dc5.

* Fix lockfile

* Restore missing esbuild@0.28.1 platform binaries in lockfile

The lockfile's esbuild@0.28.1 entry listed only 15 of 26 platform
optionalDependencies, missing @esbuild/linux-x64 (and other x64/ia32
targets). On Linux CI, esbuild's postinstall resolved a mismatched
binary and failed the version assertion, breaking every job at install.

Repaired with pnpm --fix-lockfile after evicting stale esbuild metadata
from the pnpm cache, so all 26 platforms resolve with integrity.

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-07-01 07:01:40 +01:00
Matt Kane c056060cdb perf(core): fold byline + taxonomy hydration into the content query (#1619)
* perf(core): fold byline + taxonomy hydration into the content query

loadEntry/loadCollection now hydrate bylines and taxonomy terms via correlated JSON-array subqueries in the content query instead of two follow-up round trips. Per-page query counts drop substantially (demo article 24->17, index 11->6).

- Dialect-aware aggregation (SQLite json_group_array/json_object string; Postgres json_agg/json_build_object parsed JSON), validated on both.
- Terms correlate on the entry's own locale (#1441); credits sorted in the consumer (dialect-neutral).
- Entries with byline custom fields or a missing/locale-mismatched explicit credit fall back to the existing query path.

* ci: update query-count snapshots

* fix(core): address fold-hydration review findings

- Carry avatarMediaId in the folded byline JSON; it is a required
  BylineSummary field templates read to render author avatars, so the
  fold dropping it broke avatars on the fast path.
- Fall back to the byline query path when the custom-field probe throws
  a non-missing-table error, instead of serving folded bylines with
  empty customFields.
- Prime the per-entry term request cache from the folded-terms fast path
  (new primeFoldedEntryTerms helper), so subsequent getEntryTerms calls
  hit the cache instead of issuing an N+1 query.
- Regenerate the SQLite query-counts snapshot to match the emitted SQL.
- Add a public-shape test (avatarMediaId, term-cache priming).

* ci: update query-count snapshots

* perf(core): drop the taxonomy-defs lookup from folded-term priming

The previous priming seeded [] for taxonomies applicable to the
collection but absent from the entry, which required a getTaxonomyDefs
query on every fold render. That added one round trip to routes that
never call getEntryTerms for an absent taxonomy (contributors, about,
rss, search), eroding the fold's win.

Prime only the wildcard and present-taxonomy keys, straight from the
folded data with no DB lookup. getEntryTerms(id, absentTaxonomy) falls
through to its own cached query instead, which is the rarer case.

* ci: update query-count snapshots

* chore: regenerate D1 query-text snapshot for the hydration fold

The D1 query-text snapshot was last written in #1580 and still showed
the pre-fold path (separate byline and term queries). CI regenerates it
but only commits the count snapshots, so it never got refreshed. Counts
are unchanged; this just updates the recorded SQL to the folded query.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-06-25 13:22:03 +01:00
Matt Kane ca47da485d fix(core): capture streaming queries in query instrumentation (#1580)
The per-query log (EMDASH_QUERY_LOG=1) flushed its recorder when middleware
returned, i.e. when the response headers were ready but before the body
streamed. Queries issued by components during streaming were appended to the
recorder but never emitted, so the query-count harness only saw pre-header
queries. Astro 7's queued rendering moves more queries into the streaming
phase, which made the gap obvious (the post-detail fixture route reported 9
queries while the request actually ran 24).

Flush the recorder when the body finishes streaming instead (in the same
stream-end transform that already snapshots the metrics), and keep a fallback
flush in the middleware finally for bodyless responses (redirects, 304s). The
flush is now idempotent so the two paths can't double-emit.

Also adds a companion query-text snapshot to the harness
(query-counts.queries.{target}.json): a per-route map of the actual SQL to its
occurrence count, so a count change shows which query appeared or vanished, not
just that the number moved. Snapshots regenerated for both targets now reflect
full per-request query counts.
2026-06-22 14:38:26 +01:00
Matt Kane 3d423a796d perf(core): cut redundant queries on content pages (#1498)
* perf(core): cut redundant queries on content pages

- Cache getWidgetArea per request (it was the only content helper not
  request-cached).
- Fetch taxonomy term usage-counts once per request via a shared
  request-cached aggregate, instead of re-running the full content_taxonomies
  GROUP BY for every taxonomy widget (Categories + Tags each ran it).
- Make getTermsForEntries cache-aware: reuse already-hydrated per-entry terms
  from the request cache and only query the misses. Returns private copies so
  callers mutating the result can't poison the cache, and orders the miss path
  by label to match the hydration primer.

All same-shape, all backends. Adds regression tests for cache reuse and
mutation-safety. Updates the sqlite query-count snapshot (post route -1).

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-06-16 14:06:13 +01:00
Matt Kane f9362d7a89 perf(core): emit stream-end metrics to expose post-header query cost (#1407)
* perf(core): emit stream-end db metrics after body streaming completes

Server-Timing db.* counters are snapshotted when middleware's next()
returns, but Astro streams the body afterwards and components issue
more DB queries that headers can never report (85-320ms of hidden
post-header query time measured in production).

When query instrumentation is enabled (EMDASH_QUERY_LOG=1), pipe the
rendered response body through an identity TransformStream and emit a
final [emdash-stream-end] NDJSON snapshot (db count/total/offsets,
cache hits/misses, total elapsed) in flush(), once the body finishes
streaming. The metrics object is mutated in-place by the Kysely log
hook, so the flush-time read observes every post-header query. The
wrapper forwards the astro.cookies symbol and drops Content-Length;
it is a no-op when instrumentation is off, the body is null, or no
request metrics are attached.

The query-counts harness now parses the new prefix and prints a
per-route stream-end report (informational only; snapshot files are
unchanged since timings are machine-dependent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: user-facing changeset wording

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:16:57 +01:00
Mohamed Hassan 28432b9b5a feat: extensible bylines (#1258)
* feat(bylines): types and storage interfaces for custom fields

* feat(bylines): migration 041 for custom field tables

* test(migrations): assert byline-field tables in dialect-compat fresh-run

* feat(bylines): BylineSchemaRegistry with version counter

* feat(bylines): per-isolate field-defs cache + request-cache invalidation helper

* feat(bylines): atomic version bumps around schema mutations for cache coherence

* feat(bylines): hydrate customFields in BylineRepository

* feat(bylines): batched customFields hydration for getBylinesForEntries

* feat(bylines): registry helpers, error codes, and reorder slug reservation for admin API

* feat(bylines): zod schemas for byline custom-field admin API

* feat(bylines): handler layer for byline-fields and update routes

* feat(bylines): admin API routes for byline custom fields

* test(bylines): admin API coverage for byline custom fields

* fix(admin): RTL-safe spacing, Lingui placeholder, error states for byline-schema

* test(admin): byline-schema permission + sidebar visibility coverage

* feat(admin): API client for byline custom-field schema

* feat(admin): register /byline-schema route

* fix(bylines): editors can read byline field defs via schema:read

* feat(admin): custom field inputs in byline edit form

* test(admin): byline edit form forwards customFields on save

* test(e2e): byline custom fields round-trip + changeset + query-counts snapshot

* feat(bylines): accept customFields on POST create route

* feat(admin): inline byline custom fields and surface schema link in page header

* refactor(admin): drop Byline Schema entry from sidebar

* fix(bylines): translatable hydration, url scheme, atomic create+update, D1 recovery

* fix(bylines): parity-aware dirty + always-advance clean for the field-defs cache

* fix(admin): unify byline-fields cache key and harden custom-field inputs

* ci: update query-count snapshots

* fix(bylines): qualify options.value in version SQL for postgres

* fix(bylines): lint

* fix(bylines): e2e test fix

* Update packages/core/src/database/repositories/byline.ts

Co-authored-by: ask-bonk[bot] <249159057+ask-bonk[bot]@users.noreply.github.com>

* fix(bylines): restore success return in coerceFieldValue url case

* feat(bylines): cache field-defs promise to coalesce concurrent reads

* style: format

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
Co-authored-by: ask-bonk[bot] <249159057+ask-bonk[bot]@users.noreply.github.com>
Co-authored-by: Matt Kane <mkane@cloudflare.com>
2026-06-03 19:35:42 +01:00
Matt Kane cd2dcc6a56 perf: resolve byline avatar storage key during hydration (#1298)
* perf: resolve byline avatar storage key during hydration

Byline content-credit hydration now LEFT JOINs the media table so
entry.data.bylines carry avatarStorageKey/avatarAlt, letting list pages
build a direct avatar URL without a per-author MediaRepository.findById
(an N+1). The fields are additive/optional and null on the plain byline
finders. Inferred author bylines (findByUserIds) get the same join.

Also adds opt-in seed support for byline avatars (bylines[].avatar,
hermetic: no download) and a perf-fixture route plus query-count
snapshots measuring the optimized vs naive per-avatar-lookup paths.

* fix: harden seed avatar validation and cleanup per PR review

- deleteMediaRow is now best-effort (logs+swallows) so cleanup failure
  can't mask the original byline-write error (emdashbot)
- reject leading/trailing whitespace on avatar storageKey/filename/mimeType,
  which are used verbatim in the media lookup (Copilot)
- correct docstrings: storage keys aren't constrained to {ulid}{ext}, and
  the new BylineSummary fields are null (not absent) on plain finders (Copilot)
2026-06-02 20:12:39 +01:00
Matt Kane 4f49c20e05 fix(deps): bump catalog to @astrojs/cloudflare 13.5.3 + @astrojs/node 10.1.1 (#1116)
* fix(templates): pin @astrojs/cloudflare to 13.5.1

13.5.2 introduced an astroFrontmatterScanPlugin that breaks any
`virtual:*` module import inside .astro frontmatter (e.g. astro-iconset,
astro-expressive-code). 13.5.0/13.5.1 also expose a cascade-reload bug
in workerd that surfaces as "Astro is not defined" or
"chunk-XXX.js does not exist" on the first request after a clean cache,
unless every transitive dep is pre-bundled.

13.5.1 escapes both: no frontmatter scan, and its dep-resolution fixes
keep the cascade quiet for the deps emdash actually uses. Pinning the
catalog to exact `13.5.1` (no caret) propagates to all four
`templates/*-cloudflare/package.json` files via the catalog sync, so
`create-emdash` scaffolded sites install a known-good combination.

13.5.1 declares peerDependencies.astro: ^6.3.0, so bump the catalog
astro range to ^6.3.0 to satisfy it.

Add a picomatch override to ^4.0.4 so the workspace's transitive
graph dedups to a single version. Without it, `@rollup/pluginutils`
pulls picomatch@4.0.3 on a sibling path that the adapter's
`astro > picomatch` pre-bundle doesn't cover, and workerd hits raw
CJS at module load. End-user installs happened to dedup correctly,
but local dev in this repo did not.

* fix(playground): bind SESSION KV id for wrangler preview deploys

@astrojs/cloudflare 13.2+ writes the auto-provisioned SESSION binding
into both the top-level config and previews.kv_namespaces of the built
wrangler.json, but with no id field. wrangler preview reads from
previews.kv_namespaces and forwards namespace_id verbatim; unlike
wrangler deploy it has no provisionBindings path. The API rejects the
binding-without-id with error 10021.

Declare the SESSION KV id explicitly in the playground's wrangler.jsonc
at both locations so the adapter's customizer skips re-injecting and
the user-supplied id propagates to both. Uses the existing
emdash-playground-session namespace already bound to the production
worker.

* fix(deps): bump @astrojs/node catalog to ^10.1.1

@astrojs/node@10.0.0 calls app.getAdapterLogger() at server start,
which doesn't exist on the new Astro v6 App class. Crashes any node
adapter consumer (including fixtures/perf-site, used by the query-
counts CI workflow):

  TypeError: app.getAdapterLogger is not a function
      at createAppHandler
      at createStandaloneHandler

@astrojs/node@10.1.1 updates to the v6 Adapter API. The astro@^6.3.0
catalog bump that #1116 introduced (required by @astrojs/cloudflare
13.5.1) made this surface.

* ci: update query-count snapshots

* fix(deps): bump @astrojs/cloudflare to 13.5.3, drop picomatch override

Replaces the exact 13.5.1 pin with ^13.5.3, the upstream-shipped fix
(withastro/astro#16801) that reverts the broken esbuild scan plugin
from 13.5.2.

The picomatch override is also no longer needed. The override was
defending against a workspace-only dual-resolve (picomatch@4.0.3 and
@4.0.4 both installed). With the catalog now on @cloudflare/vite-plugin
^1.36.3 and @astrojs/cloudflare 13.5.3, transitive consumers dedup
cleanly to a single 4.0.4 and the workerd dev runtime no longer hits
the unbundled CJS path.

Temporarily exempts astro and @astrojs/cloudflare from
minimumReleaseAge so the catalog can pick up the fix versions that are
still inside the 24h supply-chain cooldown. The exclusions should be
removed once the cooldown passes.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-05-20 15:23:13 +01:00
Matt Kane 792f73c12c chore: bump pnpm to 11.1.3 and pin scaffolded sites to a recent pnpm (#1115)
- Bump root packageManager via `corepack use pnpm@latest`.
- Sync script bakes the root's packageManager into each template's
  package.json, so scaffolded sites auto-track the monorepo pin.
- create-emdash strips packageManager when the user picks npm/yarn/bun
  so corepack doesn't force pnpm on a non-pnpm user.
- Drop dead `pnpm.onlyBuiltDependencies` from demo/fixture/template
  package.json files; pnpm 11 ignores `package.json#pnpm` and the root
  `allowBuilds` already covers these binaries.
- AGENTS.md / auto-implementer.md: drop `--silent` from lint commands;
  pnpm 11 prints the `$ command` line to stderr, so JSON pipes cleanly
  without it.
2026-05-20 08:13:53 +01:00
Matt Kane 6e62b90e14 fix: resolve #1053 type errors for strict consumers (#1076)
* fix: resolve #1053 type errors for strict consumers

Closes #1053.

Two parts:

1. wordpress-plugin.ts: the analyze-endpoint error body from
   response.json() is unknown under @cloudflare/workers-types; narrow
   it before reading .message (was the reported TS18046). The other
   reported error (byline.ts kysely Transaction variance) was TS5.x
   behaviour, resolved by the TS6 upgrade in #1074.

2. Stop shipping raw .ts for the source-exported subpaths
   (emdash/routes/*, emdash/api/route-utils, emdash/api/schemas,
   emdash/auth/providers/*). They are compiled to dist (.mjs +
   .d.mts), so a strict consumer's tsc only ever sees declarations
   (skipLibCheck covers them), eliminating the dual-package Database
   identity wall that is #1053's root cause. ./ui and .astro stay
   source (the consumer's Astro build must process them).

   - tsdown: route entrypoints fed via inputOptions.input (literal
     object, not entry globs -- [param] dirs are glob char-classes).
     entryFileNames + resolveRoute share one routeArtifactName() so
     rolldown's reserved [name]/[hash] placeholders cannot mangle
     dynamic-route artifacts.
   - A fast static guard (scripts/typecheck-public-source.mjs, wired
     into CI) fails if any subpath export ships raw .ts/.tsx again.

Verified: pnpm build, pnpm typecheck, demos+templates typecheck,
demo build (route injection e2e), lint baseline unchanged.

* style: format

* fix(build): externalize self/optional deps so route entries don't bundle them

Compiling the route/admin entries made tsdown try to bundle deps it
could not resolve at build time -- 'emdash' (the package importing
itself) and '@cloudflare/kumo' (admin-only, not an emdash dep). CI
escalates tsdown's bundling advisory to a fatal error.

- Externalize 'emdash' (self): compiled routes import it; resolved at
  the consumer's runtime where the package is installed.
- Externalize @aws-sdk/* (optional S3 deps, runtime-only).
- Keep the *-admin.tsx providers as source (bridge the admin React +
  @cloudflare/kumo runtime, like .astro/./ui); revert their exports.
- inlineOnly: false -- nothing is unintentionally bundled (all deps
  external, only our own src is); silences the CI-escalated advisory.

Guard's allowlist generalized to RUNTIME_COUPLED (Astro + admin React).

* test: route-injection entrypoint now resolves to compiled artifact

resolveRoute resolves emdash/routes/* to the compiled dist artifact
with routeArtifactName applied ([ ] -> _), so the media catch-all
route's entrypoint is api/media/file/_...key_ not [...key].ts. The
catch-all pattern assertion (the actual guarantee) is unchanged.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-05-17 13:02:35 +01:00
Matt Kane 40579b1fdc Per-template AGENTS.md with template-specific guidance (#1071)
* Per-template AGENTS.md with template-specific guidance

Each template's AGENTS.md was previously a verbatim copy of
templates/starter/AGENTS.md, so portfolio, blog, marketing, and starter
all carried the same generic platform-level rules and nothing about what
made each template distinct. An agent reading portfolio-cloudflare/AGENTS.md
had no way to know it was a portfolio template, what CSS variables drove
the design, or what not to touch.

Restructure the sync so each template can carry its own design and
schema guidance without duplicating the shared rules:

- scripts/agents-base.md is the shared base (commands, key files, skills,
  docs MCP, platform rules), extracted from the old starter/AGENTS.md.
- templates/{name}/AGENTS-template.md is the per-template body, authored
  by hand. Covers what the template is, its pages, its schema, its visual
  character, which CSS variables matter, and what not to do.
- scripts/sync-template-skills.sh concatenates base + body to produce
  each templates/{name}/AGENTS.md. The *-cloudflare variants fall back
  to the base variant's body, so portfolio-cloudflare and portfolio get
  identical AGENTS.md.
- scripts/sync-templates-repo.mjs excludes AGENTS-template.md from the
  public emdash-cms/templates mirror; only the assembled AGENTS.md ships.

All four themed templates plus blank get template-specific bodies. The
content is declarative facts about each template, useful to both local
coding agents (Claude Code, Cursor) and the chat-harness BuilderAgent
in emdash-build.

* style: format

* Address review: portfolio gallery shape, marketing hero image

- portfolio: clarify that the projects.gallery JSON field expects
  { url, alt? } records, not EmDash image objects. Distinguish it
  from featured_image to avoid the footgun.
- marketing: remove the inaccurate "hero image" mention from the
  hand-entered URL fields list. The marketing.hero block has no
  image field in the editor schema; the renderer falls back to the
  bundled /hero-visual.svg. Document where to swap that.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-05-17 09:18:27 +01:00
Matt Kane f28eb7b532 fix(templates): drop trustPolicy: no-downgrade (#1067)
no-downgrade fires on provenance/trust regressions anywhere in the
transitive graph (chokidar@4.0.3, @portabletext/toolkit, ...) -- deps
the template author doesn't control. Each blocks every scaffolded
install and needs its own exclusion. Untenable; remove the policy and
its exclude. Keep allowBuilds, minimumReleaseAge, blockExoticSubdeps.
2026-05-16 20:53:49 +01:00
Matt Kane 84d75b174c fix(templates): exclude chokidar@4.0.3 from trustPolicy; trim comments (#1066)
Follow-up to #1063. That PR landed the base pnpm-workspace.yaml
generation but not these two changes: without the chokidar exclusion
every scaffolded install fails trustPolicy: no-downgrade, because
Astro pins chokidar@4.0.3 whose npm publish provenance regressed.
Narrow, version-pinned, self-expiring (stops matching once Astro
upgrades chokidar). Also trims the over-long generated comments.
2026-05-16 17:44:50 +00:00
Matt Kane 796bd18e78 fix(templates): generate pnpm-workspace.yaml; 2-space package.json (#1063)
The standalone templates' build-script policy lived in package.json
`pnpm.onlyBuiltDependencies`, which pnpm 11 removed (replaced by
`allowBuilds`). With pnpm >=10.26/11 and `strictDepBuilds` on by
default, scaffolded sites failed `pnpm install` with
ERR_PNPM_IGNORED_BUILDS (better-sqlite3, esbuild, sharp, workerd).

The sync now synthesizes a per-template `pnpm-workspace.yaml` in the
standalone repo (it can't be committed to source: templates/* are
members of this monorepo's workspace, and a nested pnpm-workspace.yaml
would break workspace:* resolution -- the reason it was removed
downstream). It carries:

- allowBuilds: only the native builds each runtime needs are true;
  the rest are false so strictDepBuilds treats them as reviewed.
  cloudflare: esbuild/workerd; node: esbuild/better-sqlite3/sharp.
- minimumReleaseAge 1440 (emdash/@emdash-cms/* excluded)
- trustPolicy: no-downgrade
- blockExoticSubdeps: true

Also: the sync wrote package.json with tabs; emit 2-space (npm/pnpm
convention, and package.json is formatter-ignored). Drop the now-dead
`pnpm.onlyBuiltDependencies` from the 8 synced template package.json
and normalize them to 2-space. pnpm-workspace.yaml is added to the
prune-preserve set so the dest-clean step doesn't drop it.

Works on pnpm >=10.26 and pnpm 11 with no packageManager pin.
2026-05-16 16:42:16 +00:00
Matt Kane fbed118b23 chore(scripts): commit perf query-dump tooling (#868)
Adds the query-dump harness and analysis scripts that produced the
catalogue used to drive PRs #838, #839, #840:

- scripts/query-counts-dump.mjs — sibling of query-counts.mjs that
  emits per-route × phase NDJSON dumps under scripts/query-dumps/.
- scripts/build-perf-d1.mjs — standalone "build the perf fixture for
  d1" wrapper, useful when iterating with the dump harness without
  re-running the full counts harness.
- scripts/query-dumps/{classify,cold-only,inspect-other}.mjs — analysis
  helpers for slicing the dumps. classify.mjs writes a markdown
  classification report; cold-only.mjs surfaces the d1 cold-isolate tax;
  inspect-other.mjs prints distinct SQL for a class.
- scripts/query-dumps/README.md — workflow doc.

The dump JSON itself and the generated classification.{target}.md
reports stay gitignored — they're analysis artefacts that regenerate
from the harness in seconds. The query-dumps analysis scripts are
excluded from oxlint (one-off tooling, not production code).
2026-04-30 20:17:10 +01:00
all3f0r1 a9c29ea584 fix(redirects): fire for unauthenticated visitors (#817)
* fix(redirects): fire for unauthenticated visitors (#808)

The redirect middleware bailed when `locals.emdash.db` was missing, which is
the intentional state for public visitors -- so 301/302 rules from
`_emdash_redirects` only fired for logged-in admins, edit-mode sessions and
preview tokens. WordPress migration redirects, manual rewrites and the
`Auto: slug change` rows did nothing for real traffic, and `hits` /
`_emdash_404_log` stayed at zero.

Falls back to `getDb()` (ALS-aware -- returns the per-request scoped session
when one is active, the singleton otherwise) when `locals.emdash.db` is
absent. Same accessor the loader and template helpers use, so the public
render boundary stays minimal.

Adds a regression test covering exact and pattern matches on the
public-visitor branch, plus the existing authenticated branch and the
"no db configured" fallback.

* ci: update query-count snapshots

* perf(redirects): cache exact matches alongside patterns

One query loads both kinds at cold-start; warm requests issue zero
queries. Exact rules indexed by source path in a Map (O(1) lookup),
pattern rules pre-compiled into an array. Empty-redirect sites cache
an empty Map + array, eliminating the per-request probe.

Addresses ascorbic's review on #817: snapshot was showing +1 query on
every public request because findExactMatch ran uncached while pattern
lookups already went through the module-level cache.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: update query-count snapshots

* ci: update query-count snapshots

---------

Co-authored-by: Matt Kane <mkane@cloudflare.com>
Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 14:26:05 +00:00
ask-bonk[bot] e73bb5f3b5 Remove blank template from CLI (#852)
Co-authored-by: ask-bonk[bot] <ask-bonk[bot]@users.noreply.github.com>
Co-authored-by: ascorbic <ascorbic@users.noreply.github.com>
2026-04-30 14:43:31 +01:00
Matt Kane 64bf5b9812 perf(core): dedupe taxonomy-def and posts-list fetches per request (#840)
* perf(core): dedupe taxonomy-def and posts-list fetches per request

Two intra-request duplicate-fetch patterns showed up on the perf-fixture
post-detail render:

1. `_emdash_taxonomy_defs` was fetched three times per render — once
   unfiltered (during entry-term hydration) and twice with `WHERE name=?`
   (one per taxonomy widget). Have `getTaxonomyDef(name)` peek the
   already-cached full list before falling through to a narrower query.

2. `getEmDashCollection("posts", { limit: N })` was issued three times at
   different small limits (4 from the page body, 5 from RecentPosts, no
   limit from Archives) — each duplicate dragged byline + term hydration
   along with it. Bucket small limits up to a shared minimum (10) so
   sibling widgets at slightly different sizes share one fetch + slice
   to their requested size in the wrapper. Cursor-paginated calls are
   exempt; nextCursor is recomputed from the slice boundary.

Trade-off: bucketed fetches over-fetch up to ~10 entries (and the +1
over-fetch the loader already does) for callers who would otherwise have
asked for fewer. On the perf fixture this nets out to fewer queries
overall because the bucket is shared. For a page with a single small-
limit query and no sibling widgets the bucket fetches more rows than
strictly needed, but stays at the same query count.

Cuts the post-detail snapshot from 33 to 27 queries (sqlite target).

* ci: update query-count snapshots

* fix(core): align bucket-slice cursor with loader's encoding

Three fixes for the bucket-then-slice path in getEmDashCollection:

- encodeEntryCursor now skips invalid orderBy field names via the same
  FIELD_NAME_PATTERN check the loader's getPrimarySort uses, so a malformed
  first key falls back to created_at instead of producing a cursor against
  a phantom column.
- For date columns, encodeEntryCursor now reads the raw stored string from
  a hidden CURSOR_RAW_VALUES symbol the loader stashes on each entry,
  rather than round-tripping through new Date().toISOString(). The
  round-trip mints `.000Z` for ISO strings without milliseconds, which
  would lex-mismatch the stored value and re-include the cursor row on
  the next page.
- sliceCollectionResult docstring updated to match the actual behavior
  (early-return for entries already within the limit, shallow-copy only
  on truncation).

Adds tests pinning the bucket → slice → re-paginate round-trip, including
exact cursor equivalence with a direct loader call at the same limit.

Resolves Copilot review comments on PR #840.

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-04-30 11:02:09 +01:00
Matt Kane 0d98c620a5 perf(core): cache site:* settings prefix scan across requests (#839)
* perf: cache site:* settings prefix scan across requests

Site settings change rarely but were re-fetched on every public page
load. A globalThis-scoped holder caches the resolved settings until the
next site:* write bumps its version, dropping one prefix-scan query per
public route. Cross-isolate staleness is bounded by isolate lifetime.

* fix(core): invalidate site-settings cache on partial-write failure

OptionsRepository.setMany iterates row-by-row without a transaction. If
a write fails mid-loop, earlier rows are committed but
invalidateSiteSettingsCache() never ran — leaving the globalThis cache
pointing at pre-write data while the DB has post-write data on some
rows. Wrapping the call in try/finally guarantees invalidation runs
whether the write succeeded or threw.

Adversarial-review finding from PR #800-class perf branch.

* ci: update query-count snapshots

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-04-29 22:04:05 +00:00
Matt Kane c22fb3a10d perf(core): drop redundant author_id lookup after list fetch (#838)
* perf(core): drop redundant author_id IN lookup after list fetch

The byline hydration helper used to refetch `author_id` from the content
table for every entry without explicit byline credits, even though the
loader already exposes `data.authorId` on each row. Pass it through from
the call site so the inferred-byline fallback uses the value already in
hand. Saves ~30 queries across the perf-fixture suite, including 4 on
post-detail.

* ci: update query-count snapshots

---------

Co-authored-by: emdashbot[bot] <emdashbot[bot]@users.noreply.github.com>
2026-04-29 22:47:22 +01:00
Matt Kane 54f6283888 fix(ci): query-counts-apply workflow no longer silently fails (#828)
* fix(ci): query-counts-apply workflow no longer silently fails

The Apply workflow has been silently broken for ~2 days, never pushing
auto-updated snapshots to PRs that drifted. Two distinct bugs:

(1) Same-repo PRs: the artifact was unpacked into
    $GITHUB_WORKSPACE/artifact, but the later actions/checkout step
    defaults to clean: true and wipes the workspace, deleting our
    payload before the Apply step's cp could read it. Stage under
    $RUNNER_TEMP/qc-artifact instead, which sits outside the workspace.

(2) Fork PRs: the cross-check used listPullRequestsAssociatedWithCommit
    against the base repo, which doesn't reliably surface fork PRs from
    that view, so valid fork artifacts were rejected as 'not associated
    with commit'. Replace with pulls.get(prNumber) and verify
    pr.head.sha === workflow_run.head_sha — equivalent tamper-resistance
    (a forged pr-number won't match), works for forks.

Add pull-requests: read so pulls.get works when the default token
permissions are restricted.

* ci: reconcile query-count snapshot baseline

The Apply workflow has been silently broken for ~2 days, so PRs that
landed in that window (#820, #816, #811) merged with stale snapshots
in the repo. CI's Measure on this PR shows the real per-route counts
on top of current main: uniform +1 cold and +1 warm on all public
routes except /pages/about and /rss.xml. This is pre-existing drift,
not caused by this PR — it's the natural consequence of the Apply
workflow failing to commit the auto-updated snapshots that the
Measure step correctly produced on those merged PRs.

Apply CI's regenerated snapshots from this PR's Measure run so main
ends up in a consistent state once this lands. Future PRs will then
get their own real drift signal rather than inheriting this baseline
correction.
2026-04-29 16:47:26 +01:00
Matt Kane 0896ec8106 perf: drop has-any probes, join widget_areas + widgets (#659)
* perf: drop has-any probes, join widget_areas + widgets

Two query-count reductions on the request hot path:

- Widget areas now fetch in one query instead of two — area lookup +
  widget fetch collapse into a single left join. Saves one query per
  <WidgetArea> rendered.

- Drop the hasAnyBylines / hasAnyTermAssignments probes. They fired
  on every hydration call to save a single query on sites with zero
  bylines/terms — exactly backwards: the batch join already handles
  empty sites for the same cost. Pre-migration DBs (tables missing)
  are still handled via an isMissingTableError catch. Saves two
  queries per render on pages that hydrate both.

invalidateBylineCache and invalidateTermCache are kept as no-op
exports so existing callers (seed apply, admin routes) compile
unchanged.

Snapshot deltas on the fixture post-detail page:
  sqlite /posts/[slug] cold: 34 -> 32, warm: 34 -> 32
  d1     /posts/[slug] cold: 43 -> 39, warm: 33 -> 31
Other routes: -1 each from the widget JOIN.

* address Copilot review on the perf cleanup

- widgets: include `created_at` in the left-join select and the
  constructed WidgetRow, so the cast is actually sound. No other
  caller needs additional fields but WidgetRow declares created_at
  as non-null.
- taxonomies: drop the stale "no assignments short-circuit" claim
  from the getAllTermsForEntries doc comment — that behavior was
  removed with the probe.
2026-04-19 13:56:28 +01:00
Matt Kane 03847341b0 ci: auto-update query-count snapshots instead of failing (#655)
* ci: auto-update query-count snapshots on PR instead of failing

Three workflows replace the single fail-on-drift job:

1. Query Counts (pull_request) — runs the harness, uploads the
   regenerated snapshots as an artifact if they drift. Always passes,
   runs with read-only permissions and PR-authored code.

2. Query Counts — Apply (workflow_run) — triggered when the measure
   workflow completes. Downloads the artifact, cross-verifies the PR
   number against the workflow_run's head SHA to guard against a
   tampered artifact, and pushes the regenerated snapshots back to the
   PR branch (same-repo directly, forks via GIT_ASKPASS). Never runs
   PR-authored code, so it can hold the app token safely.

3. Query Counts — Label (pull_request_target, paths-filtered) —
   applies the "query-count changed" label when a PR diff touches
   either snapshot file (either because the author changed them, or
   because workflow 2 just auto-pushed).

* ci(query-counts): fail if fork push fails

If the bot can't push snapshots to a fork PR (most likely because the
contributor has 'Allow edits by maintainers' disabled), fail the Apply
workflow loudly instead of silently succeeding. That way the PR shows
a red check and the reviewer knows to either ask the contributor to
enable maintainer edits or run the harness themselves.

* ci(query-counts): fix artifact upload and eliminate false-positive drift

Two issues in the first run on main:

1. actions/upload-artifact treats dirs starting with a dot as hidden and
   skips them when include-hidden-files is false (the default), so the
   staging dir .query-counts-out/ uploaded zero files and failed the
   step. Renamed to query-counts-out/ (and gitignored).

2. The harness wrote snapshots with JSON.stringify's default 2-space
   indent. oxfmt then reformatted the committed files to tabs (per the
   repo's prettier config), which produced a permanent whitespace diff
   every time CI regenerated the files. The drift check interpreted
   that as a real count change. Switched the harness to tab indent so
   its output matches the formatted file verbatim.

* ci(query-counts): address Copilot review

- Add actions:read to the Apply workflow's permissions so the API
  calls to list and download artifacts are authorised. With explicit
  permissions, everything not named becomes none.

- Check out the measured SHA from workflow_run.head_sha instead of
  the branch ref. If the PR branch has advanced between measure and
  apply, push HEAD:ref would fail non-fast-forward instead of
  silently applying stale snapshots to a newer tree. The next PR
  event kicks off a fresh cycle against the new head.

- Keep client-id (not app-id) for create-github-app-token — app-id
  is deprecated in v3.1.1 in favour of client-id per the action's
  own action.yml. Other workflows in the repo will migrate at their
  next touch.
2026-04-19 08:44:21 +01:00
Matt Kane 943d54060e Dedup repeat DB queries within a single render (#654)
* perf: dedup repeat DB queries within a single render

Two classes of fix:

1. `hasBylinesSingleton` and `hasTermAssignmentsSingleton` were module-
   scoped, but the bundler duplicates those modules across chunks so
   each chunk got its own local singleton — the probe ran per chunk
   instead of per worker. Stored on globalThis with a Symbol key (same
   pattern as request-context.ts) so all chunks share one value.

2. Wrapped `getCollectionInfo`, `getTaxonomyDef`, `getTaxonomyTerms`,
   and `getEmDashCollection` in the request-scoped cache so two callers
   with the same args in the same render share one query.

Fixture snapshot deltas:
  sqlite post detail: 37 → 34 cold, 35 → 34 warm
  d1 post detail:    45 → 43 cold

* fix(query-cache): make collection cache key insertion-order-stable

JSON.stringify is sensitive to object key order, so two callers
passing semantically identical filters in different key orders
would miss the cache and fire duplicate queries. Build the key
from fixed top-level fields, sort where-clause keys (their order
is irrelevant), and preserve orderBy key order (that's the sort
priority — reordering would be semantically different).

Flagged by Copilot review.
2026-04-19 08:30:20 +01:00
Matt Kane f97d6ab0f1 Add query-count perf harness + instrumentation (#653)
* feat: add query-count perf harness + instrumentation

Opt-in Kysely log hook gated behind EMDASH_QUERY_LOG=1 emits per-request
NDJSON on stdout so a harness can count DB queries per route. Zero
overhead when disabled. Exposed at emdash/database/instrumentation so
@emdash-cms/cloudflare can wire the same hook into its per-request D1
session Kysely.

Adds fixtures/perf-site (minimal blog-style fixture, dual sqlite/d1
config), scripts/query-counts.mjs (pnpm query-counts), committed
snapshot files for both targets, and a CI job that runs both.

* fix(perf): invoke emdash CLI directly in query-counts harness

pnpm exec emdash fails in CI because bin symlinks aren't linked for
workspace-local packages (see scripts/relink-bins-if-needed.mjs, which
early-exits under CI). Invoke the built CLI entry by absolute path
instead so the harness works in both CI and local dev.

* ci(perf): build all packages for query-counts job

The fixture config imports from @emdash-cms/cloudflare for the d1 path,
so `pnpm run --filter emdash... build` (which only walks emdash's
deps, not its dependents) leaves cloudflare unbuilt and astro fails
to resolve the import when loading the config.

* fix(perf): wait for TCP port instead of parsing stdout for ready

The ready-regex approach was fragile — in CI, the cloudflare adapter's
dev mode wraps output in [vite] prefixes and the "ready in" line
sometimes never matches (observed on the D1 seed step: typegen POST
succeeded but ready timeout still fired).

TCP-connect is the real question anyway ("is the server accepting
connections?"). It also doesn't warm a fresh workerd isolate —
workerd defers isolate creation to the first HTTP request — so the
per-route cold-isolate measurement stays honest.

* fix(perf): seed D1 before building for preview

`astro dev` (the seed step) leaves .wrangler/deploy/ without the
build-time config.json that cloudflare adapter's preview requires, so
running `astro build` after the seed is what makes the subsequent
`astro preview` spins work.
2026-04-19 07:55:41 +01:00
Matt Kane 6410c81444 fix: preserve symlinks in template sync script
cpSync was dereferencing symlinks, causing "src and dest cannot be the
same" errors for CLAUDE.md -> AGENTS.md and producing broken absolute
symlinks in the templates repo. Also removes pnpm-workspace.yaml from
the preserved file set.
2026-04-12 22:29:30 +01:00
Matt Kane 10f9859250 fix: use --force instead of --force-with-lease for bot-managed branch
force-with-lease fails in shallow clones because the local branch is
created from main, not the fetched remote branch. Since only this
script writes to the sync branch, --force is safe and simpler.
2026-04-12 16:08:55 +01:00
Matt Kane 9e01479e98 fix: pass --head to gh pr create for temp dir clones
gh can't infer the head branch from a shallow clone to a temp dir.
2026-04-12 15:46:50 +01:00
Benjamin Price a378912697 fix: relink CLI bins after build only when needed (#482)
* fix: relink CLI bins after build only when needed

pnpm only creates bin symlinks for workspace packages when the target
file exists at install time. Since the CLI lives in dist/, it doesn't
exist until after the first build, so `emdash seed` fails for new
contributors. Add a postbuild script that detects missing or stale
bins and relinks only when necessary — zero overhead on normal builds.

* fix: rewrite relink script as Node for cross-platform support

Replace the shell script with a Node script so it works on Windows.
Also drop the mtime check — the bundler rewrites the CLI on every
build so it would trigger a spurious relink every time. Now only
relinks when the built CLI exists but the bin symlink is missing.
2026-04-12 14:59:08 +01:00
Matt Kane 1dc19b0f3a fix: restore worker.ts in cloudflare templates and add template sync (#478)
* fix: restore worker.ts in cloudflare templates and add template sync

Cloudflare templates were missing src/worker.ts and the "main" field in
wrangler.jsonc, so the custom worker entrypoint (Astro handler +
PluginBridge re-export) was never used. Also adds worker-configuration.d.ts
to the three templates that were missing it.

Adds scripts/sync-templates-repo.mjs to sync templates to the standalone
emdash-cms/templates repo with resolved dependency versions. A new
sync-templates workflow runs after publish or on manual dispatch.

Also fixes sync-cloudflare-templates.sh to use rsync with --exclude so
it preserves worker.ts when copying src/ from base templates.

* fix: lint error in sync-templates-repo.mjs

* fix: address review feedback on sync scripts

- Use gh repo clone instead of embedding token in clone URL
- Use execFileSync with arg array for gh pr create (no shell injection)
- Use -B and --force-with-lease for re-runnable branch push
- Guard against non-directory dest in sync-cloudflare-templates.sh

* fix: eliminate shell injection and harden sync workflow

- Replace all execSync/run() calls with execFileSync arg arrays so
  interpolated values (version strings from package.json) never pass
  through a shell
- Add ref guard (github.ref == refs/heads/main) to sync-templates
  workflow so it can't be dispatched from arbitrary branches
- Wrap post-clone logic in try/finally to clean up temp dir on error

* fix: address second round of review feedback

- Run gh auth setup-git after clone so git push works with GH_TOKEN
- Pin Node version in sync-templates workflow
- Handle dest-is-directory in file copy branch of sync-cloudflare-templates.sh

* fix: validate --local requires a path argument

* fix: handle re-runs gracefully (existing branch/PR)

- Fetch remote branch before force-with-lease so the lease has a ref
- Check for existing open PR before creating a new one
2026-04-12 11:44:56 +01:00
Matt Kane 43fcb9a131 first commit 2026-04-01 10:44:22 +01:00