clawlter/llm-wikipedia
Watch
1
0
Fork
You've already forked llm-wikipedia
 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

fix: tolerate malformed wiki frontmatter #5

Merged
clawlter merged 1 commit from fix/resilient-frontmatter-health into main 2026-06-10 10:54:59 -04:00
Conversation 0 Commits 1 Files changed 10 +129 -50
clawlter commented 2026-06-10 10:51:15 -04:00
Owner
Copy link

Summary

  • skip markdown files whose YAML frontmatter fails to parse instead of aborting the entire wiki snapshot
  • report skipped files in /api/health as indexErrors
  • keep /api/health returning 200 when the wiki root is readable and at least valid pages can be indexed
  • remove the experimental /health alias because local production verification showed it could return misleading stale output; Docker and docs use /api/health

Root cause

The container healthcheck reached /api/health successfully. The failure was from gray-matter/js-yaml parsing llm-wiki frontmatter with an unquoted colon, for example:

title: 2026 Democratic Primary Voter Guide: Democratic primary candidates

That is invalid YAML unless quoted. Previously one malformed file made loadWikiSnapshot() throw, causing health to return 503.

Testing

  • npm test -- tests/wiki-store.test.ts
  • npm test -- tests/wiki-store.test.ts tests/health-endpoints.test.ts
  • npm run lint
  • npm run typecheck
  • npm test
  • NEXT_TELEMETRY_DISABLED=1 LLM_WIKI_ROOT=/opt/hermes-runtime/wiki/brain npm run build
  • npm audit --omit=dev
  • local production server with actual llm-wiki at /opt/hermes-runtime/wiki/brain: /api/health returned 200, pages: 306, indexErrors: 39
  • local Docker image runtime with copied malformed-frontmatter fixture: /api/health returned 200, pages: 1, indexErrors: 1

Note: Docker bind mounts in this sandbox surfaced the mounted test directories as empty, so actual content-in-container verification used docker cp into /tmp/wiki. The app-level local production server was verified against the actual llm-wiki data directly.

## Summary - skip markdown files whose YAML frontmatter fails to parse instead of aborting the entire wiki snapshot - report skipped files in `/api/health` as `indexErrors` - keep `/api/health` returning 200 when the wiki root is readable and at least valid pages can be indexed - remove the experimental `/health` alias because local production verification showed it could return misleading stale output; Docker and docs use `/api/health` ## Root cause The container healthcheck reached `/api/health` successfully. The failure was from `gray-matter`/`js-yaml` parsing llm-wiki frontmatter with an unquoted colon, for example: ```yaml title: 2026 Democratic Primary Voter Guide: Democratic primary candidates ``` That is invalid YAML unless quoted. Previously one malformed file made `loadWikiSnapshot()` throw, causing health to return 503. ## Testing - npm test -- tests/wiki-store.test.ts - npm test -- tests/wiki-store.test.ts tests/health-endpoints.test.ts - npm run lint - npm run typecheck - npm test - NEXT_TELEMETRY_DISABLED=1 LLM_WIKI_ROOT=/opt/hermes-runtime/wiki/brain npm run build - npm audit --omit=dev - local production server with actual llm-wiki at `/opt/hermes-runtime/wiki/brain`: `/api/health` returned 200, `pages: 306`, `indexErrors: 39` - local Docker image runtime with copied malformed-frontmatter fixture: `/api/health` returned 200, `pages: 1`, `indexErrors: 1` Note: Docker bind mounts in this sandbox surfaced the mounted test directories as empty, so actual content-in-container verification used `docker cp` into `/tmp/wiki`. The app-level local production server was verified against the actual llm-wiki data directly.
fix: tolerate malformed wiki frontmatter
All checks were successful
Build and publish container image / Build Docker image and publish package tags (pull_request) Successful in 1m42s
Details
CI / Validate formatting, linting, types, tests, and build (pull_request) Successful in 1m45s
Details
Build and publish container image / Build Docker image and publish package tags (push) Successful in 1m19s
Details
CI / Validate formatting, linting, types, tests, and build (push) Successful in 1m43s
Details
4a9144da18
 
Skip malformed markdown frontmatter during wiki indexing and report skipped pages through /api/health so content issues do not make the container unhealthy.
clawlter deleted branch fix/resilient-frontmatter-health 2026-06-10 10:54:59 -04:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
clawlter/llm-wikipedia!5
No description provided.