Linting non-markdown facts with external sources

Gate facts that live outside markdown – rows in JIRA, pages in Notion, clauses traced in a Prolog knowledge base – with the same rules, dependency graph, and exit codes ctxgrd already applies to your ADRs and PRDs.

What a source is

ctxgrd only reads .md files with frontmatter from disk. A source is how you lint everything else: an executable script that emits document envelopes on stdout. ctxgrd runs it, parses its output, and lints those envelopes alongside your local markdown using the same rules and the same namespace configuration.

Reach for a source when the thing you want to gate cannot be expressed as a markdown file at all – an API-backed system of record, or a fact that exists by construction in some other file format. If the facts already live in .md files, you want a namespace with paths, not a source; see Configuring namespaces.

Prerequisites

• ctxgrd 2.0.0 or later (for the expect_min floor covered below).
• A ctxgrd.toml already at the project root. Run ctxgrd init first if you don’t have one.

1. Lay out the script

A source is a directory under sources/ containing an executable named run. The directory name becomes the source name:

mkdir -p sources/statute
touch sources/statute/run
chmod +x sources/statute/run

2. Emit document envelopes

run writes one JSON object per line to stdout – JSONL, not a JSON array. Each line needs an id, a body, and a location:

{"id": "STATUTE-1", "body": "# Clause 1\n\nTraced to entrepreneur.pl.", "location": "src/entrepreneur.pl#clause-1"}

Here is a real motivating shape: a project encoding a statute in s(CASP) needs to prove that every clause of the law has a citing rule in its knowledge base. The source walks the encoding and emits one envelope per traced clause:

#!/usr/bin/env bash
set -euo pipefail

for clause in $(list_traced_clauses.sh); do
  printf '{"id":"STATUTE-%s","body":"# Clause %s\n\nTraced to entrepreneur.pl.","location":"src/entrepreneur.pl#clause-%s"}\n' \
    "$clause" "$clause" "$clause"
done

The namespace derives from id alone – [STATUTE].paths never enters into it, because source-emitted documents don’t have a path on disk that you control. Configure [STATUTE] rules in ctxgrd.toml exactly as you would for any other namespace.

3. Activate the source

Discovery is automatic; running it is not. A source with no config table never executes:

[sources.statute]

An empty table is valid – the script receives CTXGRD_SOURCE_PARAMS='{}'. Add keys and they become the script’s parameters:

[sources.statute]
regulation = "1290/2002 §6"

4. Test the script directly

Run it standalone before wiring it into ctxgrd, so a malformed line is your problem to catch, not ctxgrd’s:

CTXGRD_SOURCE_NAME=statute CTXGRD_SOURCE_PARAMS='{}' sources/statute/run | jq .

jq . errors loudly on anything that isn’t valid JSON. Once every line parses, run ctxgrd --root . and watch it validate the envelopes with the rules you configured for [STATUTE].

5. Set a floor with expect_min

This is the step worth not skipping. Writing [sources.statute] is a statement that the source is expected to produce something – so ctxgrd holds it to a floor of one document by default, and warns if it comes back under that floor after running to completion:

warning[src.too-few-documents]: source 'statute' emitted 0 of an expected minimum of 1

This exists because a source that exits 0 without writing anything produces a run byte-identical to one where every clause passed – the gate disappears and the summary line says nothing changed. core.min-docs cannot catch this on its own: it counts markdown documents, and a source that emits 3 of 8 expected envelopes still satisfies “at least one.”

Set the floor to the number you actually expect. For the statute encoding above, that’s the clause count – eight clauses in Työttömyysturvalaki §6:

[sources.statute]
expect_min = 8   # one per clause of the statute

Now a partial emit warns by name:

$ ctxgrd --root .
warning[src.too-few-documents]: source 'statute' emitted 3 of an expected minimum of 8
  help: check the `run` script for source 'statute', or lower `expect_min` under [sources.statute] — 0 accepts a source that may legitimately emit nothing
  note: the source exited cleanly, so nothing else reports this — without the floor the run would be indistinguishable from one where every document passed

And a full emit lints clean:

$ ctxgrd --root .
ok: 8 documents · 4 rules · 0 diagnostics

Set expect_min = 0 for a source that may legitimately emit nothing – an optional feed, a system that’s sometimes empty by design. That opts out of the warning entirely:

[sources.optional-feed]
expect_min = 0

expect_min is ctxgrd’s own key. It’s stripped from the parameters before the script runs, so CTXGRD_SOURCE_PARAMS never carries it.

6. Cross-namespace dependencies still work

A source-emitted document participates in the same dependency graph as local files. STATUTE-1 can depends_on a local PRD-001, and core.dep-resolved validates the link exists regardless of which side is on disk and which came from a script.

7. Report a warning from the script

A source can print a warning line instead of an envelope. The line is a JSON object with a warning string and no id:

{"warning": "the cache is older than 24 hours; refresh it"}

ctxgrd reports it as warning[src.warning]: source '<name>': <text>. It is a warning only, so it never changes the exit code. It does not count toward expect_min. Use it when the source ran but you want the reader to know something, such as stale data. ctxgrd status prints the same message on stderr, so ctxgrd status --format json | jq stays clean. This needs ctxgrd 5.11.0 or later.

8. Start from the GitHub-issue source

The repository ships a working source for GitHub issues in examples/sources/github-issues/run. It lets an ADR wait on an issue: status shows the ADR as blocked until the issue is closed as completed. An issue closed as not planned keeps the ADR blocked.

The script reads a cache file and never calls the network. ctxgrd runs every source on each lint, status and turn-end hook, and it passes no arguments, so a lint can never call gh. You refresh the cache yourself.

1. Copy the script to sources/github-issues/run and make it executable. Add .ctxgrd/cache/ to .gitignore.

2. Configure the source, a namespace for the issues, and the status that settles a dependency on an issue:

   [sources.github-issues]
   # repo = "acme/app"     # default: the repository gh finds from the cwd
   # max_age_hours = 24    # warn when the cache is older
   # expect_min = 0        # if the repository may have no issues
   
   [GH]
   rules = ["core.id"]
   
   [ADR."core.dep-status"]
   terminal = ["accepted", "superseded"]
   satisfied-by = { GH = ["closed"] }

3. Fetch the issues. This is the only step that calls gh:

   sources/github-issues/run --refresh

4. Add GH-7 to an ADR’s depends_on, then run ctxgrd status.

Run --refresh whenever you want current issue state. Read ctxgrd docs sources for the full script text.

Troubleshooting

The source never runs. Check for a [sources.<name>] table in ctxgrd.toml – a script on disk with no config entry is inert by design.

error[src.runtime-error] – the script exited non-zero. ctxgrd reports it and keeps linting everything else; fix the script and rerun.

warning[src.too-few-documents] – the script ran to completion but returned fewer envelopes than expect_min. Either the script has a bug (a set -e that aborted an emit loop partway is the usual cause), or the floor is set higher than reality; fix whichever one is wrong.

warning[src.warning] – the script printed a {"warning": ...} line. Read the text; it says what the script wants you to fix. With the GitHub-issue source it means the cache is stale: run sources/github-issues/run --refresh.

error[src.runtime-error] from the GitHub-issue source – the cache file does not exist yet. Run sources/github-issues/run --refresh once.

Next steps

• Writing custom rules – the rule side of the same extension mechanism, for markdown documents.
• Configuring namespaces – paths, rules, and parameters in depth.
• How ctxgrd decides what to lint – why a namespace pointed at non-markdown ingests nothing, and routes you here.