How mining works

The pipeline

git log + 0-inbox/*.md
        │
        ▼
  collect + label            per repo, per note, per bullet
        │
        ▼
  fetch mining-context       the refs already known, plus voice/pillars/persona
        │
        ▼
  drop known refs            the primary dedup, before any AI call
        │
        ▼
  your AI backend            claude CLI, or the Anthropic API
        │
        ▼
  POST /ideas/import         in chunks of 25

Steps 1 to 4 are what --dry-run shows you. Nothing is spent and nothing is pushed.

Refs, and why they are the identity

Every source item gets a stable ref:

Item	Ref	Example
A commit	<label>@<short-sha>	myrepo@a1b2c3d
A note	vault:<filename>	vault:2026-08-14-queue-bug.md
An inbox bullet	vault:content-inbox.md@<hash>	vault:content-inbox.md@3f9a21bc

Idea text is AI output. The same commits through two runs, or two models, produce different headlines. So text can never answer "have I already mined this". The ref can, because it identifies the source rather than the output.

Consequences worth knowing:

• Running mine twice in a row costs one API call and zero AI tokens.
• Re-running after adding one bullet mines that bullet only.
• Reformatting a bullet (- to *) keeps its ref, so it is not re-mined.
• Rewording a bullet changes its ref, so it is mined again. Correct: the idea changed.

Four layers of dedup

1. Before the AI call. Refs already attached to an idea, within a 90 day server-side window, are dropped from the work log. This is the one that matters. It works across machines and it saves tokens.
2. Citation. Each idea the AI proposes cites the refs it drew from. The CLI validates those against what it actually collected, and falls back to attributing the whole relevant section if a citation is missing or invalid.
3. The cap. Never more than maxIdeas pushed, whatever the model returns.
4. Server side. POST /ideas/import computes an idempotency key from the sorted refs plus the headline, so an overlapping window or a retry after a failed push updates the existing idea rather than duplicating it.

AI backends

Tried in order:

1. The claude CLI, if it is on your PATH. The prompt goes over stdin, never as a command-line argument, so raw commit text does not land in your shell history or the process table. It uses whatever model that CLI is configured with.

2. ANTHROPIC_API_KEY, calling the Anthropic Messages API directly.

3. Neither. It refuses to run:

No AI backend available. Either install the Claude CLI so `claude` is on PATH,
or set ANTHROPIC_API_KEY to call the Anthropic Messages API directly.

That refusal is deliberate. Silently falling back to something else with your commit text would be worse than stopping.

What the AI is asked for

The prompt carries your brand's persona, voice guide and content pillars, the filtered work log, and a summary of what is already covered or queued. It asks for JSON only, with:

Field	Meaning
idea	The headline
angle	Why it is worth posting
pillar	Which content pillar it fits
format	post, carousel or reel
recording	For a reel, concrete screen-recording instructions
refs	The refs it drew from

It is told at least two ideas should be reels, every reel needs a concrete recording note, and each idea must cite at least one verbatim ref.

There is also an explicit grounding rule: never claim you built something that only a note describes. Commits are evidence of work. Notes are evidence of thinking.

Inspecting a run

postlyra mine --dry-run --json

Gives you exactly what would be collected, what got filtered as already mined, and the work log the AI would see. Without spending a token or pushing anything.

Run it any time you are unsure why a run produced what it did.