Spec: Document new bib architecture¶

Goal¶

(promoted from note)

Context¶

(see source note)

Prompt¶

Fix the issue described below (source: docs/log/issue/issue-arash-20260402-220025-468258.md). Understand the problem, then implement the proposed fix.

Spec: Document new bib architecture — sources vs artifacts separation¶

Update docs/specs/ with the new bib/ architecture design:

Design Decision¶

All generated/shipped artifacts move to .projio/, bib/ holds only human-managed sources.

New layout¶

.projio/
  render.yml                    # config (human-edited, exists)
  render/
    compiled.bib                # biblio_compile output — THE single bib for pandoc + mkdocs-bibtex
    pandoc-defaults.yaml        # generated (move from bib/)
    csl/                        # shipped by projio, copied during sync
      apa.csl
  filters/
    include.lua                 # shipped by projio (exists)
  biblio/
    merged.bib                  # biblio_merge intermediate
    config/                     # biblio.yml, library.yml (move from bib/config/)
    logs/                       # merge/quality/duplicate logs (move from bib/logs/)
  pipeio/
    modkey.bib                  # pipeio_modkey_bib intermediate

bib/
  srcbib/           # Zotero exports (human-managed)
  articles/         # PDFs

Compilation pipeline¶

srcbib/*.bib  →  biblio_merge  →  .projio/biblio/merged.bib  ─┐
                                                                ├→  biblio_compile  →  .projio/render/compiled.bib
             pipeio_modkey_bib →  .projio/pipeio/modkey.bib  ─┘

render.yml changes¶

bibliography: .projio/render/compiled.bib
csl: .projio/render/csl/apa.csl
bib_sources:
  - .projio/biblio/merged.bib
  - .projio/pipeio/modkey.bib

Update: CLAUDE.md architecture section, docs/specs/ if a biblio or render spec exists, and the notio manuscript spec to reference compiled.bib.

issue-arash-20260402-015659-415628.md — Both concern biblio pipeline architecture — batch docling operates on the same source bib files managed under the new layout
issue-arash-20260402-121445-694073.md — Manuscript dual-render setup consumes compiled.bib and pandoc-defaults.yaml whose paths are redefined in this architecture spec
issue-arash-20260328-174906-119594.md — Biblio graph expansion is a downstream consumer of the merged/compiled bib artifacts whose locations this spec redefines
issue-arash-20260328-145700-791144.md — Graph migration likely involves biblio data paths that are being restructured under .projio/biblio/ in this spec
issue-arash-20260331-172245-820267.md — New bib architecture introduces path and tool changes that require MCP tools reference docs to be updated

Acceptance Criteria¶

[ ]

Result¶

Completed in 1363.6s (1 steps, model: opus)

Session: 5770b78c-f4c8-4794-9f73-aedd6692dafa Result note: /storage2/arash/worklog/workflow/captures/20260402-231301-465c3d/note.md

Output¶

All 6 tasks complete. Here's a summary of what was done:

**Task 1** — Created `docs/specs/biblio/bib-architecture.md` spec documenting sources vs artifacts separation, updated CLAUDE.md architecture section, updated manuscript spec to reference `compiled.bib`.

**Task 2** — Changed `mcp_modkey_bib()` default output from `docs/pipelines/modkey.bib` to `.projio/pipeio/modkey.bib` in both pipeio and the projio MCP wrapper.

**Task 3** — Moved biblio defaults: `out_bib` → `.projio/biblio/merged.bib`, `duplicates_log` → `.projio/biblio/logs/`, config → `.projio/biblio/biblio.yml`, library → `.projio/biblio/library.yml`, run ledger → `.projio/biblio/logs/runs/`, RAG config → `.projio/biblio/rag.yaml`. All paths have backward-compat fallback to legacy `bib/config/` locations.

**Task 4** — Added `compile_bib()` in biblio's `bibtex.py`, `biblio_compile()` MCP function in biblio's `mcp.py`, projio wrapper in `mcp/biblio.py`, and registered `biblio_compile` tool in `server.py`.

**Task 5** — Downloaded 4 CSL files (apa, ieee, chicago-author-date, vancouver) to `src/projio/data/csl/`, updated `pyproject.toml` package-data, added `_sync_csl_files()` to sync.py, updated render.py with `bib_sources` field, new defaults (`compiled.bib`, `csl/apa.csl`), and moved pandoc-defaults output to `.projio/render/`.

**Task 6** — Updated `ecosystem_status()` to use new merged/compiled bib paths with staleness checking, updated CLAUDE.md MCP tool list to include `biblio_compile`, updated `manuscript_cmd.py` render defaults.

## Next Steps
1. Run `projio sync` on an existing project (e.g. sirocampus) to test the CSL copy, pandoc-defaults generation at the new path, and verify backward compat with existing `bib/config/` files.
2. Add a `biblio_compile` step to `projio sync` so the full pipeline (`biblio_merge → modkey_bib → biblio_compile`) runs automatically during sync.
3. Update the biblio CLI help text strings (40+ references to `bib/config/biblio.yml`) to mention the new `.projio/biblio/

Batch Result¶

status: done
batch queue_id: 9ef168fb767a
session: 5770b78c-f4c8-4794-9f73-aedd6692dafa
batch duration: 1363.6s