Skip to content

Fix registry_scan has_docs always false

Goal

(promoted from note)

Context

(see source note)

Prompt

Fix pipeio registry_scan so it correctly detects flow-local docs/ directories and populates has_docs and doc_path in the registry.

The bug: pipeio_registry_scan sets has_docs: false and doc_path: null for all flows, even when the flow has a docs/ directory with content. Mod-level doc_path should also be set when {flow}/docs/{mod}/ exists with theory.md or spec.md.

Steps: 1. Read the registry_scan implementation in packages/pipeio (likely src/pipeio/registry.py or similar) 2. Find where has_docs and doc_path are set during scanning 3. Fix the path resolution so it checks for {flow}/docs/ relative to the pipeline root 4. Fix mod-level doc_path detection for {flow}/docs/{mod}/ containing theory.md or spec.md 5. Test by running pipeio_registry_scan on the projio project itself

Acceptance Criteria

  • [ ]

Result

Completed in 331.5s (1 steps, model: opus)

Session: 43e7513d-7002-4b28-af9e-41b897fb07d7 Result note: /storage2/arash/worklog/workflow/captures/20260408-042820-66f97e/note.md

Output

Everything looks correct. Here's a summary of the changes:

## Changes Made

**`packages/pipeio/src/pipeio/registry.py`** — 2 bugs fixed:

1. **`_find_doc_path`** (line 349): Added `flow_dir` parameter. Now checks the flow-local `{flow_dir}/docs/` directory as a fallback when the external `docs_dir/{flow}` doesn't exist. External docs take precedence.

2. **`_discover_mods`** (line 330): Changed mod doc detection from looking for the legacy flat file `docs/mod-{mod_name}.md` to the faceted directory `docs/{mod_name}/` containing `theory.md` or `spec.md`.

3. **All three call sites** in `_discover_flows` (lines 225, 244, 260): Updated to pass `flow_dir` (was previously gated by `if docs_dir else None`, meaning no docs were ever detected without an external docs directory).

**`packages/pipeio/tests/test_registry.py`** — 2 new tests:
- `test_scan_with_flow_local_docs` — flow-local `docs/` detected without external docs_dir
- `test_scan_external_docs_preferred_over_local` — external docs_dir takes precedence

**`packages/pipeio/tests/test_mod_discovery.py`** — 1 updated, 2 new tests:
- Updated `test_discover_mods_with_doc_path` to use faceted `docs/{mod}/theory.md`
- `test_discover_mods_with_spec_only` — spec.md alone sets doc_path
- `test_discover_mods_empty_doc_dir_ignored` — empty mod dir doesn't set doc_path

All 40 registry + mod discovery tests pass.

## Next Steps

1. Run `pipeio_registry_scan` on a project with actual pipelines (e.g., pixecog/sirocampus) to confirm `has_docs: true` appears for flows with local docs.
2. Commit the fix in the pipeio subdataset and save via datalad.
3. Check whether `docs_collect` and `mod_context` correctly use the updated `doc_path` values from the registry.