Skip to content

Commit pipeio fixes + add input stage resolution to PipelineContext

Part 1: Commit existing fixes (already applied, just need commit)

Changes already made in this session, uncommitted:

packages/pipeio/src/pipeio/registry.py

  • Added find_registry(root) — public function that checks .projio/pipeio/registry.yml first, then .pipeio/registry.yml

packages/pipeio/src/pipeio/resolver.py

  • PipelineContext.from_registry() now uses find_registry() instead of hardcoded .pipeio/registry.yml

packages/pipeio/src/pipeio/mcp.py, cli.py, docs.py

  • Three duplicate _find_registry() implementations replaced with imports from pipeio.registry.find_registry

Verify all changes are correct, run tests if available, then commit.

Part 2: Add input stage resolution to PipelineContext

The Stage.resolve(sess, prefer) method only works with output registry groups. But notebooks need to resolve input data too — e.g. stage_prefer = ("raw", "interpolate", "linenoise", "preprocess") where "raw" is the input_dir, not an output group.

What needs to happen:

  1. PipelineContext should expose input stages (from input_dir + pybids_inputs in config) alongside output registry groups
  2. Stage.resolve() should be able to check input stage paths on disk
  3. The raw input stage should resolve paths using input_dir and the pybids_inputs filters (suffix, extension, datatype)

Design constraint:

  • Keep it simple — a "raw" stage just needs to resolve {root}/{input_dir}/sub-{subject}/ses-{session}/{datatype}/ paths using the pybids_inputs member definitions
  • Don't need full pybids integration — just path construction from entities + filters

Test:

ctx = PipelineContext.from_registry("preprocess_ieeg", root=repo_abs())
sess = ctx.session(subject="01", session="04", task="free")
# This should work:
stg = ctx.stage("raw")  # from input_dir
stg.have(sess)  # checks if raw files exist
stg.resolve(sess, ("raw", "interpolate", "preprocess"))  # returns first existing

After implementing, run the pipeio test suite to verify nothing breaks.

Source context: pixecog

PixEcog (pixecog): Neuropixels and ECoG dataset and analysis

Recent commits: 

9b2f6fa Scaffold ecephys TTL removal mod, flow overview + mod docs, demo notebook
80194af Add TTL characterization & removal demo notebook (preprocess_ieeg)
dc93496 Update mkdocs pipeline nav

README:

type: readme

Quick Start for Collaborators

Follow this checklist to get started with Pixecog documentation and workflows.

🐀 Pixecog Project — Compact Overview

Core principles

  • One immutable BIDS raw dataset (raw/) as the canonical baseline
  • Each analysis pipeline ha