Skip to content

DAG tool returns 0 edges for ecephys despite clear rule dependencies

pipeio_dag(pipe="preprocess", flow="ecephys") returns 14 nodes but 0 edges, and every rule appears in both roots and leaves. For ieeg, only 2 edges are found (raw_zarr → ttl_removal) despite 31 rules with extensive out_paths() cross-references.

Expected behavior

The DAG should resolve out_paths("group", "member") references to connect rules. For ecephys: - validate_inputs → downsample_for_feature (via out_paths("validate", "inputs")) - downsample_for_feature → feature (via out_paths("downsample", ...)) - feature → badlabel (via out_paths("feature", "zarr")) - etc.

Root cause

The DAG edge detection likely only matches rules.X.output.Y syntax (direct rule references) but not out_paths() / in_paths() helper function calls. Since pixecog uses out_paths() for all I/O routing, the DAG misses nearly all edges.

Impact

  • DAG visualization is useless for pixecog flows
  • pipeio_dag(target=...) ancestor filtering won't work
  • Agents cannot reason about rule ordering from DAG output

Suggested fix

Parse out_paths("group", "member") and in_paths("input", "member") expressions in rule input/output dicts. When rule A's output uses out_paths("X", "Y") and rule B's input uses the same call, add an edge A → B.

Source context: pixecog

PixEcog (pixecog): Neuropixels and ECoG dataset and analysis

Recent commits: 

c309f45 Fix pipeline doc naming drift, populate registry doc_path, close 3 issues
84d605b Migrate 43 scripts from utils.smk.smk_log
5808910 [DATALAD] removed content

README:

type: readme

Quick Start for Collaborators

Follow this checklist to get started with Pixecog documentation and workflows.

🐀 Pixecog Project — Compact Overview

Core principles

  • One immutable BIDS raw dataset (raw/) as the canonical baseline
  • Each analysis pipeline ha
  • [[issue-arash-20260330-031350-558222.md]] — Also a pipeio bug discovered in pixecog context; mentions ttl_removal which appears in the current note's source context
  • [[issue-arash-20260331-034216-830892.md]] — Co-discovered in the same session; both are pipeio tool correctness bugs (DAG edges vs config_patch formatting)
  • [[issue-arash-20260326-232732-019412.md]] — Tracks pipeio implementation work; DAG edge resolution is a core pipeio capability gap described here
  • [[issue-arash-20260326-191057-730671.md]] — pipeio_mod_resolve and out_paths/in_paths helpers are part of the same module routing system the DAG needs to parse