Skip to content

title: "# pipeio_nb_exec: add kernel/env option and fix papermill resolution status: done created: 2026-04-07 updated: 2026-04-07 timestamp: 20260407-033705-872924 tags: [issue] source: agent-observation project_primary: projio capture_id: 20260407-033704-34b30f confidence: 1.0 transcript_file: /storage2/arash/worklog/workflow/captures/20260407-033704-34b30f/transcript.txt

pipeio_nb_exec: add kernel/env option and fix papermill resolution

Problem

pipeio_nb_exec fails when the project's PYTHON variable (from Makefile/projio.mk) points to a conda environment that doesn't have papermill installed. In pixecog, PYTHON resolves to the cogpy env, so cogpy/bin/python -m papermill fails with No module named papermill.

The kernel specified in notebook.yml is only passed as -k to papermill for cell execution — it doesn't influence which Python is used to run papermill itself.

Current behavior

  1. _resolve_project_python() reads PYTHON from Makefile → gets cogpy env binary
  2. Runs {python} -m papermill {ipynb} {output} -k {kernel}
  3. Fails because papermill is not in cogpy env

Implementation: packages/pipeio/src/pipeio/mcp.py lines ~4240-4245, helper at src/projio/mcp/pipeio.py lines 8-16.

Proposed fix

  1. Decouple papermill runner from notebook kernel. Papermill should be invoked from an environment that has it installed (e.g., the MCP server's own env or a configurable runner env), while the -k flag controls which Jupyter kernel executes the cells.

  2. Add optional env or kernel parameter to pipeio_nb_exec so callers can override the execution environment. If not provided, fall back to notebook.yml's kernel, then PYTHON, then system Python.

  3. Validate papermill availability before attempting execution — return a clear error message ("papermill not found in {env}, install it or specify an env that has it") instead of a raw module-not-found traceback.

Workaround

Install papermill in the cogpy env: conda run -n cogpy pip install papermill

Source context: pixecog

PixEcog (pixecog): Neuropixels and ECoG dataset and analysis

Recent commits: 

9b2f6fa Scaffold ecephys TTL removal mod, flow overview + mod docs, demo notebook
80194af Add TTL characterization & removal demo notebook (preprocess_ieeg)
dc93496 Update mkdocs pipeline nav

README:

type: readme

Quick Start for Collaborators

Follow this checklist to get started with Pixecog documentation and workflows.

🐀 Pixecog Project — Compact Overview

Core principles

  • One immutable BIDS raw dataset (raw/) as the canonical baseline
  • Each analysis pipeline ha