Skip to content

Implement questio-iterate skill

Goal

Write the questio-iterate skill prompt that teaches the agent the iteration loop pattern — implement, run, evaluate, adjust based on human feedback.

Context

The loop mechanisms spec at docs/specs/research-orchestration/loop-mechanisms.md (section 3) defines the iterate loop. This replaces the rejected standalone questio-dispatch — iteration is always within a human feedback loop, not fire-and-forget.

Key difference from investigate: investigate is reactive (something went wrong), iterate is proactive (improving toward a goal).

Prompt

Step 1: Read the specs. - Read /storage2/arash/projects/projio/docs/specs/research-orchestration/loop-mechanisms.md section 3 (iterate loop) - Read /storage2/arash/projects/projio/docs/specs/research-orchestration/design.md sections 8.2-8.3 (inner loop, sequences) - Read existing skills for format reference

Step 2: Write the skill prompt at /storage2/arash/projects/projio/docs/specs/research-orchestration/skills/questio-iterate.md.

The skill should guide the agent through:

  1. Establish the goal: What are we trying to achieve? Link to questio milestone and flow:
  2. questio_gap(question_id) — what milestone are we working toward?

  3. What does "success" look like? (from grounding: literature values, prior results, human-stated criteria)

  4. Ground before first iteration:

  5. paper_context — expected methods, values, pitfalls
  6. codio_discover — existing implementations
  7. rag_query — prior attempts at this analysis

  8. Synthesize: approach, expected results, quality criteria

  9. Pre-flight check:

  10. pipeio_flow_status(flow) — is the flow ready?
  11. For expensive runs: present what will execute (targets, estimated scope) and get human confirmation

  12. For cheap runs (single notebook, single subject): proceed with note

  13. Execute:

  14. pipeio_run(targets) or pipeio_nb_exec(notebook) depending on scope
  15. Monitor: pipeio_run_status for pipeline runs

  16. On failure: enter investigate loop (reference questio-investigate skill)

  17. Evaluate:

  18. Read outputs: pipeio_target_paths → inspect files, pipeio_nb_read for notebook results
  19. Compare against grounded expectations
  20. Create observation note: what was run, what was produced, assessment

  21. Present results to human with: key metrics, comparison to expectations, recommendation

  22. Human feedback gate:

  23. Wait for human response before next cycle
  24. Possible feedback: "looks good" → record evidence, "adjust X" → modify and re-iterate, "investigate Y" → switch to investigate loop, "stop" → record current state and exit

  25. The agent does NOT autonomously decide to iterate again — the human directs

  26. Record:

  27. Each iteration: observation note (lightweight, tags=[observation, iterate])
  28. On success: result note via questio-record with full evidence schema
  29. On exit (human says stop): observation note summarizing what was tried and current state

  30. Propose milestone status update (propose-review-confirm, not auto-update)

  31. Convergence tracking:

  32. After 3+ iterations, the agent should summarize the trajectory: "iteration 1 gave X, iteration 2 gave Y, iteration 3 gave Z — the trend is [improving/plateauing/degrading]"
  33. If plateauing or degrading after 3 iterations: suggest changing approach rather than continuing to tune

Step 3: Commit with message: "Add questio-iterate skill template"

Acceptance Criteria

  • [ ] Skill prompt at docs/specs/research-orchestration/skills/questio-iterate.md
  • [ ] Human feedback gate is explicit — no autonomous re-iteration
  • [ ] Dry-run/confirmation for expensive runs
  • [ ] References questio-investigate for failure cases
  • [ ] Convergence tracking after 3+ iterations
  • [ ] Committed

Batch Result

  • status: done
  • batch queue_id: d63b4b31684a
  • session: be6951d6-6bf8-4098-b8ae-20e0a01542ea
  • batch duration: 527.5s