Skip to content

Agent plan preapprove#133

Merged
mdeyell-valid-mind merged 54 commits into
mainfrom
mdeyell/gh-90/agent-plan-preapprove
Jul 23, 2026
Merged

Agent plan preapprove#133
mdeyell-valid-mind merged 54 commits into
mainfrom
mdeyell/gh-90/agent-plan-preapprove

Conversation

@mdeyell-valid-mind

@mdeyell-valid-mind mdeyell-valid-mind commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

We would like to give agents the option to submit plans for preapproval. Similar to normal invocations rules can be applied to plans.

Summary

Adds agent plan preapproval, allowing agents to submit plans for approval before executing their individual tool calls.

Plan approval uses the existing invocation rules. Each planned action is evaluated against relevant rules in order, while a shared LLM judge reviews the entire plan for charter
compliance at most once.

What changed

  • Added plan submission, approval, rejection, cancellation, expiration, revision, and completion workflows.
  • Added plan persistence, API endpoints, MCP tools, SSE events, and UI pages.
  • Added plan support to the Amp, Claude Code, Codex, Cursor, and Pi integrations.
  • Added plan notifications and links between plans and their invocations.

Plan approval behavior

For each action in a submitted plan, relevant invocation rules run from top to bottom:

  • auto_deny rejects the entire plan immediately.
  • auto_approve approves that action.
  • ai_evaluation requires whole-plan charter review.
  • Human-approval rules are treated as preapproved once the plan is approved.
  • No matching rule falls back to human plan approval.

A mandatory LLM judge reviews the entire plan for charter compliance:

  • It runs at most once per plan, even when multiple actions trigger AI-evaluation rules.
  • It may be skipped when an invocation rule deterministically denies the plan.
  • A noncompliant plan is rejected.
  • A missing charter is rejected.
  • Evaluator or configuration failures fall back to human approval.
  • An LLM revise result cannot override an auto_deny.
  • Empty local evaluator configuration now correctly uses the default enabled configuration.

Plan execution behavior

When an approved plan runs:

  • Invocations are matched to the newest applicable active plan.
  • Current auto_deny rules still block execution.
  • Rules already satisfied during plan approval are treated as preapproved.
  • An adherence judge verifies that the concrete invocation arguments follow an approved plan action and the charter.
  • Steps cannot execute backwards after a later step has run, though skipping forward is allowed.
  • Duplicate candidate actions bind to the earliest eligible step.
  • Status polling does not advance plan progress.
  • Completing the final successful action completes the plan.

The implementation supports both authenticated and no-auth deployments, including managed-agent identities and aliases.

UI

  • Added the Plans page and plan detail views.
  • Added human approval and rejection controls.
  • Added live SSE updates and approval notifications.
  • Added navigation between plans and related invocations.

Testing

Added coverage for:

  • Invocation-rule and judge decision combinations.
  • Multiple AI rules sharing exactly one judge call.
  • Auto-deny precedence over judge revisions.
  • Deterministic auto-deny skipping the judge.
  • Auto-approve plans without a charter being rejected.
  • Evaluator failure and default LLM configuration handling.
  • Authenticated and no-auth charter resolution.
  • Plan ordering, adherence, ownership, expiration, and completion.
  • Plan API, UI, SSE, and harness integrations.

#90
Screenshot 2026-07-13 at 11 47 50
Screenshot 2026-07-13 at 11 47 34

Screenshot 2026-07-13 at 11 48 59

Note that because claude managed agents doesn't have a harness it won't get the hook telling it about this feature.

@mdeyell-valid-mind
mdeyell-valid-mind marked this pull request as ready for review July 13, 2026 20:10
@mdeyell-valid-mind
mdeyell-valid-mind force-pushed the mdeyell/gh-90/agent-plan-preapprove branch from 8aa8af9 to 9a192b8 Compare July 13, 2026 22:00

@hunner hunner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid, carefully-engineered implementation — the fail-closed plumbing (parse failures → denied/human, LLM errors → human review or normal gating, missing charter → deny), the status-poll fast pass with its adversarial test matrix, and the overall test discipline are genuinely good. The concerns below are at the policy-design layer, not the implementation.

The through-line of the inline comments: the plan path is not judge-equivalent to unplanned execution. The plan judge necessarily sees agent-authored summaries rather than concrete arguments (inherent to preapproval), so the adherence judge is the mechanism that holds execution to the approved intent — but it currently only runs for AI-approved plans, its verdict set can't catch charter violations that follow the plan, and the judge/reviewer are deciding under a one-shot mental model of a pass that is actually unlimited-use until expiry.

Summary of the comments:

Fixes requested

  1. Run the adherence judge on human- and auto-approved plans too (approvedPlanPass)
  2. Give the adherence judge a charter-violation path (planAdherenceSystemPrompt)
  3. Default an omitted action.server to the submitting source instead of matching any upstream
  4. Persist TTLSeconds in PlansRepo.Update
  5. Test coverage for the Invoke plan-pass branch (and dedupe with Submit)

Discussion starters
6. Should approved-plan passes be consumable/counted rather than unlimited-use within TTL? (planJudgeSystemPrompt)
7. Should invocation-scoped auto_deny rules stay authoritative over plan passes?

🤖 Automated reply

Comment thread internal/invocation/plan_service.go Outdated
Comment thread internal/invocation/local_evaluator.go Outdated
Comment thread internal/invocation/plan_service.go Outdated
Comment thread internal/store/plans.go
Comment thread internal/invocation/service.go Outdated
Comment thread internal/invocation/local_evaluator.go
Comment thread internal/invocation/service.go Outdated
Comment thread examples/shared-agent-hook/atryum-hook.mjs Outdated
@mdeyell-valid-mind
mdeyell-valid-mind marked this pull request as draft July 15, 2026 17:27
@mdeyell-valid-mind

Copy link
Copy Markdown
Contributor Author

I talked to @hunner about this in a meeting and am putting it back in draft while i revise it

@mdeyell-valid-mind
mdeyell-valid-mind force-pushed the mdeyell/gh-90/agent-plan-preapprove branch from 9a192b8 to 0eebf64 Compare July 15, 2026 20:10
@mdeyell-valid-mind
mdeyell-valid-mind marked this pull request as ready for review July 17, 2026 00:14
@mdeyell-valid-mind
mdeyell-valid-mind force-pushed the mdeyell/gh-90/agent-plan-preapprove branch 2 times, most recently from dd15fb2 to 8449cdc Compare July 20, 2026 18:55

@even-steven even-steven left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know @nibalizer had brought this up with one of his demos that Invocations were not displaying in real-time, so I'm assuming the same is true for Plans

Invocations → SSE (server-push). useApprovalNotifications opens a streaming connection to /api/v1/admin/invocations/stream?status=pending_approval with Accept: text/event-stream, reads it via response.body.getReader(), and reacts to named invocations events as the server pushes them. Pending invocations therefore surface in near real-time.

Plans → polling (client-pull). There's no plan stream endpoint. Two independent 5-second setTimeout loops fetch /api/v1/admin/plans?status=pending_approval:

one in useApprovalNotifications (for the desktop notification), and
one in usePlans via react-query's refetchInterval (for the Plans page).
The hook author even flags it explicitly in usePlans.ts: // Plans have no SSE stream — poll every 5s so pending submissions show up.

@mdeyell-valid-mind

Copy link
Copy Markdown
Contributor Author

I know @nibalizer had brought this up with one of his demos that Invocations were not displaying in real-time, so I'm assuming the same is true for Plans

Invocations → SSE (server-push). useApprovalNotifications opens a streaming connection to /api/v1/admin/invocations/stream?status=pending_approval with Accept: text/event-stream, reads it via response.body.getReader(), and reacts to named invocations events as the server pushes them. Pending invocations therefore surface in near real-time.

Plans → polling (client-pull). There's no plan stream endpoint. Two independent 5-second setTimeout loops fetch /api/v1/admin/plans?status=pending_approval:

one in useApprovalNotifications (for the desktop notification), and one in usePlans via react-query's refetchInterval (for the Plans page). The hook author even flags it explicitly in usePlans.ts: // Plans have no SSE stream — poll every 5s so pending submissions show up.

Excellent feedback, thanks

@even-steven
even-steven self-requested a review July 20, 2026 23:09

@even-steven even-steven left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏽

@mdeyell-valid-mind
mdeyell-valid-mind force-pushed the mdeyell/gh-90/agent-plan-preapprove branch from 4d5ba0d to c4d0a1e Compare July 21, 2026 16:01
@mdeyell-valid-mind
mdeyell-valid-mind force-pushed the mdeyell/gh-90/agent-plan-preapprove branch from e8c48b8 to dcbd3ad Compare July 22, 2026 22:41

@hunner hunner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗓️

Nice work!

The only interesting thing I saw is that if a plan has actions that are 100% covered by auto_approve rules, plan_service still runs the evaluateMandatoryPlanCharter against the plan. This means an agent without a charter can run auto-approved commands, but would get a plan denied for the lack of a charter. Just a weird gap that doesn't really matter in real scenarios I don't think.

@mdeyell-valid-mind
mdeyell-valid-mind merged commit c0d9acb into main Jul 23, 2026
4 checks passed
@mdeyell-valid-mind
mdeyell-valid-mind deleted the mdeyell/gh-90/agent-plan-preapprove branch July 23, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants