diff --git a/BUGS.md b/BUGS.md index d1c33d31..4eb4737a 100644 --- a/BUGS.md +++ b/BUGS.md @@ -2,6 +2,122 @@ Open (not merged) bugs only. +## Bug 7: dipper pushes a SIGNED RCA proposal; indexer-agent expects an unsigned one and rejects every proposal (`non_empty_signature`) + +**Symptom.** DIPs happy path is fully blocked at D-2/D-3. Origination succeeds (dipper +returns a request id; IISA selects the indexer; indexer-service logs `RCA accepted` at the +gRPC layer and queues a `pending_rca_proposals` row), but ~4s later the agent flips the row +to `rejected` and no on-chain offer is ever posted. The dipper agreement stays `status=-1` +(CREATED); the indexing-payments subgraph never gets an `Offer`. Agent log: + +``` +level 50: Pending RCA proposal has non-empty signature (producer regression); rejecting signatureLength=65 +level 30: Rejected proposal : non_empty_signature +``` + +**Root cause.** Producer/consumer protocol skew. The indexer-agent's pending-RCA consumer +(`indexer-common/dist/indexing-fees/pending-rca-consumer.js`, `decodeRow`) requires the +pushed proposal's signature to be **empty** (`0x`/absent) — the design is that dipper pushes +an *unsigned* RCA and the agent handles on-chain acceptance; a non-empty signature is treated +as a "producer regression" and hard-rejected. But dipper `sha-cb89726` (main) signs the RCA +(65-byte sig) before pushing. The two pinned/running versions disagree on whether the pushed +proposal carries a signature, so every proposal dies at the agent boundary. + +Versions: indexer-agent = `local` build (pkg 0.25.10, has the guard; `.env.local` overrides +the pinned `sha-8ed8fb8` / PR #1238 main-dips); dipper = `DIPPER_VERSION=sha-cb89726` (main). + +**Actual root cause (resolved): local-network runs a STALE indexer-agent build that predates +the reconciliation PR.** The signature question was already settled across the two repos: + +- **dipper (main, incl. pinned `cb89726`):** deliberately *signs* the gRPC proposal so the + indexer can recover/authenticate the sender (dipper PR **#626**; + `indexer_rpc_client.rs::build_proposal_request` "Always sign the gRPC proposal…"). This is + intended and stays. +- **indexer (`main-dips`, current tip `aefddf61`, 2026-07-11):** *ignores* the signature. + PR **#1241** "fix: ignore a stored RCA signature instead of rejecting it" (merged + 2026-07-09) changed the consumer from reject→ignore: + `pending-rca-consumer.ts` → "An embedded signer signature is **ignored** — acceptance is + offer-based, so the agent has no use for it." (`const { rca } = signedRca` — signature not + read.) Current `origin/main-dips` contains this. + +So the contract is coherent now: **dipper signs, agent ignores the signature, acceptance is +offer-based.** The only reason local saw `non_empty_signature` rejection is that the running +agent image (`ghcr.io/graphprotocol/indexer-agent:local`, built 2026-06-12) predates #1241 +and still has the old reject guard. The pinned `sha-8ed8fb8` (#1240) may also predate #1241. + +**Repo / fix (local-network config).** Run/pin an indexer-agent that includes #1241 (current +`main-dips`, `aefddf61` or later). Either bump `INDEXER_AGENT_VERSION` to a published image +tag ≥ #1241, or run current `main-dips` from source via the `compose/dev/indexer-agent.yaml` +override (INDEXER_AGENT_SOURCE_ROOT). No upstream code fix needed — the upstream repos already +agree. + +**PR.** none needed upstream. local-network action only: advance the pinned/ran indexer-agent +past #1241 (the pinned `sha-8ed8fb8` and the `local` image are stale). Consider bumping the +`.env` pin once a suitable image tag is confirmed. + +**Verified fixed.** Ran current `main-dips` (`aefddf61`, includes #1241) from source via the +`compose/dev/indexer-agent.yaml` override. A fresh origination was accepted end-to-end: +proposal `completed` (signature ignored, no `non_empty_signature`), dipper agreement +`AcceptedOnChain` (6), new allocation opened via multicall (D-2/D-3.2 pass). + +## Bug 8: dev override `run-override.sh` is stale — missing all DIPs agent config → agent crashes on startup + +**Symptom.** Running indexer-agent from source via `compose/dev/indexer-agent.yaml` +(INDEXER_AGENT_SOURCE_ROOT), the agent crashes during network setup: +`TypeError: Cannot read properties of undefined (reading 'status')` at +`indexer-common/src/subgraph-client.ts:156` (from `network.ts:168`, creating the +IndexingPayments subgraph client). The management API (7600) never comes up. + +**Root cause.** `containers/indexer/indexer-agent/dev/run-override.sh` predates the DIPs work +in the normal `containers/indexer/indexer-agent/run.sh`. It omits +`INDEXER_AGENT_INDEXING_PAYMENTS_SUBGRAPH_ENDPOINT` (so the agent builds an IndexingPayments +subgraph client with neither endpoint nor deployment → `deploymentInfo!.status` on undefined), +plus `INDEXER_AGENT_TAP_SUBGRAPH_ENDPOINT`, `INDEXER_AGENT_TAP_ADDRESS_BOOK`, and the entire +DIPs enablement block (`ENABLE_DIPS`, `DIPS_*`, `DIPPER_ENDPOINT`, `OFFCHAIN_SUBGRAPHS` pin). +Without those, even if it started it would never run the DIPs accept path. + +**Repo / fix (local-network).** Ported the missing config from `run.sh` into +`run-override.sh` (indexing-payments + tap subgraph endpoints, tap address book, and the +`recurring_collector`-gated DIPs block). Agent then boots and enables DIPs. Longer term the +override should source/share the config with `run.sh` so they don't drift again. + +**PR.** committed to local-network (this repo) — the `run-override.sh` edit. + +## Bug 9: `start-indexing` image ships indexer-cli 0.23.8 — incompatible with the Horizon/main-dips agent close path → forced closes fail (IE070) + +**Symptom.** `graph-indexer indexer allocations close --force` (from the +`start-indexing` image) against the main-dips agent fails with +`[GraphQL] Failed to query latest valid epoch and block hash` (IE070). Agent logs show +`blockHashFromNumber(networkAlias="hardhat", blockNumber=null)` → +`Invalid value provided for argument "blockNumber": Null`, retried 5× then IE070. + +**Root cause.** Version mismatch. The image's indexer-cli is **0.23.8**, whose `close` +signature is `close ` — no block number. The main-dips **agent** (0.25.10) +close path (`monitor.ts::_resolvePublicPOI`) needs a Horizon `blockNumber` (+ `publicPOI`) to +resolve the POI block hash; with `--force` it short-circuits only when `publicPOI !== +undefined`, else it calls `blockHashFromNumber(alias, blockNumber)` with `blockNumber=null`. +The matching CLI (0.25.10, in the indexer source) has +`close --network ` — the Horizon-aware form. +This blocks any forced close via the shipped CLI: D-7.3 (force-close), D-8.3 (opt-out close), +and post-test allocation cleanup. + +**Repo / fix (local-network).** Bump the indexer-cli in `containers/indexer/start-indexing` +(and anywhere the CLI image is used) to a version matching the agent (≥0.25.x, Horizon +close args). Interim: run the CLI from the main-dips source (packages/indexer-cli) with the +` ` args, or drive the close via the management API with those fields. + +**PR.** none yet — local-network image/pin bump. + +**D-7 note.** D-7.1 (non-forced close rejected by the DIPs guard) and D-7.2 (allocation not +auto-closed) PASS. D-7.3 (force-close cancels agreement → state 2) is blocked by this CLI +mismatch, not by DIPs logic — the DIPs `assertSafeToCloseAllocation` force path works; the +failure is the generic Horizon POI-block resolution. + +**Secondary observation (not yet exercised).** The rejected proposal's `terms` carried +`conditions: 2`, but CLAUDE.md notes local-network should always use `conditions = 0` +(non-zero makes RecurringCollector staticcall the payer for an eligibility callback). This +never got exercised because the signature check rejects first; revisit once Bug 7 is resolved. + ## Bug 1: indexer-agent reconciliation races operator allocation changes, then blocklists the deployment **PR.** graphprotocol/indexer #1242 and #1243 (open, CI green), plus #1244 stacked on @@ -13,3 +129,88 @@ stick. **PR.** edgeandnode/dipper #664 (startup validation of the window and its sibling duration bounds) and #665 (decode the revert, fail the job, let expiry reassign), both open and CI green. + +## Bug 4: `just reset` silently leaves `config-local` (stale contract addresses) → Phase 3 aborts + +**Symptom.** After `just reset && just up`, `graph-contracts` exits 1 in Phase 3 with +`Sync failed for : no code on-chain` (e.g. RewardsManager_Implementation, or +IssuanceAllocator "stale (no code)"). Phase 1/2 succeed; the whole DIPs/issuance deploy +aborts and every downstream service stays `Created`. + +**Root cause.** `just reset` (`docker compose down -v`) only tears down services in the +*active* `COMPOSE_PROFILES`. A container from a **disabled** profile that is still running +(here `eligibility-oracle-node`, `rewards-eligibility` profile) keeps `config-local` +mounted, so `down -v` cannot remove that volume. Chain state (`chain-data`, used only by +the active `chain` service) *is* wiped, so the next `up` deploys a fresh chain — but the +persisted `config-local` still holds the previous deploy's address book. Phase 3's +`syncComponentsFromRegistry` reads those stale addresses, finds no code at them on the +fresh chain, and hard-fails (unlike Phase 1, which self-heals via a direct `cast code` +check). Confirmed: `config-local` `CreatedAt` stayed weeks old while `chain-data` was +recreated fresh. + +**Repo / fix (local-network).** Make `reset`/`down` tear down *all* profiles so stray +containers can't pin volumes — e.g. run the recipe with `COMPOSE_PROFILES` set to every +profile (or `docker compose --profile "*" down -v`). Optionally have Phase 3's sync +fall back to redeploy when the recorded address has no code, matching Phase 1. Immediate +unblock: `docker rm -f` the stray container, then `docker compose down -v` (it can now +remove `config-local`), then `just up`. + +**PR.** none yet. + +## Bug 5: escrow-manager's one-shot gateway-signer authorization is lost on a transient tx timeout → all paid queries 402, dipper never bootstraps + +**Symptom.** Fresh deploy comes up but `dipper` stays `unhealthy`, looping +`initial topology fetch failed ... failed to fetch subgraphs info`. The gateway returns +`bad indexers: {0xf4ef…: BadResponse(402)}`; the indexer's 402 body is +`No sender found for signer 0x70997970C51812dc3A010C7d01b50e0d17dc79C8` (the gateway's TAP +receipt signer, ACCOUNT1). Network is fully synced and allocated, so it is purely the TAP +query-payment authorization. + +**Root cause.** `graph-tally-escrow-manager` authorizes its signers once at startup, +signing from `ACCOUNT0_SECRET` — the same deployer account under heavy load during the +startup window. The `authorizeSigner` tx for the gateway signer (ACCOUNT1) timed out +(`transaction was not confirmed within the timeout`) and was **never retried**; the +process fell through to its steady-state receipt/RAV loop with the gateway signer still +unauthorized. Indexer-service therefore can't map the signer to any sender and rejects +every gateway query with 402, so dipper can never fetch topology. Restarting +`graph-tally-escrow-manager` re-runs the authorization (now uncontended) and both signers +go `authorized=true`; the 402 clears and dipper reaches `healthy` immediately. + +**Repo / fix.** Upstream `graph_tally_escrow_manager`: retry failed signer authorizations +in the update loop instead of one-shot at startup. local-network: order +`graph-tally-escrow-manager` to start after the deployer is quiescent (it currently +`depends_on start-indexing:service_completed_successfully`, which is not enough), and/or +give it a signer secret distinct from the deployer to remove the nonce race. Immediate +unblock: `docker compose restart graph-tally-escrow-manager`. + +**PR.** none yet. + +## Bug 6: Phase 3 deploy is not idempotent on a warm resume — RewardsManager left mid-upgrade breaks re-runs + +**Symptom.** A *fresh* `just reset && just up` deploys cleanly (Phase 3 completes, stack +healthy). But `just up` (or `just up --build`) on **preserved volumes** — chain state and +`config-local` fully consistent, Phase 1/2 idempotently SKIP, first Phase 3 stage re-syncs +"44 contracts synced" — then fails in `GIP-0088:upgrade,configure`: +`AbiFunctionNotFoundError: Function "GOVERNOR_ROLE" not found on ABI` at +`packages/deployment/deploy/rewards/reclaim/04_configure.ts:56` (ReclaimedRewards +configure). `graph-contracts` exits 1 and downstream services stay `Created`. + +**Root cause.** Phase 1/Phase 3 leave RewardsManager half-upgraded: the proxy points at +the live implementation (`0x9A9f…`) while the deploy records a **pending upgrade** to a +newer implementation (`0x5c74…`, which does define `GOVERNOR_ROLE`). On a single-pass +fresh deploy this never bites. On a resume, rocketh reloads the pending-upgrade record and +`04_configure`'s `extraDependencies` encodes `GOVERNOR_ROLE()` against the *old* impl's ABI +(no such function) and throws client-side before any chain call. Same dangling +RewardsManager pending implementation that also produced Bug 4's "no code on-chain" sync +failure when the chain was fresh — the common defect is that the deploy never applies (or +never clears) the pending RewardsManager upgrade, so Phase 3 is non-idempotent. + +**Repo / fix (contracts / graph-contracts run.sh).** Drive the RewardsManager upgrade to +completion (apply the pending impl and clear the pending record) so a re-run sees a settled +state, or make the reclaim/configure scripts tolerate a pending upgrade (resolve the ABI +from the pending impl, or skip when already configured) instead of throwing. Until then, +the resume path is unreliable — always redeploy clean. Immediate unblock: +`just reset && just up` (a clean single pass; `reset` now actually wipes `config-local` +since the volume-pinning stray containers from Bug 4 are gone). + +**PR.** none yet. diff --git a/containers/indexer/indexer-agent/dev/run-override.sh b/containers/indexer/indexer-agent/dev/run-override.sh index 9f143f3e..01aa5402 100755 --- a/containers/indexer/indexer-agent/dev/run-override.sh +++ b/containers/indexer/indexer-agent/dev/run-override.sh @@ -37,6 +37,12 @@ export INDEXER_AGENT_INDEX_NODE_IDS=default export INDEXER_AGENT_INDEXER_GEO_COORDINATES="1 1" export INDEXER_AGENT_VOUCHER_REDEMPTION_THRESHOLD=0.01 export INDEXER_AGENT_NETWORK_SUBGRAPH_ENDPOINT="http://graph-node:${GRAPH_NODE_GRAPHQL_PORT}/subgraphs/name/graph-network" +# indexing-payments subgraph is deployed by subgraph-deploy; the agent crashes on +# startup (undefined .status) if this is unset. TAP subgraph URL likewise must be +# defined or the agent crashes; point it at a stale 404 endpoint (DIPs doesn't use it). +export INDEXER_AGENT_INDEXING_PAYMENTS_SUBGRAPH_ENDPOINT="http://graph-node:${GRAPH_NODE_GRAPHQL_PORT}/subgraphs/name/indexing-payments" +export INDEXER_AGENT_TAP_SUBGRAPH_ENDPOINT="http://graph-node:${GRAPH_NODE_GRAPHQL_PORT}/subgraphs/name/semiotic/tap" +export INDEXER_AGENT_TAP_ADDRESS_BOOK=/opt/config/tap-contracts.json export INDEXER_AGENT_NETWORK_PROVIDER="http://chain:${CHAIN_RPC_PORT}" export INDEXER_AGENT_MNEMONIC="${INDEXER_MNEMONIC}" export INDEXER_AGENT_POSTGRES_DATABASE=indexer_components_1 @@ -49,6 +55,40 @@ export INDEXER_AGENT_MAX_PROVISION_INITIAL_SIZE=200000 export INDEXER_AGENT_CONFIRMATION_BLOCKS=1 export INDEXER_AGENT_LOG_LEVEL=trace +# DIPs: enable the agent's on-chain accept path when RecurringCollector is deployed +# (mirrors run.sh). Without it the agent never polls pending_rca_proposals, so every +# offer expires. This block is required for the DIPs dev workflow to work at all. +recurring_collector=$(contract_addr RecurringCollector.address horizon 2>/dev/null) || recurring_collector="" +if [ -n "$recurring_collector" ]; then + # Pin the indexing-payments subgraph offchain so reconcileDeployments doesn't + # pause it (the indexer has no allocation for it). Poll for it (up to 3m). + echo "Waiting for indexing-payments subgraph..." + INDEXING_PAYMENTS_DEPLOYMENT="" + for _ip_attempt in $(seq 1 36); do + INDEXING_PAYMENTS_DEPLOYMENT=$(curl -s "http://graph-node:${GRAPH_NODE_GRAPHQL_PORT}/subgraphs/name/indexing-payments" \ + -H 'content-type: application/json' \ + -d '{"query":"{ _meta { deployment } }"}' 2>/dev/null \ + | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['_meta']['deployment'])" 2>/dev/null || true) + [ -n "${INDEXING_PAYMENTS_DEPLOYMENT}" ] && break + [ $((_ip_attempt % 6)) -eq 0 ] && echo " still waiting for indexing-payments subgraph (attempt ${_ip_attempt}/36)..." + sleep 5 + done + if [ -n "${INDEXING_PAYMENTS_DEPLOYMENT}" ]; then + echo "Adding indexing-payments (${INDEXING_PAYMENTS_DEPLOYMENT}) to offchain subgraphs" + export INDEXER_AGENT_OFFCHAIN_SUBGRAPHS="${INDEXING_PAYMENTS_DEPLOYMENT}" + else + echo "WARNING: indexing-payments subgraph not found after 3m — DIPs accept path will stall" + fi + + echo "Enabling DIPs (RecurringCollector=${recurring_collector})" + export INDEXER_AGENT_ENABLE_DIPS=true + export INDEXER_AGENT_DIPS_EPOCHS_MARGIN=1 + export INDEXER_AGENT_DIPPER_ENDPOINT="http://dipper:${DIPPER_INDEXER_RPC_PORT}" + export INDEXER_AGENT_DIPS_ALLOCATION_AMOUNT=1 + export INDEXER_AGENT_DIPS_COLLECTION_TARGET=1 + export INDEXER_AGENT_POLLING_INTERVAL=15000 +fi + cd /opt/indexer-agent-source-root nodemon --watch . \ diff --git a/docs/dips-local-test-run.md b/docs/dips-local-test-run.md new file mode 100644 index 00000000..c7a22b9b --- /dev/null +++ b/docs/dips-local-test-run.md @@ -0,0 +1,303 @@ +# DIPs Local-Network Test Run — Execution Runbook + +Adapts `contracts/packages/subgraph-service/docs/dips/testing/LocalNetworkTestPlan.md` +to **this local host** (no `lnet-test` VM; everything on `localhost`). + +Scope: **full** — D-1→D-8, E-1→E-4, N-1→N-5. Drive: **cycle-by-cycle checkpoints**. +Target: **dedicated test subgraph(s)**. + +## Environment (resolved this deploy — addresses are per-deploy, re-read after any redeploy) + +| Var | Value | +|---|---| +| RPC | http://localhost:8545 (chain id 1337) | +| AGENT_URL (mgmt API) | http://localhost:7600 | +| NETWORK_SUBGRAPH_URL | http://localhost:8000/subgraphs/name/graph-network | +| INDEXING_PAYMENTS_SUBGRAPH_URL | http://localhost:8000/subgraphs/name/indexing-payments | +| Graph-node status | http://localhost:8030/graphql | +| DIPPER_ADMIN_RPC | http://localhost:9000 | +| PG | localhost:5432, db `indexer_components_1`, user `postgres` | +| SubgraphService | 0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 | +| RecurringCollector | 0x4A679253410272dd5232B3Ff7cF5dbB88f295319 | +| RecurringAgreementManager (RAM = on-chain payer) | 0x3347b4d90ebe72befb30444c9966b2b990ae9fcb | +| PaymentsEscrow | 0x322813Fd9A801c5507c9de605d63CEA4f2CE6c44 | +| RewardsManager | 0xa82fF9aFd8f496c3d6ac40E2a0F282E47488CFc9 | +| EpochManager | 0x7a2088a1bFc9d81c55368AE168C2C02570cB814F | +| SAO (subgraphAvailabilityOracle) | 0x15d34AAf54267DB7D7c367839AAf71A00a2C6A65 (Hardhat acct #4) | +| INDEXER | 0xf4EF6650E48d099a4972ea5B414daB86e1998Bd3 | +| dipper origination signing key | RECEIVER 0x2ee789a68207020b45607f5adb71933de0946baebbaaab74af7cbd69c8a90573 (only allowlisted caller) | + +Origination uses the **dipper-cli `indexings set-target-candidates`** admin RPC (compose +`tools` profile image), NOT the doc's Redpanda `indexing-requirements` topic (doc drift). + +## Known env-specific wrinkles (resolve as we hit them) + +- **Payer is the RAM contract**, not an EOA. Affects D-8.1 (`cancelIndexingAgreementByPayer` + can't be signed by an EOA "payer") and the `getBalance(payer,…)` reads — use RAM as the + payer arg; for payer-cancel we need to route through RAM or use the SP-cancel paths. +- **IISA needs Redpanda query history** or it selects 0 candidates. Phase 0 seeds it. +- **Time-travel bounds:** advance with `anvil_mine `; keep epoch churn + under the ~9-epoch allocation-expiration limit. +- **~15-min cooldown** after closing an allocation before the same deployment re-allocates — + cancellation re-runs use a fresh deployment. +- **D-7.3 `--force` is destructive** (ends the agreement) — last in its arc. + +--- + +## Phase 0 — Test fixtures & tooling (pre-D-1) ⟵ first checkpoint + +0.1 Resolve + export all env vars above; verify each address has code. +0.2 Deploy **3 dedicated test subgraphs**, actually indexed on graph-node and allocated: + - `dips-target` — reward-earning primary target (D-1…D-8 main line). + - `dips-denied` — for D-4.2 rewards-denied sizing. + - `dips-extra` — for E-4 concurrent + D-8 cancellation re-runs (avoids cooldown). + Steps: build minimal subgraph → deploy to graph-node → publish/curate so it's in the + network subgraph → set `always` rule → confirm agent opens an allocation → confirm synced. +0.3 Seed IISA: send gateway queries against target deployments, run an IISA scoring pass, + confirm `indexers=N` (≥1) not degraded. +0.4 Build **proposal-injection tooling** for N-1…N-5: a helper to encode a `SignedRCA` + (empty sig) and insert/adjust `pending_rca_proposals` rows, and to post/skip the on-chain + offer. (Mirror the agent's `encode_rca` / `insert_proposal` / `post_rca_offer_on_chain`.) +0.5 Confirm SAO key (Hardhat #4) can send `setDenied` (needed for D-4.2). + +**Checkpoint:** fixtures indexed+allocated, IISA scoring healthy, injection tool smoke-tested. + +## Phase 1 — D-1 Environment readiness +Run the plan's Prerequisites checklist; capture baseline (pick `dips-target` as reward-earning, +`dips-denied` as denied). Confirm both subgraphs synced to head, provision active, agent +`--enable-dips true`. + +## Phase 2 — D-2 Proposal origination +Seed IISA (0.3) → `set-target-candidates` for `dips-target` → verify dipper agreement `CREATED`, +`pending_rca_proposals` row `pending`, `Offer` entity on indexing-payments subgraph. + +## Phase 3 — D-3 Acceptance +D-3.1 reuse existing allocation; D-3.2 force no-allocation → multicall(startService,accept). +Mine a block after accept so `waitForTransaction` resolves. Verify state `Accepted` (1), +`dips` rule, dipper `ACCEPTED_ON_CHAIN`. + +## Phase 4 — D-4 Sizing +D-4.1 reward-earning tokens == rule `allocationAmount`/default. D-4.2 deny via SAO key → +tokens == `--dips-allocation-amount` (0) → undeny. + +## Phase 5 — D-5 Indexing & reconciliation +Deployment healthy on graph-node; agreement `Accepted` on subgraph; `dips` rule persists ≥2 +reconcile cycles; allocation not auto-closed. + +## Phase 6 — D-6 Recurring collection +Drive 2–3 windows (anvil_mine). D-6.1 first-collection `maxInitialTokens` bonus; D-6.2 recurring; +D-6.3 escrow drains; D-6.4 window/target placement; D-6.5 payout scales with entities. + +## Phase 7 — D-7 Protection +D-7.1 non-forced close rejected; D-7.2 not auto-closed; D-7.3 `--force` → state +`CanceledByServiceProvider` (2). **Run last in arc.** + +## Phase 8 — D-8 Cancellation (fresh agreement on `dips-extra`) +D-8.1 payer cancel → state 3 + final collection (resolve RAM-payer wrinkle); D-8.2 protection +releases when non-collectable; D-8.3 `never` rule → SP cancel (2) + rule reaped + alloc closed. + +## Phase 9 — E-1…E-4 Edge cases +Restart durability; late collection cap; two proposals one tick; concurrent agreements. + +## Phase 10 — N-1…N-5 Negative checks (injection-driven) +Deadline-expired; offer-never-posted; excessive slippage; escrow underfunded; missing provision; +stale subgraph (pause indexing-payments >5min); offer-hash mismatch. + +## Phase 11 — Teardown +Restore `always` rules; close test allocations; delete injected proposal rows; undeny; note +cooldown. Record pass/fail per test. + +--- + +## Automated regression (fresh reset, 2026-07-14) + +Ran `tests/dips-e2e.sh` (Phase 0 + D-1..D-8) on a clean `just reset && just up`. +**Result: all DIPs behaviors PASS.** Raw score 20/23; the 3 "failures" were test-script +artifacts, not DIPs issues, both since fixed in the script: +- D-4.1/D-4.2 sizing: `tokens()` helper read `sed -n '3p'` but `cast` prints the allocation + tuple on one line; the value was actually `10000000000000000` (0.01 GRT) — sizing correct. +- D-6.2: collection-count threshold `>=3` too strict for a fresh deploy's short window (got + 2); D-6.5 (entity-scaling) passed on real collection data, so recurring collection works. + +Regression also confirmed the fixes hold on a clean reset: no Bug 5 (dipper healthy on its +own), no Bug 6 (graph-contracts exit 0), Bug 7+8 fixes intact (agent boots from main-dips +source with #1241, DIPs enabled), Bug 9 worked around (D-7.3 force-close via the 0.25.10 CLI). + +## Phase 0 run log & corrections + +**Fixture design (corrected).** DIPs sizing/allocation must be driven by the agreement, so: +- **T-primary** — fresh deployment, **no `always` rule, not pre-allocated**. The DIP acceptance + creates the allocation via `multicall` (D-3.2), and it carries D-4.1/D-5/D-6/D-7. +- **T-denied** — fresh, no rule, not pre-allocated. D-4.2 (deny → DIP accept → sized to the + DIPs amount, which is **1 wei** here, not 0 — see doc-feedback #6). +- **T-reuse** — one deployment WITH an `always` rule + pre-existing plain allocation, for the + D-3.1 reuse path only. +- Do NOT put an `always` rule on the DIP-driven targets — it races the D-3.2 multicall + (doc-feedback #7). + +**Corrections to the first Phase-0 attempt:** +- Dropped the arbitrary `20 GRT` allocationAmount I had set on dips-test-1 — it came from no + config. Reward-earning sizing (D-4.1) is verified against the actual rule/default + (`0.01 GRT` global default), not an invented number. +- The first attempt set `always` on all three test subgraphs (via the management API, which + made the agent auto-allocate all of them) — wrong: it only exercises D-3.1 and pre-empts + D-3.2. Superseded by the fixture design above. + +**Env facts captured:** default allocation amount `0.01 GRT` (global rule); DIPs allocation +amount `1` wei (`indexer-agent/run.sh:125`); management-API rules need +`protocolNetwork: eip155:1337`. + +## Results tracker +Filled in as we go (test id → pass/fail → evidence). + +### Edge cases (E) +- **E-2 (late collection / downtime beyond window max)** — PASS. Agreement `0x10c28ca1…` on + dips-1. Stopped the agent, advanced chain time +2000s, then: `getCollectionInfo` → + `collectionSeconds = 660` (capped at `maxSecondsPerCollection`, not ~2028); restarted agent → + "Successfully collected indexing fees", `lastCollectionAt` advanced; payout `3.358e16` / + 660 ≈ `5.09e13`/s (a normal rate — would be ~`1.0e17` if the full elapsed time counted, so the + excess was forfeited); agreement state stayed `1` (Accepted), no cancellation. +- **E-4 (multiple concurrent agreements across deployments)** — PASS. Originated on 3 fresh + deployments at once (dips-2/3/4) alongside the live dips-1 = 4 concurrent. All 4 reached + AcceptedOnChain with **distinct allocations** (`0xa7b78cb6`, `0xf78fc729`, `0x725cba6a`, + `0xb0c34f85`). Driving windows advanced **all 4/4** `lastCollectionAt` independently, each + staying `Accepted`. (Optional underfunded-isolation sub-variant skipped — shared + protocol-funded escrow.) +- **E-1 (agent restart durability)** — PASS. With dips-1..4 accepted+collecting, originated a + fresh proposal on dips-5, caught the `pending` row at t=1s, and restarted the agent while it + was pending. Post-restart: (a) dips-5's pending proposal survived and reached AcceptedOnChain; + (b) dips-1..4 all 4/4 resumed collecting (`lastCollectionAt` advanced); (c) all dips rules + persisted (dips-1..4) plus dips-5's freshly-accepted rule (5 total, via management API). +- **E-3 (two proposals, same deployment, one tick)** — dedup PASS; plan's final outcome is + wrong (see doc-feedback #13). Injected two proposals (nonces 890001/890002) for the same + unallocated deployment dips-7, each with a posted on-chain offer (Tier 2). First tick: A + accepted (created allocation `0x6baaca5a…`), **B deferred (stayed `pending`)** — the anti-race + dedup works. But "both accepted sharing one allocation" is **not achievable**: accepting B + against A's allocation reverts `AllocationAlreadyHasIndexingAgreement(0x6baaca5a…)` + (selector `0x333e316d`) — the contract enforces one agreement per allocation, and there is one + allocation per (indexer, deployment). Correct outcome = **one accepted, one rejected** (B + rejected terminal). +### Negative checks (N) +- **N-1 (deadline-expired proposal)** — PASS. Injected expired-deadline proposal (dips-9) → + agent rejected with `deadline_expired`; no `dips` rule left for the deployment. +- **N-2 (offer never posted)** — PASS. Injected valid future-deadline proposal (dips-8) with no + on-chain offer → stayed `pending` across ~70s; agent logged "Offer not yet on subgraph; leaving + proposal pending". +- **N-5 (offer hash mismatch)** — PASS. Posted an on-chain offer with terms X, then injected a + proposal with different terms Y but the same agreement id (`0x19273f06…`). Agent: "on-chain + offerHash does not match local RCA hash" → `offer_hash_mismatch`, terminal (not retried). +- **N-4 (stale indexing-payments subgraph)** — NOT REPRODUCIBLE via the lag path (doc-feedback + #14). The staleness guard uses `Date.now() - subgraphChainTimestamp`; the local chain + time-travels ~2880s AHEAD of real time, so `lagSeconds` is negative and the lag branch never + fires. Collection-continues was verified (on-chain reads, unaffected). The reaper-skip's + `null`/unreadable branch is the only locally-triggerable path (requires breaking the subgraph + query — not forced, to avoid destabilizing the stack). +- **N-3 (excessive slippage)** — PASS (core criteria). The agent caps its collect at the payer + cap, so a *static* over-cap ask alone doesn't revert; the slippage revert is gated by + `--dips-collection-slippage` (`maxSlippage = ask × slippagePct/100`, default 1%). Injected an + agreement with `maxOngoing ≈ tokensPerSecond` + huge `tokensPerEntityPerSecond` (accepts, but + ask exceeds cap). With slippage=0 the collect **fails and the agent throttles** ("Failed to + collect agreement, will retry after throttle"), agreement stays `Accepted` (no cancel) — while + normal-terms agreements keep collecting. Revert is `RecurringCollectorExcessiveSlippage` + (the only slippage-gated revert; confirmed causally — failed only at slippage=0). Resume: + mechanism is the same gate; the artificial agreement's overage grew past 1% so it didn't + resume at slippage=1 (test-terms artifact, not a defect). +- **Escrow underfunded (removed from suite, #15)** — The DIPs payer escrow is + protocol-funded (issuance tops up RAM's escrow before each collect), so it can't be brought + below the amount owed without disabling issuance — not achievable non-destructively here. + Mechanism is clear from source (`collect` reverts `PaymentsEscrowInsufficientBalance` when + escrow < owed; agent throttles without cancelling). Deleted from the plan. +- **Missing provision (removed from suite, #15)** — (decision: too + destructive). Triggering the revert requires zeroing `getProviderTokensAvailable(indexer, + SubgraphService)`, i.e. thawing the indexer's entire ~871k GRT provision (2h thaw period), + which risks the agent auto-closing every allocation and forces a stack reset. Mechanism + verified from source: `RecurringCollector` requires `getProviderTokensAvailable > 0` else + reverts `RecurringCollectorUnauthorizedDataService` (an anti-siphoning guard); the agent + throttles-and-retries without cancelling and resumes once re-provisioned. Deleted from the plan. + +### N cycle summary +Plan renumbered to N-1…N-5 after removing the two destructive/infeasible checks (escrow +underfunded, missing provision). PASS: N-1 (deadline), N-2 (offer-never-posted), N-3 (excessive +slippage), N-5 (offer-hash mismatch). NOT REPRODUCIBLE locally: N-4 (stale subgraph — wall-clock +vs time-traveling chain, #14). REMOVED from the suite (#15) — mechanisms verified from source: +escrow underfunded, missing provision. + +- **Tooling:** built the proposal-injection tool (`tests/inject-rca.cjs` encoder + + `tests/inject-proposal.sh` inserter, incl. Tier-2 `--post-offer` via `RAM.offerAgreement`). + Validated via N-1 (expired-deadline → rejected) and a single-proposal accept (offer + inject → + agent accepts). Note the metadata `version` field is the enum `IndexingAgreementVersion{V1}` + so **V1 == 0** (not 1) — a wrong value makes the on-chain `decodeRCAMetadata` revert. + +- **D-1.1** readiness — PASS (fresh redeploy healthy; graph-contracts exit 0; dipper healthy; provision 100k GRT; both subgraphs synced). +- **D-1.2** baseline — PASS (T-primary=QmNePZ…gmyA reward target; T-denied=QmTBep…5rV7; T-reuse=QmcbVh…3tMy pre-allocated). +- **D-2.1** origination — PASS (dipper request id `019f5c8e-c552-7cf2-a720-ac7a215780ce`; IISA selected the indexer; indexer-service `RCA accepted` at gRPC). +- **D-2.2 / D-2.3** — initially FAIL (Bug 7): stale June-12 agent rejected with + `non_empty_signature`. **RESOLVED** by running current `main-dips` (PR #1241 ignores the + signature) from source via the `compose/dev/indexer-agent.yaml` override. Fixing that also + surfaced Bug 8 (stale dev override missing DIPs config) — fixed in `run-override.sh`. +- **D-2 (re-run on T-extra-a `QmTo7B…BUKo`)** — PASS. Request + `019f5caf-8396-7462-a57f-b985596d6526`; proposal accepted (signature ignored, row + `completed`); dipper agreement `AcceptedOnChain` (6). +- **D-3.2 (new allocation via multicall)** — PASS. Previously-unallocated T-extra-a got a + fresh allocation `0xc9d35f6df4695331596fea979b652ff73957998c` opened by the DIP acceptance + (multicall startService+acceptIndexingAgreement). On-chain agreement state AcceptedOnChain. + (Note: original T-primary `QmNePZ…` agreement Expired (5) while blocked; main-line target + reassigned to T-extra-a.) + +**Environment change:** indexer-agent now runs from source (`main-dips` @ `aefddf61`) via the +dev override — `.env.local` sets `INDEXER_AGENT_SOURCE_ROOT=/home/maikol/dev/the-graph/dips-worktrees/indexer` +and `COMPOSE_FILE=docker-compose.yaml:compose/dev/indexer-agent.yaml`. + +- **D-3.1 (accept reusing existing allocation)** — PASS (core behavior) on T-reuse + (`QmcbVh…3tMy`). Request `019f5cb4-…`; dipper `AcceptedOnChain` (6); agreement + `0x3ce653d3…` state Accepted; allocation `0xa1315a77…` **reused** (still Active, + `createdAtBlockNumber` unchanged at 386 → not reopened; only one active allocation). + Deviation: no `dips` rule added — the pre-existing `always` rule persists (see + doc-feedback #10); agent still tracks it ("2 active on subgraph"). +- **D-4.1 (reward-earning sizing)** — PASS. T-extra-a allocation tokens = `1e16` (0.01 GRT) = + the global default `allocationAmount` (no deployment-specific amount). Source named. +- **D-4.2 (rewards-denied sizing)** — PASS. Denied T-denied (`QmTBep…`) via SAO key + (`0x15d34AAf…`, Hardhat #4); DIP accepted, allocation `0xd7e63c28…` opened with tokens = + `1e18` (1 GRT) = `INDEXER_AGENT_DIPS_ALLOCATION_AMOUNT=1` **parsed as GRT** (distinct from + the 0.01 GRT reward default). Undenied afterward (`isDenied=false`). Note (doc-feedback #6): + value is 1 GRT, not 0 / not 1 wei. +- **D-5.1 / D-5.2 / D-5.3 (indexing & reconciliation)** — PASS on T-extra-a. D-5.1: deployment + synced+healthy on graph-node (no fatalError, latest≈head). D-5.2: agreement `0x544201b4…` + state `Accepted` on the indexing-payments subgraph. D-5.3: across 3 reconcile cycles the + `dips` rule persisted and allocation `0xc9d35f6d…` stayed Active (same id, block 3109 — not + auto-closed). +- **D-6 (recurring collection)** on T-extra-a — 24 collections observed. + - **D-6.1** — N/A: agreements use `maxInitialTokens=0` (doc-feedback #12); first collection + succeeded, no bonus by design. + - **D-6.2** — PASS: recurring across windows; `lastCollectionAt` advances each window; agent + logs "Successfully collected indexing fees" per cycle. + - **D-6.3** — DEVIATION (doc-feedback #11): payer escrow `getBalance` stayed constant + (`20.37 GRT`) — protocol-funded RAM+issuance replenishes it. Payment is real + (`tokensCollected` recorded & growing), but escrow doesn't drain as the plan expects. + - **D-6.4** — PASS: collections land ~75s apart (min 60s + ~15s poll) = near the start of the + [60,660]s window, matching `DIPS_COLLECTION_TARGET=1` (collect near window open). + - **D-6.5** — PASS: `tokensCollected` scales monotonically with `entities` + (4303→4752 entities → 4.139e15→4.269e15 tokens across steady windows). +- **D-7 (long-lived allocation protection)** on T-extra-a (alloc `0xc9d35f6d…`): + - **D-7.1** — PASS: non-forced close **rejected** with the DIPs guard message + ("has a DIPS agreement that can still collect fees … Use force=true"); allocation stays Active. + - **D-7.2** — PASS: allocation not auto-closed (still Active, block 3109, across 24 collections). + - **D-7.3** — PASS (verified with the correct CLI). Built indexer-cli 0.25.10 from source + (`packages/indexer-cli`) and ran `allocations close + --network hardhat --force` from the agent container. Result: "✔ Allocation closed"; + on-chain agreement state → **2 (CanceledByServiceProvider)**; allocation → **Closed** + (block 6966); indexing-payments subgraph agreement → `CanceledByServiceProvider`. Confirms + force-close cancels the DIPs agreement on-chain in the same tx. (Bug 9 remains: the shipped + start-indexing CLI 0.23.8 can't do this; local-network should bump it to ≥0.25.x.) +- **D-8 (cancellation)** — all three paths PASS, each on a fresh agreement: + - **D-8.1 (payer cancel)** on T-extra-b (`QmeQtq…`, alloc `0xbc58ca4c…`): payer cancel via + dipper-cli `set-target-candidates --num-candidates 0` → agreement state → **3 + (CanceledByPayer)**; then a **final collection** (`lastCollectionAt` 1783970773→1783970960); + `getCollectionInfo` (true,32,0)→**(false,0,3)** (drained/non-collectable). + - **D-8.2 (protection releases)**: once non-collectable, the agent's *internal* close ran — + allocation Active→**Closed** ("Populating unallocate transaction"→"Successfully closed + allocation"). Confirms the agent close path works (Bug 9 is external-CLI-only). + - **D-8.3 (indexer opt-out `never`)** on T-spare (`QmPFU7…`, alloc `0x54656b84…`): set `never` + → agent SP-cancels on-chain → agreement state → **2 (CanceledByServiceProvider)**, best-effort + final collection attempted ("Preparing to present POI"), allocation **Closed**, and no `dips` + rule remains (replaced by the `never` opt-out rule). diff --git a/docs/dips-test-plan-doc-feedback.md b/docs/dips-test-plan-doc-feedback.md new file mode 100644 index 00000000..8a3b81f8 --- /dev/null +++ b/docs/dips-test-plan-doc-feedback.md @@ -0,0 +1,216 @@ +# DIPs Test-Plan Doc Feedback + +Issues found while executing the contracts-repo DIPs test plan against local-network. +Source docs live in `graphprotocol/contracts` → +`packages/subgraph-service/docs/dips/testing/` (branch `mde/dips-integration-testing-docs`). +Fix these upstream after the run. + +Status legend: **confirmed** (reproduced) · **to-verify** (suspected, not yet hit in the run). + +--- + +## 1. Origination mechanism is wrong for the current dipper — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` D-2.1 says produce a message to the Redpanda + `indexing-requirements` topic to trigger origination. +- **Reality (dipper `sha-cb89726`):** origination is driven by the dipper-cli admin RPC + `indexings set-target-candidates --num-candidates N` + (signing key = RECEIVER `0xf4EF6650…`, the only allowlisted caller). There is no + `indexing-requirements` consumer path in this build. +- **Fix:** replace the D-2.1 Redpanda snippet with the dipper-cli `set-target-candidates` + flow (and note the IISA query-history prerequisite, see #4). + +## 2. `just up indexing-payments` is not a valid command here — confirmed + +- **Doc:** `LocalNetworkDetails.md` line 7 — "Bring it up with the `indexing-payments` + recipe: `just up indexing-payments`". +- **Reality:** this repo's `up` recipe forwards args to `docker compose up -d `, so + `just up indexing-payments` resolves to a **service** named `indexing-payments`, which + doesn't exist. The profile is enabled via `COMPOSE_PROFILES=…,indexing-payments` in `.env` + (or `.env.local`), then a plain `just up`. +- **Fix:** correct the bring-up instruction to set the profile in `.env`/`.env.local`, not a + `just up ` arg. + +## 3. Payer-cancel (D-8.1) assumes an EOA payer, but the payer is the RAM contract — confirmed (mechanism), cancel path to-verify + +- **Doc:** `LocalNetworkTestPlan.md` D-8.1 — + `cast send --private-key "$PAYER_SECRET" "$SUBGRAPH_SERVICE" "cancelIndexingAgreementByPayer(bytes16)" …` +- **Reality:** the on-chain `payer` is the `RecurringAgreementManager` (RAM) contract + (`0x3347…`), not an EOA — dipper offers via RAM using `AGREEMENT_MANAGER_ROLE`. There is no + `$PAYER_SECRET` that can sign as the payer; a payer-side cancel must be routed through RAM. +- **Fix:** describe payer-cancel via RAM (or the appropriate role-holder call), and drop the + `$PAYER_SECRET` EOA assumption. Confirm the exact RAM method during D-8. + +## 4. IISA query-history prerequisite is undocumented — confirmed + +- **Doc:** D-2 lists dipper/IISA "running" as the only origination prerequisite. +- **Reality:** IISA only selects indexers that have Redpanda **query history**; on a fresh + deploy it scores in degraded mode / 0 candidates, so origination yields no proposal until + gateway queries are sent and an IISA scoring pass runs. +- **Fix:** add a prerequisite step: seed gateway query traffic + run an IISA scoring pass + before D-2. + +## 5. "escrow funding service (TBD)" is now concrete — to-verify + +- **Doc:** `LocalNetworkDetails.md` services table lists "escrow funding service (TBD)". +- **Reality:** query-payment escrow/signer auth is `graph-tally-escrow-manager`; DIPs payer + escrow is topped up via issuance/RAM. Name them once verified during collection (D-6). + +--- + +## 6. DIPs rewards-denied allocation amount: doc says 0, local config is 1 wei — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` D-4.2 — "`--dips-allocation-amount` (env + `INDEXER_AGENT_DIPS_ALLOCATION_AMOUNT`), default `0`. A zero-token allocation is valid." +- **Reality (local-network):** `containers/indexer/indexer-agent/run.sh:125` sets + `INDEXER_AGENT_DIPS_ALLOCATION_AMOUNT=1`. **Verified in D-4.2:** the denied deployment's + allocation came out to `1e18` wei = **1 GRT** — i.e. the value is parsed as a **GRT amount**, + not wei. So the denied-variant allocation sizes to 1 GRT here, not `0` and not `1 wei`. The + pass criterion should read "equals `--dips-allocation-amount` (here 1 GRT)", and note the + GRT (not wei) interpretation; and/or local-network should set it to `0` to match the doc's + documented default and the "zero-token allocation is valid" claim. + +## 7. D-1.2 (target has `always` rule) races D-3.2 (new allocation via multicall) — confirmed by design + +- **Doc:** D-1.2 says pick the reward-earning target as a deployment with an `always` rule; + D-3.2 says that same target must have **no active allocation** so the DIP acceptance opens + one via `multicall(startService, acceptIndexingAgreement)`. +- **Conflict:** an `always` rule makes the agent's normal reconcile continuously (re)open a + plain allocation, which races/pre-empts the DIP multicall — you can't stably hold + "`always` rule + no allocation". A clean DIP-driven target should have **no `always` rule**; + the accepted agreement creates a `dips` rule that drives indexing+allocation. +- **Fix:** split the fixtures in the plan — a **reuse target** (pre-allocated, for D-3.1) and a + separate **DIP-driven target** (no rule, no allocation, for D-3.2 and the D-4→D-7 lifecycle). + "Reward-earning" only means *not denied*; it does not require an `always` rule. + +## 8. Setting an indexing rule via the management API requires `protocolNetwork` — confirmed + +- **Doc/toolbox:** indexing-rule examples don't mention `protocolNetwork`. +- **Reality:** `setIndexingRule` mutation requires `protocolNetwork: "eip155:1337"` (CAIP-2); + omitting it errors `Field "protocolNetwork" of required type "String!" was not provided`. + (The `graph indexer rules` CLI supplies it via `--network`; direct API callers must include + it.) + +## 9. IISA selection mechanics are undocumented (query-history requirement + unsynced pool) — confirmed from source + +- **Doc:** D-2 treats "dipper and IISA running; IISA can select the target indexer" as a given, + with no explanation of what makes an indexer selectable. +- **Reality (verified in `iisa` src, `iisa_http_endpoints.py` + `indexer_selection.py`):** + IISA selection uses two independent inputs: + 1. **Per-indexer quality history** (`_state.history`): stake-to-fees, latency, uptime, + success-rate, price — keyed by **indexer address**, derived from gateway query traffic in + the `gateway_queries` Redpanda topic. An indexer with **no** query history is absent from + `history` and cannot be selected at all. The traffic can be for **any** subgraph — it is + not per-target-deployment. (Hence the seeding step: send gateway queries so the indexer is + scored.) + 2. **Per-deployment synced set** (`sync_status_loader`, deployment → synced indexers): used as + a **preference, not a gate**. Candidates split into a "synced" pool and an "unsynced" pool; + an unsynced indexer still "competes on merit in the unsynced pool" (`indexer_selection.py` + ~546–573). This is what lets DIPs onboard an indexer onto a deployment it does **not** yet + serve — critical, and non-obvious. + - There is also a **price pre-filter** (`_filter_by_price`): an indexer whose advertised DIPs + price exceeds the request's `max_grt_per_30_days` is dropped before scoring. +- **Fix:** add a D-2 prerequisite documenting (a) seed gateway query history so the indexer is + scored, (b) the indexer need not already index the target (unsynced pool), (c) the indexer's + advertised price must be under the request ceiling. + +## 10. D-3.1 "a `dips` rule exists" is environment-dependent — not true when the reused allocation came from an `always` rule — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` D-3.1 pass criteria include "A `dips` indexing rule exists + for the deployment — `decisionBasis == "dips"`". +- **Reality:** the natural way to pre-open an allocation in local-network (so there is one to + reuse) is to set an `always` rule. After the DIP is accepted against that allocation, the + agent **keeps the `always` rule and does NOT add a `dips` rule** — yet it still tracks the + agreement (agent logs "Ensuring DIPS indexing rules: … 2 active on subgraph, 2 unique + deployments"; on-chain agreement `Accepted`; allocation reused, `createdAtBlockNumber` + unchanged). The `dips` rule only appears when the deployment had **no** prior rule (D-3.2). +- **Fix.** Either (a) reword D-3.1 to require "the deployment is tracked as a DIPs agreement + (active on subgraph / on-chain `Accepted`)" rather than specifically a `dips` rule, noting + that a pre-existing `always` rule is left in place; or (b) have D-3.1 open the reusable + allocation by a means other than an `always` rule so a `dips` rule is genuinely added. Also + clarify that rule-based criteria in D-5.3 / D-8.3 apply to the D-3.2 (no-prior-rule) target, + not the `always`-seeded reuse target. + +## 11. D-6.3 "escrow drains cumulatively" doesn't hold in the protocol-funded flow — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` D-6.3 — "The payer escrow balance decreases with each + collection … re-run `getBalance` per cycle and compare." +- **Reality:** on local-network the payer is the `RecurringAgreementManager` and escrow is + **replenished by issuance** (graph-contracts routes 6 GRT/block to RAM, which tops up escrow + before collection). Observed: `getBalance(RAM, RC, indexer)` stayed **bit-identical** + (`20370370370370372000`) across 3 driven windows even though 6 collections landed in that + span (24 total). Payment is real — `indexingFeeCollections.tokensCollected` is recorded and + grows per collection — but the payer-side escrow does **not** monotonically drain. +- **Fix.** For the protocol-funded (RAM + issuance) configuration, D-6.3 should verify + cumulative **`tokensCollected`** increasing (or the indexer's received total), not payer + escrow decreasing. Note escrow drain only applies to a fixed-deposit payer (e.g. a + consumer/gateway that deposits once), which is not the local-network DIPs setup. + +## 12. D-6.1 first-collection `maxInitialTokens` bonus is unobservable — local agreements use `maxInitialTokens = 0` — confirmed + +- **Doc:** D-6.1 — "The first collection includes `maxInitialTokens`, a one-time amount added + only when `lastCollectionAt == 0` … confirm the first is larger by roughly `maxInitialTokens`." +- **Reality:** dipper's RCA terms on local-network carry `max_initial_tokens = 0x0`, so there + is no bonus to observe; the first collection is just `collectionSeconds × rate` like the + rest. D-6.1 cannot be positively demonstrated without configuring a non-zero + `maxInitialTokens` in dipper's offer. +- **Fix.** Either note D-6.1 is N/A when `maxInitialTokens = 0` (local default), or have dipper + offer a non-zero `maxInitialTokens` for this test. + +## 13. E-3 "both proposals accepted, sharing one allocation" is impossible — contract enforces one agreement per allocation — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` E-3 pass criteria — "On the next tick the deferred proposal + is accepted against the now-existing allocation (the D-3.1 reuse path) — both end `accepted`, + sharing one allocation." +- **Reality:** the allocation id is deterministic per (indexer, deployment), so two proposals + for the same deployment target the **same** allocation, and the on-chain `SubgraphService` + enforces **one indexing agreement per allocation**. After the first proposal is accepted + (creating the allocation + agreement), accepting the second reverts + `AllocationAlreadyHasIndexingAgreement(allocationId)` (selector `0x333e316d`). Verified by + injecting two same-deployment proposals: A accepted, B deferred one tick (dedup works), then B + **rejected** with that contract error. "Both accepted sharing one allocation" cannot happen. +- **Fix.** Reword E-3: the *dedup* behavior (accept one per deployment per tick to avoid racing + the deterministic allocation id) is the real thing under test and it works; the correct final + outcome is **one accepted, the other rejected** (`AllocationAlreadyHasIndexingAgreement`), not + both accepted. Optionally note the agent could reject the duplicate off-chain before the + guaranteed on-chain revert (minor efficiency; end state is already correct). + +## 14. N-6 stale-subgraph (lag) scenario is not reproducible on local-network — staleness is wall-clock based vs a time-traveling chain — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` N-6 — "Pause the indexing-payments subgraph … and let it + fall more than 5 minutes behind chain head" → agent skips the reaper. +- **Reality:** the agent's staleness guard (`dips.ts`) computes + `lagSeconds = Math.floor(Date.now()/1000) - subgraphHeadBlockTimestamp` and skips the reaper + when `lagSeconds > 300` **or** the read is `null`. On local-network the chain time-travels + forward via `anvil_mine`, so the chain's block timestamps run **ahead of real wall-clock** + (measured: chain ~2880s ahead). Thus `lagSeconds` is **negative** and the lag branch never + fires — pausing the subgraph + advancing chain time makes it *less* stale, not more. Collection + continues regardless (reads on-chain), which was verified. Only the `null` (unreadable) branch + is triggerable locally, and only by making the subgraph query fail (remove/break it — + disruptive). +- **Fix.** Either (a) note in N-6 that the lag scenario is not reproducible on a time-traveling + local chain and test the guard via the *unreadable* branch instead (make the indexing-payments + subgraph query fail); or (b) upstream, consider measuring lag against chain head timestamp + rather than `Date.now()` so the check behaves on non-real-time chains. On a live network the + two agree, so this is primarily a local-testing limitation. + +## 15. N-4 and N-5 aren't suitable for a repeatable local-network integration suite (destructive / infeasible) — confirmed + +- **Doc:** `LocalNetworkTestPlan.md` lists N-4 (escrow underfunded) and N-5 (missing provision) + as negative checks. +- **Reality:** + - **N-4** — the DIPs payer is the `RecurringAgreementManager` and its escrow is topped up by + issuance before every collect (protocol-funded). There is no non-destructive way to bring + escrow below the amount owed (would require disabling issuance), so + `PaymentsEscrowInsufficientBalance` can't be provoked here. + - **N-5** — the revert requires `getProviderTokensAvailable(indexer, SubgraphService) == 0`, + i.e. thawing the indexer's entire provision (large, 2h thaw period). That risks the agent + auto-closing every allocation and forces a full stack reset — it destroys all other state. +- **Fix.** Mark N-4/N-5 as **not part of the repeatable local-network suite** (their mechanisms + are verifiable from source or via contract unit tests). If they must be exercised, do them on + a throwaway single-agreement stack as the *last* action before a reset, or cover them with + Foundry unit tests against `RecurringCollector`/`PaymentsEscrow` rather than the full stack. + +--- + +_Add entries as we go. Each: doc file + location, what it says, what actually happens, the fix._ diff --git a/tests/dips-e2e.sh b/tests/dips-e2e.sh new file mode 100755 index 00000000..637b8b9f --- /dev/null +++ b/tests/dips-e2e.sh @@ -0,0 +1,144 @@ +#!/bin/bash +# DIPs end-to-end regression: Phase 0 fixtures + D-1..D-8. +# Assumes a fresh, healthy stack (agent from main-dips source via dev override). +set -u +cd "$(dirname "$0")/.." + +RPC=http://localhost:8545 +GQL=http://localhost:8000 +STATUS=http://localhost:8030/graphql +MGMT=http://localhost:7600 +DRPC=http://localhost:9000 +CLI_IMG=ghcr.io/edgeandnode/dipper-cli:sha-cb89726 +RECEIVER_KEY=0x2ee789a68207020b45607f5adb71933de0946baebbaaab74af7cbd69c8a90573 +SAO_KEY=0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a +BEARER=deadbeefdeadbeefdeadbeefdeadbeef +# deterministic hardhat addresses +SS=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 +RC=0x4A679253410272dd5232B3Ff7cF5dbB88f295319 +ESCROW=0x322813Fd9A801c5507c9de605d63CEA4f2CE6c44 +RM=0xa82fF9aFd8f496c3d6ac40E2a0F282E47488CFc9 +EM=0x7a2088a1bFc9d81c55368AE168C2C02570cB814F +RAM=0x3347b4d90ebe72befb30444c9966b2b990ae9fcb +INDEXER=0xf4EF6650E48d099a4972ea5B414daB86e1998Bd3 +ZP=0x0000000000000000000000000000000000000000000000000000000000000000 +# deterministic deployment ids (same subgraph content each deploy) +T_PRIMARY=QmNePZ64EJ8LZSBSmrw2xtPF8kdYVtfDu7849t8r5wgmyA +T_REUSE=QmcbVhNwLfuXnowqwRkJmLsEhWSoJKrGnzdQtxWbnF3tMy +T_DENIED=QmTBepkzkmSFuQmSEvRMTXW8QkDPLxhiTy8egfJgaC5rV7 +T_A=QmTo7BjJzXQMcTrKtWzFEkeo7ztuRKC1b23MLtqj1xBUKo +T_B=QmeQtqvVMcWeBj7Sy5nvfUUu9DkQms12WZKj7V3hRb3trj + +P=0; F=0 +pass(){ echo " PASS: $1"; P=$((P+1)); } +fail(){ echo " FAIL: $1"; F=$((F+1)); } +gql(){ curl -s -m8 "$1" -H 'content-type: application/json' -d "$2" 2>/dev/null; } +agstate(){ timeout 15 docker exec chain cast call --rpc-url $RPC "$RC" "getAgreement(bytes16)(address,uint64,uint32,address,uint64,uint32,address,uint64,uint32,uint256,uint256,bytes32,uint64,uint16,uint8)" "$1" 2>/dev/null | tail -1; } +colinfo(){ timeout 12 docker exec chain cast call --rpc-url $RPC "$RC" "getCollectionInfo(bytes16)(bool,uint256,uint8)" "$1" 2>/dev/null; } +allocof(){ gql "$GQL/subgraphs/name/graph-network" "{\"query\":\"{ allocations(where:{status:Active, subgraphDeployment_:{ipfsHash:\\\"$1\\\"}}){ id } }\"}" | jq -r '.data.allocations[0].id // "none"'; } +allocstatus(){ gql "$GQL/subgraphs/name/graph-network" "{\"query\":\"{ allocations(where:{id:\\\"$1\\\"}){ status } }\"}" | jq -r '.data.allocations[0].status // "none"'; } +dstat(){ docker exec postgres psql -U postgres -d dipper_1 -tAq -c "SELECT status FROM dipper_reg_indexing_agreements WHERE deployment_id='$1' ORDER BY updated_at DESC LIMIT 1;" 2>/dev/null; } +agid(){ gql "$GQL/subgraphs/name/indexing-payments" "{\"query\":\"{ indexingAgreements(where:{allocationId:\\\"$1\\\"}){ id } }\"}" | jq -r '.data.indexingAgreements[0].id // empty'; } +lca(){ gql "$GQL/subgraphs/name/indexing-payments" "{\"query\":\"{ indexingAgreements(where:{allocationId:\\\"$1\\\"}){ lastCollectionAt } }\"}" | jq -r '.data.indexingAgreements[0].lastCollectionAt // "0"'; } +# cast returns the tuple on ONE line: (addr, bytes32, TOKENS [..], ...) — take the 3rd comma field, first token +tokens(){ timeout 12 docker exec chain cast call --rpc-url $RPC "$SS" "getAllocation(address)((address,bytes32,uint256,uint256,uint256,uint256,uint256,uint256))" "$1" 2>/dev/null | sed 's/[()]//g' | awk -F', ' '{print $3}' | awk '{print $1}'; } +originate(){ docker run --rm --network host $CLI_IMG indexings set-target-candidates --server-url $DRPC --signing-key $RECEIVER_KEY "$1" 1337 --num-candidates "${2:-1}" 2>&1 | tail -1; } +setrule(){ python3 - "$1" "$2" <<'PY' | curl -s -m8 "http://localhost:7600" -H 'content-type: application/json' --data @- >/dev/null +import json,sys +print(json.dumps({"query":"mutation S($r:IndexingRuleInput!){setIndexingRule(rule:$r){identifier}}","variables":{"r":{"identifier":sys.argv[1],"identifierType":"deployment","protocolNetwork":"eip155:1337","decisionBasis":sys.argv[2]}}})) +PY +} +mine(){ timeout 10 docker exec chain cast rpc --rpc-url $RPC anvil_mine 0x1 "${1:-0x78}" >/dev/null 2>&1; } +wait_accept(){ for i in $(seq 1 25); do [ "$(dstat "$1")" = "6" ] && [ "$(allocof "$1")" != "none" ] && { allocof "$1"; return; }; sleep 6; done; echo none; } +bytes32of(){ gql "$GQL/subgraphs/name/graph-network" "{\"query\":\"{ subgraphDeployments(where:{ipfsHash:\\\"$1\\\"}){ id } }\"}" | jq -r '.data.subgraphDeployments[0].id'; } + +echo "############ Phase 0: fixtures ############" +python3 scripts/deploy-test-subgraph.py 6 dips >/dev/null 2>&1 && echo " published 6 subgraphs" || echo " publish failed" +setrule "$T_REUSE" always +echo " set always on T-reuse; waiting for its allocation..." +RA=none; for i in $(seq 1 24); do RA=$(allocof "$T_REUSE"); [ "$RA" != "none" ] && break; sleep 10; done +[ "$RA" != "none" ] && pass "Phase0: T-reuse pre-allocated ($RA)" || fail "Phase0: T-reuse not allocated" +# seed IISA +for i in $(seq 1 40); do curl -s -m6 "http://localhost:7700/api/deployments/id/QmU9fn4xn2WZ2xTBwUswyqxQqt7DmbLTQhm5dQ5sMfXxur" -H 'content-type: application/json' -H "Authorization: Bearer $BEARER" -d '{"query":"{ _meta{block{number}} }"}' >/dev/null; done +echo " sent gateway queries; running IISA scoring..." +SC=$(docker compose --env-file .env --env-file .env.local run --rm iisa-cronjob 2>&1 | sed -E 's/\x1b\[[0-9;]*m//g' | grep -oE 'indexers=[0-9]+' | tail -1) +[ "$SC" = "indexers=1" ] && pass "Phase0: IISA scored ($SC)" || fail "Phase0: IISA scoring ($SC)" + +echo "############ D-2/D-3.2: originate + accept (new allocation) ############" +originate "$T_A" >/dev/null; AA=$(wait_accept "$T_A") +[ "$AA" != "none" ] && pass "D-2/D-3.2: T-A accepted, new alloc $AA" || fail "D-2/D-3.2: T-A not accepted" +[ "$(dstat "$T_A")" = "6" ] && pass "D-2.2: dipper AcceptedOnChain" || fail "D-2.2: dipper not 6" +AGA=$(agid "$AA"); [ -n "$AGA" ] && [ "$(gql "$GQL/subgraphs/name/indexing-payments" "{\"query\":\"{ offer(id:\\\"$AGA\\\"){ id } }\"}" | jq -r '.data.offer.id // empty')" = "$AGA" ] && pass "D-2.3: Offer indexed" || fail "D-2.3: Offer missing" +[ "$(agstate "$AGA")" = "1" ] && pass "D-3.2: on-chain agreement Accepted(1)" || fail "D-3.2: agreement not 1" +[ "$(gql "$MGMT" '{"query":"{ indexingRules(merged:false){ identifier decisionBasis } }"}' | jq -r ".data.indexingRules[]|select(.identifier==\"$T_A\").decisionBasis")" = "dips" ] && pass "D-3.2: dips rule created" || fail "D-3.2: no dips rule" + +echo "############ D-3.1: accept reusing existing allocation ############" +RUALLOC=$(allocof "$T_REUSE") +originate "$T_REUSE" >/dev/null; wait_accept "$T_REUSE" >/dev/null +[ "$(dstat "$T_REUSE")" = "6" ] && pass "D-3.1: T-reuse AcceptedOnChain" || fail "D-3.1: T-reuse not 6" +[ "$(allocof "$T_REUSE")" = "$RUALLOC" ] && pass "D-3.1: allocation reused ($RUALLOC)" || fail "D-3.1: allocation changed" + +echo "############ D-4: sizing ############" +[ "$(tokens "$AA")" = "10000000000000000" ] && pass "D-4.1: reward sizing 0.01 GRT" || fail "D-4.1: tokens=$(tokens "$AA")" +B32=$(bytes32of "$T_DENIED") +timeout 20 docker exec chain cast send --rpc-url $RPC --confirmations 0 --private-key $SAO_KEY "$RM" "setDenied(bytes32,bool)" "$B32" true >/dev/null 2>&1 +[ "$(timeout 10 docker exec chain cast call --rpc-url $RPC "$RM" "isDenied(bytes32)(bool)" "$B32" 2>/dev/null)" = "true" ] && echo " denied T-denied" || echo " deny failed" +originate "$T_DENIED" >/dev/null; DA=$(wait_accept "$T_DENIED") +[ "$DA" != "none" ] && [ "$(tokens "$DA")" = "1000000000000000000" ] && pass "D-4.2: denied sizing 1 GRT" || fail "D-4.2: denied tokens=$(tokens "$DA")" +timeout 20 docker exec chain cast send --rpc-url $RPC --confirmations 0 --private-key $SAO_KEY "$RM" "setDenied(bytes32,bool)" "$B32" false >/dev/null 2>&1 +echo " undenied T-denied" + +echo "############ D-5: indexing & reconciliation ############" +[ "$(gql "$STATUS" "{\"query\":\"{ indexingStatuses(subgraphs:[\\\"$T_A\\\"]){ synced health } }\"}" | jq -r '.data.indexingStatuses[0].health')" = "healthy" ] && pass "D-5.1: deployment healthy" || fail "D-5.1: not healthy" +[ "$(gql "$GQL/subgraphs/name/indexing-payments" "{\"query\":\"{ indexingAgreements(where:{allocationId:\\\"$AA\\\"}){ state } }\"}" | jq -r '.data.indexingAgreements[0].state')" = "Accepted" ] && pass "D-5.2: agreement Accepted on subgraph" || fail "D-5.2: not Accepted" +RULE1=$(gql "$MGMT" '{"query":"{ indexingRules(merged:false){ identifier decisionBasis } }"}' | jq -r ".data.indexingRules[]|select(.identifier==\"$T_A\").decisionBasis") +sleep 35 +RULE2=$(gql "$MGMT" '{"query":"{ indexingRules(merged:false){ identifier decisionBasis } }"}' | jq -r ".data.indexingRules[]|select(.identifier==\"$T_A\").decisionBasis") +[ "$RULE1" = "dips" ] && [ "$RULE2" = "dips" ] && [ "$(allocstatus "$AA")" = "Active" ] && pass "D-5.3: dips rule persists + alloc not auto-closed" || fail "D-5.3: rule/alloc changed ($RULE1->$RULE2, $(allocstatus "$AA"))" + +echo "############ D-6: recurring collection ############" +declare -a LCAS ENTS +for w in 1 2 3; do + pre=$(lca "$AA"); mine 0x78 + for _ in $(seq 1 8); do sleep 5; [ "$(lca "$AA")" != "$pre" ] && break; done +done +# read last 6 collections for entity scaling +COLS=$(gql "$GQL/subgraphs/name/indexing-payments" "{\"query\":\"{ indexingFeeCollections(where:{agreement:\\\"$AGA\\\"}, orderBy:blockTimestamp, orderDirection:asc, first:30){ tokensCollected entities } }\"}") +NCOL=$(echo "$COLS" | jq '.data.indexingFeeCollections | length') +[ "$NCOL" -ge 2 ] && pass "D-6.2: recurring collection ($NCOL collections across windows)" || fail "D-6.2: only $NCOL collections" +# entity scaling: entities monotonic up and tokens generally increase over the steady baseline +MONO=$(echo "$COLS" | jq '[.data.indexingFeeCollections[].entities|tonumber] | (. == (sort))') +[ "$MONO" = "true" ] && pass "D-6.5: entities monotonic (payout scales)" || fail "D-6.5: entities not monotonic" + +echo "############ D-7: protection ############" +NET=local-network_default; SI=local-network-start-indexing +GI023=/usr/local/lib/node_modules/@graphprotocol/indexer-cli/bin/graph-indexer +# D-7.1 non-forced close rejected +R71=$(docker run --rm --network "$NET" --entrypoint sh "$SI" -c "$GI023 indexer connect http://indexer-agent:7600 >/dev/null 2>&1; $GI023 indexer allocations close hardhat $AA $ZP --network hardhat 2>&1" 2>&1 | sed -E 's/\x1b\[[0-9;]*m//g') +echo "$R71" | grep -qi 'DIPS agreement that can still collect' && pass "D-7.1: non-forced close rejected" || fail "D-7.1: not rejected" +[ "$(allocstatus "$AA")" = "Active" ] && pass "D-7.2: allocation still Active (not auto-closed)" || fail "D-7.2: alloc not Active" +# D-7.3 force close via 0.25.10 CLI (agent container) +EB=$(timeout 10 docker exec chain cast call --rpc-url $RPC "$EM" 'currentEpochBlock()(uint256)' 2>/dev/null | awk '{print $1}') +docker exec indexer-agent sh -c "cd /opt/indexer-agent-source-root && node packages/indexer-cli/bin/graph-indexer indexer connect http://localhost:7600 >/dev/null 2>&1 && node packages/indexer-cli/bin/graph-indexer indexer allocations close $AA $ZP $EB $ZP --network hardhat --force >/dev/null 2>&1" +sleep 6 +[ "$(agstate "$AGA")" = "2" ] && [ "$(allocstatus "$AA")" = "Closed" ] && pass "D-7.3: force-close -> agreement Canceled(2) + alloc Closed" || fail "D-7.3: state=$(agstate "$AGA") alloc=$(allocstatus "$AA")" + +echo "############ D-8: cancellation ############" +# D-8.1 payer cancel on T-B +originate "$T_B" >/dev/null; BA=$(wait_accept "$T_B"); AGB=$(agid "$BA") +originate "$T_B" 0 >/dev/null +S3=no; for i in $(seq 1 12); do [ "$(agstate "$AGB")" = "3" ] && { S3=yes; break; }; sleep 6; done +[ "$S3" = "yes" ] && pass "D-8.1: payer cancel -> CanceledByPayer(3)" || fail "D-8.1: state=$(agstate "$AGB")" +preB=$(lca "$BA"); mine 0x78; for _ in $(seq 1 8); do sleep 5; [ "$(lca "$BA")" != "$preB" ] && break; done +[ "$(lca "$BA")" != "$preB" ] && pass "D-8.1: final collection after cancel" || fail "D-8.1: no final collection" +# D-8.2 protection releases -> allocation closes +CL=no; for i in $(seq 1 20); do [ "$(allocstatus "$BA")" = "Closed" ] && { CL=yes; break; }; mine 0x78; sleep 8; done +[ "$CL" = "yes" ] && pass "D-8.2: protection releases -> alloc Closed" || fail "D-8.2: alloc=$(allocstatus "$BA")" +# D-8.3 indexer opt-out (never) on a fresh agreement (reuse T-A deployment now that it's free? use T_PRIMARY) +originate "$T_PRIMARY" >/dev/null; PA=$(wait_accept "$T_PRIMARY"); AGP=$(agid "$PA") +setrule "$T_PRIMARY" never +S2=no; for i in $(seq 1 20); do [ "$(agstate "$AGP")" = "2" ] && [ "$(allocstatus "$PA")" = "Closed" ] && { S2=yes; break; }; sleep 8; done +[ "$S2" = "yes" ] && pass "D-8.3: opt-out -> SP cancel(2) + alloc Closed" || fail "D-8.3: state=$(agstate "$AGP") alloc=$(allocstatus "$PA")" + +echo "############ RESULT ############" +echo "PASS=$P FAIL=$F" diff --git a/tests/inject-proposal.sh b/tests/inject-proposal.sh new file mode 100755 index 00000000..69d5436e --- /dev/null +++ b/tests/inject-proposal.sh @@ -0,0 +1,62 @@ +#!/bin/bash +# Inject a pending_rca_proposals row into the indexer DB (bypasses dipper/indexer-service). +# Usage: inject-proposal.sh [--deadline ] [--nonce ] [--status pending] [--post-offer] [--no-row] +# --post-offer : also post the on-chain Offer via RAM.offerAgreement (DIPPER_SECRET, AGREEMENT_MANAGER_ROLE) +# --no-row : only post the offer, do not insert the DB row +# Encodes a SignedRCA (empty sig) matching what dipper produces. +set -u +cd "$(dirname "$0")/.." +# Indexer source (for ethers + toolshed). Reuse the dev-override var; read from env or .env.local. +INDEXER_SRC="${INDEXER_SRC:-${INDEXER_AGENT_SOURCE_ROOT:-$(grep -m1 '^INDEXER_AGENT_SOURCE_ROOT=' .env.local 2>/dev/null | cut -d= -f2-)}}" +[ -z "$INDEXER_SRC" ] && { echo "ERROR: set INDEXER_AGENT_SOURCE_ROOT (your indexer main-dips checkout) in .env.local, or export INDEXER_SRC"; exit 1; } +GQL=http://localhost:8000 +DEP_IPFS="$1"; shift +STATUS=pending +POST_OFFER=no +INSERT_ROW=yes +PASS_ARGS=() +while [ $# -gt 0 ]; do + case "$1" in + --status) STATUS="$2"; shift 2 ;; + --post-offer) POST_OFFER=yes; shift ;; + --no-row) INSERT_ROW=no; shift ;; + *) PASS_ARGS+=("$1" "$2"); shift 2 ;; + esac +done + +# Resolve the deployment bytes32 from the network subgraph. +B32=$(curl -s -m8 "$GQL/subgraphs/name/graph-network" -H 'content-type: application/json' \ + -d "{\"query\":\"{ subgraphDeployments(where:{ipfsHash:\\\"$DEP_IPFS\\\"}){ id } }\"}" | jq -r '.data.subgraphDeployments[0].id // empty') +[ -z "$B32" ] && { echo "ERROR: could not resolve bytes32 for $DEP_IPFS (published on GNS?)"; exit 1; } + +OUT=$(NODE_PATH="$INDEXER_SRC/node_modules" node tests/inject-rca.cjs --deployment-bytes32 "$B32" "${PASS_ARGS[@]}") +[ $? -ne 0 ] && { echo "ERROR: encode failed"; echo "$OUT"; exit 1; } +HEX=$(echo "$OUT" | jq -r '.payloadHex' | sed 's/^0x//') +OFFERDATA=$(echo "$OUT" | jq -r '.offerDataHex') +DEADLINE=$(echo "$OUT" | jq -r '.agreementIdInputs.deadline') +NONCE=$(echo "$OUT" | jq -r '.agreementIdInputs.nonce') + +RC=0x4A679253410272dd5232B3Ff7cF5dbB88f295319 +SS=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 +RAM=0x3347b4d90ebe72befb30444c9966b2b990ae9fcb +INDEXER=0xf4EF6650E48d099a4972ea5B414daB86e1998Bd3 +DIPPER_SECRET=0x254aaacc1a4091fdb844b297b2c00db641bd3a613914e9afaa0dbfce5c5cab5a # 0x85404..., AGREEMENT_MANAGER_ROLE + +# Derive the agreement id on-chain (the value the agent uses to match the offer). +AGID=$(timeout 12 docker exec chain cast call --rpc-url http://localhost:8545 "$RC" \ + "generateAgreementId(address,address,address,uint64,uint256)(bytes16)" "$RAM" "$SS" "$INDEXER" "$DEADLINE" "$NONCE" 2>/dev/null) + +# Optionally post the on-chain Offer via RAM.offerAgreement(collector, OFFER_TYPE_NEW=1, offerData). +if [ "$POST_OFFER" = "yes" ]; then + R=$(timeout 30 docker exec chain cast send --rpc-url http://localhost:8545 --confirmations 0 --private-key "$DIPPER_SECRET" \ + "$RAM" "offerAgreement(address,uint8,bytes)" "$RC" 1 "$OFFERDATA" 2>&1 | grep -iE '^status|error|revert' | head -1) + echo " post-offer($AGID): $R" +fi + +# Insert the row. +if [ "$INSERT_ROW" = "yes" ]; then + docker exec postgres psql -U postgres -d indexer_components_1 -tAq \ + -c "INSERT INTO pending_rca_proposals (id, signed_payload, version, status, created_at, updated_at) VALUES (gen_random_uuid(), decode('$HEX','hex'), 2, '$STATUS', now(), now());" >/dev/null \ + && echo "injected: deployment=$DEP_IPFS nonce=$NONCE deadline=$DEADLINE agreementId=$AGID status=$STATUS" \ + || echo "ERROR: insert failed" +fi diff --git a/tests/inject-rca.cjs b/tests/inject-rca.cjs new file mode 100644 index 00000000..fb5d4640 --- /dev/null +++ b/tests/inject-rca.cjs @@ -0,0 +1,78 @@ +// Encode a SignedRCA payload (ABI-encoded, empty signature) for injecting a +// pending_rca_proposals row into the indexer DB. Mirrors toolshed's decodeSignedRCA +// wire format. Run with NODE_PATH pointing at the indexer source's node_modules: +// NODE_PATH=/node_modules node tests/inject-rca.cjs --deployment-bytes32 0x.. [opts] +// Prints JSON: { payloadHex, agreementIdInputs, rca }. +const { ethers } = require('ethers') +const { decodeSignedRCA } = require('@graphprotocol/toolshed') + +const args = Object.fromEntries( + process.argv.slice(2).reduce((a, v, i, arr) => { + if (v.startsWith('--')) a.push([v.slice(2), arr[i + 1]]) + return a + }, []), +) + +// Deterministic hardhat addresses (per-deploy stable). +const RAM = '0x3347b4d90ebe72befb30444c9966b2b990ae9fcb' // payer (RecurringAgreementManager) +const SUBGRAPH_SERVICE = '0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9' +const INDEXER = '0xf4EF6650E48d099a4972ea5B414daB86e1998Bd3' +const RECURRING_COLLECTOR = '0x4A679253410272dd5232B3Ff7cF5dbB88f295319' + +const deploymentBytes32 = args['deployment-bytes32'] +if (!deploymentBytes32) throw new Error('--deployment-bytes32 required') +const nowSec = BigInt(args['now'] ?? Math.floor(Date.now() / 1000)) +const deadline = BigInt(args['deadline'] ?? nowSec + 600n) // default 10m future +const nonce = BigInt(args['nonce'] ?? Math.floor(Math.random() * 1e15)) +const conditions = BigInt(args['conditions'] ?? 2) // RC_CONDITION_AGREEMENT_OWNER (payer=RAM) +const endsAt = BigInt(args['ends-at'] ?? '18446744073709551615') // max uint64 = never +const tokensPerSecond = BigInt(args['tokens-per-second'] ?? '0x2316a9e9a22d') +const tokensPerEntityPerSecond = BigInt(args['tokens-per-entity-per-second'] ?? '0xe5f4c8f4') +const maxOngoing = BigInt(args['max-ongoing'] ?? '0x1b69b4be86b292') +const maxInitial = BigInt(args['max-initial'] ?? 0) +const minSecs = BigInt(args['min-seconds'] ?? 60) +const maxSecs = BigInt(args['max-seconds'] ?? 660) +const signature = args['signature'] ?? '0x' // agent ignores it (main-dips #1241) + +const coder = ethers.AbiCoder.defaultAbiCoder() +const TERMS_V1 = 'tuple(uint256 tokensPerSecond, uint256 tokensPerEntityPerSecond)' +const ACCEPT_META = 'tuple(bytes32 subgraphDeploymentId, uint8 version, bytes terms)' +const RCA = + 'tuple(uint64 deadline, uint64 endsAt, address payer, address dataService, address serviceProvider, uint256 maxInitialTokens, uint256 maxOngoingTokensPerSecond, uint32 minSecondsPerCollection, uint32 maxSecondsPerCollection, uint16 conditions, uint256 nonce, bytes metadata)' +const SIGNED_RCA = `tuple(${RCA} rca, bytes signature)` + +const terms = coder.encode([TERMS_V1], [{ tokensPerSecond, tokensPerEntityPerSecond }]) +// version is the on-chain enum IndexingAgreementVersion { V1 } — so V1 == 0 (NOT 1). +// A value of 1 is out-of-range and makes the contract's abi.decode into the enum revert. +const metadata = coder.encode([ACCEPT_META], [{ subgraphDeploymentId: deploymentBytes32, version: 0, terms }]) +const rca = { + deadline, endsAt, payer: RAM, dataService: SUBGRAPH_SERVICE, serviceProvider: INDEXER, + maxInitialTokens: maxInitial, maxOngoingTokensPerSecond: maxOngoing, + minSecondsPerCollection: minSecs, maxSecondsPerCollection: maxSecs, conditions, nonce, metadata, +} +const payloadHex = coder.encode([SIGNED_RCA], [{ rca, signature }]) +// offerData for RAM.offerAgreement(collector, OFFER_TYPE_NEW, offerData) = abi.encode(rca). +const offerDataHex = coder.encode([RCA], [rca]) + +// Self-validate: round-trip through toolshed's decoder (the exact fn the agent uses). +const decoded = decodeSignedRCA(payloadHex) +if (decoded.rca.deadline !== deadline || decoded.rca.nonce !== nonce || decoded.rca.payer.toLowerCase() !== RAM) { + throw new Error('round-trip validation FAILED: ' + JSON.stringify(decoded, (_, v) => (typeof v === 'bigint' ? v.toString() : v))) +} +const meta = require('@graphprotocol/toolshed').decodeAcceptIndexingAgreementMetadata(decoded.rca.metadata) +if (meta.subgraphDeploymentId.toLowerCase() !== deploymentBytes32.toLowerCase()) { + throw new Error('metadata deployment mismatch: ' + meta.subgraphDeploymentId) +} + +console.log( + JSON.stringify( + { + payloadHex, + offerDataHex, + agreementIdInputs: { payer: RAM, dataService: SUBGRAPH_SERVICE, serviceProvider: INDEXER, deadline: deadline.toString(), nonce: nonce.toString(), recurringCollector: RECURRING_COLLECTOR }, + rca: { deadline: deadline.toString(), nonce: nonce.toString(), conditions: conditions.toString(), deployment: deploymentBytes32 }, + }, + null, + 2, + ), +)