feat: gen-schema entity port + resolver decoupling + delivery-edge unification#563
Open
sini wants to merge 70 commits into
Open
feat: gen-schema entity port + resolver decoupling + delivery-edge unification#563sini wants to merge 70 commits into
sini wants to merge 70 commits into
Conversation
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
6574fac to
a370d30
Compare
sini
force-pushed
the
feat/entity-gen-schema-port
branch
7 times, most recently
from
b2bcfd4 to
1b56211
Compare
sini
force-pushed
the
feat/entity-gen-schema-port
branch
5 times, most recently
from
e3584fe to
b1619a4
Compare
sini
changed the title
Replaces hand-rolled schemaEntryType with gen-schema mkSchemaOption. Sidecars: includes, excludes. Computed: isEntity (structural content only). Extracts resolvedCtxModule (id_hash, resolved, collisionPolicy) to _types.nix for entity type reuse. collisionPolicy flows through deferred module merge to entity instances (not a sidecar) preserving existing ctx.host.collisionPolicy resolution path.
den.hosts now accepts both forms:
- Legacy: den.hosts.x86_64-linux.igloo = { ... }
- Flat: den.hosts.igloo = { system = "x86_64-linux"; ... }
The outer option type uses a permissive submodule with deepMergeAttrs
freeformType (lib.recursiveUpdate-based merge that avoids the infinite
recursion lib.types.anything causes with cross-option references).
The apply function preprocesses flat entries into two-level form and
re-evaluates through the original attrsOf systemType, so all 6
consumers see the canonical { system.name = hostConfig } shape.
Same pattern as den.hosts: deepMergeAttrs + preprocessHosts + apply. Cross-entity host lookup and osConfig injection preserved.
Covers: id_hash, freeform, topology, meta introspection, isEntity computed, schema includes sidecar.
Update flake inputs and references to match the renamed repo at github:sini/gen-schema.
gen-schema flattened _meta into _-prefixed options and renamed sidecars → collections. nix-effects changed bindAttrs so true is a literal param, not an optionality marker — translate __args values to fx.bind.optionalArg before bind.fn.
One scope-walk.nix backs every subtree collection (default fold, route subtree collect, per-host re-walk, spawn extraction) and the edge-trace oracle, so production and oracle share ONE walk. `isolated` is a required arg with no default: the blind/aware split (census #6/denful#10) is deliberate, and the two blind callers (per-host sub-phase collect, spawn final extraction) are distinct call sites that must not collapse. dedupByKey folds three hand-rolled first-occurrence-wins go-loops (dedupProvides, extractSubtreeModules, wrapPerScope's per-class merge). dedupRoutes is left intact: its redundant-root suppression is interleaved with the key-dedup and does not separate cleanly.
Land the architectural core of the delivery-edge unification: the first mechanism (default class-fold) ported onto the edge algebra. - edges/edge.nix: the shared delivery-edge record (constructor, (T,P,S,M) sort key, id_hash scope naming, S/T constructors), EXTRACTED from edge-trace.nix so the read-only oracle and production constructors share ONE edge definition (spec §3a convergence). - edges/default.nix: the default-fold constructor — one merge edge per entity-root scope per class with content; isolation consumed at construction (subtree boundary), never as a mid-walk filter (corollary 2). - edges/materialize.nix: materialize (the ONLY mode switch, no mechanism names, no isolation reads) + assembleSubtree. merge mode = subtreeScopes + dedupByKey (the wrapPerScope/extractSubtreeModules semantics). nest arms throw explicit not-yet-ported markers (Task 8). Π(root) record per §A with per-field census provenance. - resolve.nix: mkInstantiateArgs' final per-host extraction (census variant B) routes through assembleSubtree with an EXPLICIT pi record; the old extractSubtreeModules is deleted (merge semantics moved to materialize). - edge-trace.nix: inline default-fold + mkEdge/sort/naming deleted in favor of the shared edge.nix + default.nix constructors. Pragmatic scope (§D): only mkInstantiateArgs' extraction is routed this task. Top-level (A) extraction is the wrapPerScope merge (no subtree extract to route); spawn-node (C) extraction is isolation-blind + dedup-free over a route-augmented scope set — left for Task 10 to avoid a trace move. B′ baseDrain ACCIDENT untouched (Task 11). delivery-edges 14/14 byte-stable; full CI 961/961; entity-isolation green.
….scopeContexts subtree-only
Port applySimpleRoute's four output arms (wrapRouteModules, ensureEntry, adapterWrapped, instantiateWrapped) onto the delivery-edge materializer per the §B matrix. The nest/nest-verbatim/merge mode mechanics (nestPlain, nestWithAdaptArgs, nestVerbatim, guardModule, the denful#572 single-eval combine, ensureTargetPath) move from route/wrap.nix into edges/route.nix as the materializer's route-edge mode switch (materializeRouteEdge in edges/materialize.nix). The route triple-pass dissolves into edge construction: dedupRoutes' two suppressions (adapterKey@scope identity dedup + redundant-root edge-set shadowing) and topoSortRoutes' producer→consumer ordering become edges/route.nix's suppressionVerdicts + a general index-based edge toposort with a loud cycle throw (§B Decision 5; no cycle reachable today, so byte- stable with the old noDeps-before-withDeps partition). The read-only oracle (edge-trace.nix) now renders simple+complex route edges through the SAME routeEdges constructor production materializes through, so the suppressed annotations are exact (constructor's own dedup rules), not the v0 path-dependent approximation. sourceVia for complex forwards stays unresolved (Task 9). Complex (__complexForward) routes stay inline in route/apply.nix (filterRootModules / getCollectedSource / resolveSourceFallback), unchanged — Task 9 ports them. wrap.nix reduces to collectClassMods (complex-forward only). Deleted as superseded: wrapRouteModules, nestPlain, nestWithAdaptArgs, nestVerbatim, nestModule, guardModule, adaptModule, the monolithic applySimpleRoute body, the old collectFromSubtree/dedupRoutes/topoSortRoutes/ findChildScopeKeys in route/apply.nix, and the dead dedupRoutes/ findChildScopeKeys re-exports. delivery-edges 14/14 byte-stable; route 7/7; full CI 961/961.
Delete the unreachable adapted==[] branch in materializeNest (sole caller is gated on non-empty modulesWithAdapter; ensure-empty arm fires first in materializeRouteEdge). Drop the redundant adapterPresent disjunct in the adapter arm (adapter kind only selected when modulesWithAdapter != []). Update materialize.nix header: Tasks 7-8 are live; the else-throw is permanent, not a stub. Add Task 9 deletion marker on applyComplexRoute. Fix phantom combineSingleEval in route.nix header (describe the inline denful#572 rule). Restore lost derivation comment above allMergeableAttrs.
Port the Task-9 survivors onto the edge materializer and delete them. Provides (§B Decision 1): new edges/provides.nix with applyProvidesEdges (phase-2 materialization, moved from resolve.nix:applyProvides — wrapClassModule /setDefaultModuleLocation/unsatisfied-drop preserved exactly), dedupProvides ((policyName/class/path) edge-identity key), and providesEdges (trace nest-edge constructor, mergeHalf=default-fold annotation). The nest∘merge decomposition is the materialization shape: nest into the source scope's bucket; the default-fold edge carries the merge half — no literal second edge. Complex forwards (§B Decision 2): synthesize machinery (applyComplexRouteEdge, filterRootModules, getCollectedSource, resolveSourceFallback, appendToClass, isDenDefaultModule, collectClassMods) absorbed into edges/route.nix; the collected-else-rewalk source rule and filterRootModules ownedClasses S-rule are intact. One synthesize edge per forward, identity triple (forwardId, fromClass, intoClass), content built at materialization. The whole applyRoutes fold (simple+complex dispatch) now lives in edges/route.nix; materializeRouteEdge moved there too (breaks a route<->materialize import cycle, keeps route mechanics together). resolve.nix: applyProvides/dedupProvides inline defs gone; phase-2 is applyProvidesEdges; the route wrapper delegates to routeEdges.applyRoutes. The route/ directory (apply.nix, wrap.nix, default.nix) is deleted. spawn-node keeps consuming applyProvides/applyRoutes as injected params (its phase block is Task 10) — fed applyProvidesEdges + the route wrapper, calls unchanged. sourceVia for complex forwards stays "unresolved" permanently: the trace renders construction-time identity, but the collected-else-rewalk branch is materialization-time path-dependent (spec §8: synthesize records identity, not content). Documented inline. Trace fixtures byte-stable (ZERO edits); full just ci 961/961 green.
…dges
Dissolve spawn-node's inline phase1(wrapPerScope) -> phase2(applyProvides
own-only) -> phase3(applyRoutes mergedSpawnRoutes) -> isolation-blind
dedup-free subtree concat into one assembleSpawnSubtree call (new entry in
edges/materialize.nix), routing the spawn final extraction through
assembleSubtree.
The Task-7 deferral's two blockers (different allScopeIds source; dedup-free
extraction) are resolved by extending the Pi record/materializer with two
dials, not by keeping the inline block:
- dedupMode ? "dedup": spawn passes "raw" for the dedup-free concat (phase1
wrapPerScope already key-deduped into the perScope buckets; the cross-scope
final concat must not re-dedup). collectMerge's raw arm iterates perScope
attrnames filtered by subtree membership -- the spawn's exact prior
iteration, preserving load-bearing module-list order.
- allScopeIds ? null: spawn passes mergedScopeParent + scopedRoutes keys
(a route-only scope can sit on the subtree parent-chain without a perScope
bucket -- wider than perScope alone).
Both dials default to canonical merge, so the per-host and entity-root
re-entries are untouched.
mergedSpawnRoutes (census #4 deliberate edge-identity dedup) stays as the
route-merge construction in spawn-node; census #3 (own provides only), #6
(blind extraction via isolationMode), and #2-C (host-bound pipe-key stripping)
preserved exactly. The _assertRoot self-parent guard keeps its augmented-forced
laziness. Injection seam kept (no cycle); drain-fold spawnNode call left
imperative.
delivery-edges 14/14 byte-stable (incl. host-aspects-spawn rewalk fixture),
full ci 961/961.
Replace findHostScopeId's name-infix heuristic with a spec->scope link recorded at scope creation. push-scope stamps the entity scope it creates into scopeByEntity, keyed by (parentScope, id_hash) — the same parent the instantiate spec is registered at, and the same entity record (hence id_hash) the spec carries. Both call sites (mkInstantiateArgs phase4 + hostConfigs B') resolve through entityScopeFor over that link. The (parent, id_hash) key handles multi-system same-name entities: id_hash is context-free (kind+name, not ancestry), so two same-name homes on different systems share an id_hash but have distinct system= parent scopes. The single-child fallback becomes the explicit T rule: a spec without a recorded entity scope (no id_hash / no link) targets its source scope's root (caller falls through to sourceScopeId). edge-trace resolvedRootVia annotation: "name-infix" -> "scope-link".
…ctor Extract the flake-output T-arm edge construction (spec descriptors + @System disambiguation) into edges/instantiate.nix, shared by production (resolve.nix applyInstantiates) and the read-only oracle (edge-trace.nix). The @System collision repair — qualify colliding output names with @System on multi-system, lib.warn same-entity dedup keeps last — is now the documented T-arm-local rule both consume, so the @System rule can never diverge between oracle and production (spec §3a convergence). applyInstantiates keeps ONLY the lazy thunk-tree build: descriptors and disambiguation touch path + system metadata exclusively (never spec.instantiate), so instantiate stays forced on output ACCESS — laziness preserved exactly. The oracle's parallel inline disambiguation re-derivation is deleted in favour of the shared constructor.
hostConfigs (re-entry B′) builds each peer host's full config for cross-host config-dependent pipe-thunk resolution. It built those over RAW scopeContexts (census #8) and RAW undrained imports (#2/#7) — so a peer whose config CONSUMES a pipe value (an aspect reading a quirk via context, e.g. `{ feat, ... }`) got its pipe value un-injected and threw `attribute 'feat' missing` instead of matching the peer's real instantiate output (variant B). Fix (§A option b): B′ now builds over augmentedScopeContextsNoCfg — a hostConfigs-NULL assemblePipes pass that is cycle-free (the cycle was assemblePipes-with-hostConfigs → hostConfigs) yet resolves every pipeline-parametric pipe value — plus the matching drainedForHostConfigs (mkDrained parameterized by its augmented-contexts source). Pipe-consuming and pipe-arg-deferred peers resolve correctly; a deferred include on a genuinely config-dependent pipe stays deferred under B′ (a real inter-config recursion no pass breaks — a documented limitation, not an opaque throw). Witness deadbugs/bprime-basedrain-crosshost (defect + control): the defect fails `feat missing` over raw contexts and passes under the fix; the control (non-pipe config field) is stable both ways.
Introduce the user-facing `deliver` delivery-edge constructor over the edge
layer (spec §4): `deliver { from; to; at ? []; mode ? "merge"; }`. `from` is a
class name (collect) or { module; } (inject); `mode` ∈ merge|nest|verbatim,
explicit (no reinstantiate flag, no isolation-derived verbatim magic).
route and provide become PERMANENT sugar shims over deliver, signatures
byte-stable: route maps fromClass/intoClass/path→from/to/at and
reinstantiate=true→mode=verbatim; provide maps class/module/path→module-source
deliver. Mechanism-only route fields (collectSubtree, appendToParent,
instantiate, adapterKey, ...) ride through an internal __extra escape hatch and
are NOT on the deliver surface. appendToParent is constructor-internal only,
reachable through the route shim for compat. Both shims carry a TODO comment;
no live deprecation warning.
deliver emits the same route/provide effect descriptors the edge constructors
already consume, so the materializer learns no new mechanism name (§2 invariant).
New deliver public-api suite (11 tests): each mode end-to-end, at-path nesting,
module source, mode validation, shim-equivalence (route/provide descriptor ==
deliver descriptor) + a live edge-trace oracle check. Docs: deliver reference
section; route/provide marked as shims.
…ocument binding rule Re-adds the den.lib.perHost/perUser/perHome API (removed in 70e6af2) as thin deprecated aliases — they shipped in main, so removal was a real breaking change. The restored shim drops the old hasExtras->{} self-suppression (that WAS the denful#609 bug); it now delivers the current binding rule (bind-at-scope / class-local fan-out / inert-if-misplaced), identical to a plain { host, ... }: function. Docs: parametric.mdx gains the formal binding rule + the silent-inert footgun (the only mitigation, since the lib.warn was deliberately rejected); debug.md and lib-deprecated.mdx corrected to match the restored, rule-correct shim.
…../.. The path:../.. den input only resolves inside the den tree; an external `nix flake init -t github:denful/den#flake-parts-modules` user got a broken input. Match the sibling starters (github:denful/den). CI masks this via --override-input den, so it was invisible in the matrix.
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
5868da9 to
7761d08
Compare
microvm + nvf-standalone den locks were pinned to a 2026-04-20 rev (oldest of the set); bumped to current denful/den main (dfc4617). noflake's npins den source pointed at vic/den; switched to denful/den (byte-identical mirror, same rev+hash — verified via nix store prefetch-file) for consistency with the flake templates. Other templates' pins fold into the post-merge bump.
…enful#613) policy.when guards consulted the fleet-wide flat in-flight pathSet, so a host's `host.hasAspect X` returned true whenever ANY host walked earlier had included X — an eval-order-dependent cross-host membership leak. Guards now read a scope-restricted union of pathSetByScope over currentScope + ancestors (the same scope+ancestor restriction collectScopeConstraints already applies to the constraint registry), so a host sees only its own + inherited membership, never a sibling's. pathSetByScope now mirrors the flat set's key space (both nodeKey and base key) so the guard's existing identity.key check and exclude logic are unchanged — only the data source is scoped. Regression: hasaspect-guard-cross-host (the issue's repro + order-reversal + own-include + ancestor-inheritance). fx-compile-conditional unit test seeds pathSetByScope (was injecting the flat pathSet).
… get-path-set dead
issue-609 is a fixed-bug regression, so it belongs under deadbugs/ (where issue-numbered regression tests live) rather than features/. Renamed with a descriptive suffix per the deadbugs convention; suite name unchanged. home-manager.mdx: replace the 'denful#609 fix' reference with 'earlier releases' — user-facing docs shouldn't carry tracker numbers. (The denful#222 in custom-classes is a feature-request attribution credit, kept; deadbugs tests keep their issue refs by convention — that's their traceability.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this delivers
This branch began as the gen-schema entity port and grew into a four-part stabilization of den's resolver. All four landed sequentially (each builds on the last) and are covered by CI. 975/975 tests (up from 825 at branch start).
1. gen-schema entity port (the original scope)
den.schemanow uses gen-schema'smkSchemaOption/mkSchemaEntryType(sidecar extraction, computed fields,__functorwrapping);resolvedCtxModuleextracted to shared_types.nix.id_hashvia gen-schemamkInstanceType(context-free kind+name identity).den.hosts.igloo = { … },den.homes."tux@igloo" = { … }), preprocessed to the canonical shape — all consumers see the unchanged{ system.name = entity }.den.reservedKeys(user-extensible structural keys);settingsreserved as structural.2. Isolation + delivered-child host (microvm guests)
isolatedmarker, recorded per-scope at scope creation.*.configre-runs eval with the base module list);reinstantiatekept as a core route flag.hasAspectre-keyed by entityid_hashrather than scope-string, fixing a false-negative (core.impermanence→ identity paths) for ancestor-nested hosts.guest-osclass — added and removed within this branch, never in a release — left no surface; isolated guests carry honestnixosidentity.)3. Resolver decoupling — binding half
Purifies who binds what, where emission lands, removing the host/user-specific machinery the resolver core carried (zero
host/user/homeManagerliterals remain in the fx resolver core).{ user, … }homeManager aspect no longer leaks to every user.den.lib.perHost/perUser/perHomeare kept as deprecated shims (they shipped in main) — now thin aliases over the new rule.4. Delivery-edge unification — delivery half
Collapses nine ad-hoc delivery mechanisms into one
(S, T, P, M)edge algebra: a single mechanism-free mode switch, one constructor per mechanism, a toposort for ordering, and an explicit root-context projection replacing three hand-threaded phase-fold re-entries.deliver { from; to; at ? []; mode ? "merge"; }primitive;routeandprovidesretained as permanent thin shims over it (signatures unchanged).findHostScopeId's name-infix heuristic dissolved into an exact scope-creation link.attribute missingthrow; witness test added).Breaking change
One breaking change versus the merge base — a semantic change, no API removals:
provides/policy only. This applies uniformly to plain parametric functions and to the deprecatedperHost/perUser/perHomeshims (whose old self-suppression was the Bug: homeManager content on host-included aspects is silently dropped unless the aspect is user-parametric #609 bug). Documented inexplanation/parametric.mdx(the rule + the silent-inert footgun) andreference/lib-deprecated.mdx.No public API is removed:
perHost/perUser/perHome,route,provides, the legacy two-level entity form — all retained (the first three as warning shims). Release notes will be drafted asheads-upGH discussions at release time permaintainers.mdx.Dependency note
den's root flake is a pure library (
outputs = _: import ./nix). gen-schema is resolved via the CI template lock, pinned togithub:sini/gen-schema— the same personal-repo + CI-lock-fallback pattern den already uses for nix-effects. Staying as-is for now.Verification
nix develop -c just ci).deliverAPI suite 11/11.Status
Draft. Deliverability audit complete (dependency resolution, breaking-change docs, leftover-scaffolding sweep, templates): gen-schema stays at
sini(matches nix-effects); flake-parts-modules template pin fixed; perHost shim restored; docs corrected. Specs, plans, and the per-task deviation ledger live in the den-architecture papers; a net-behavior-change summary is published as a gist.