Skip to content

chore(build): cut debug info to fix link OOM#26

Open
LKSNDRTMLKV wants to merge 1 commit into
mainfrom
chore/link-oom-debug-info
Open

chore(build): cut debug info to fix link OOM#26
LKSNDRTMLKV wants to merge 1 commit into
mainfrom
chore/link-oom-debug-info

Conversation

@LKSNDRTMLKV

Copy link
Copy Markdown
Member

What

Sets [profile.dev] debug = "line-tables-only" for the workspace. One file, additive.

Why

dpp-node's integration test binaries link 250+ object files across wasmtime, aws-lc, sqlx and the AWS SDK. With full DWARF, linking several of them in parallel exhausts memory on a 16 GB machine and the build dies at the last CI stage (test-integration), in one of two ways:

  • LINK : fatal error LNK1102: out of memory
  • a bare STATUS_STACK_BUFFER_OVERRUN (0xc0000409) rustc abort with no diagnostic, when rustc runs out first

Both present as a code failure and are not one. This was mistaken for a broken feature branch more than once.

Evidence

Measured on feat/continuity-snapshots, same tree, same tests, only this setting changed:

Run Setting Result
Before full debug LNK1102: out of memory linking dpp-node
After line-tables-only 830 tests pass, 0 OOM, full parallelism retained

Notably this change alone is sufficient — CARGO_BUILD_JOBS=1 and CARGO_INCREMENTAL=0 were also tried and are not needed, so build parallelism and incremental compilation are preserved.

Trade-off

line-tables-only keeps function names, file names and line numbers in backtraces and panics — what this workspace actually debugs with. It drops variable-level DWARF, so step-debugger variable inspection is lost. That is not how this codebase is worked on, and the file carries a comment saying so.

Verification

just check green on this branch: fmt, clippy -D warnings, debug-check, subjects-check, mod-rs-check, 440 tests, cargo audit. Built against published dpp-core 0.9.0 (no local patch), matching CI.

Caveat, stated plainly: this branch cannot reproduce the bug it fixesjust check excludes test-integration, where the OOM occurs. The green gate here shows the change breaks nothing; the evidence that it works is the table above, measured on a branch that does reach that stage.

Sequencing

Deliberately based on main rather than any feature branch, because every branch needs it — including the open dependabot PRs, which are likely hitting this same wall. Landing it here first means feature branches can rebase onto a main that already builds.

Linking dpp-node integration binaries with full DWARF exhausts memory on a 16 GB machine, failing the last CI stage as LNK1102 or a bare rustc abort that both read as code failures.
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes.

@codacy-production codacy-production Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

The pull request successfully implements the intended change to reduce debug information, which should mitigate linker memory exhaustion in 16GB RAM environments. Codacy analysis indicates the project remains up to standards with no new quality issues. However, there is a significant architectural concern: while reducing debug info helps, the root cause of the memory pressure is the high number of parallel link jobs for individual integration tests. Furthermore, the fix cannot be fully verified using the current local 'just check' command, as it excludes the problematic tests.

About this PR

  • The current verification process is limited; the 'just check' command does not include the integration tests that trigger the OOM errors, meaning local success does not guarantee the CI build will pass.

Test suggestions

  • Verify that the development build completes successfully on a 16GB machine without linker memory exhaustion.
  • Verify that a panic in a development build still provides a backtrace with file names and line numbers.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify that the development build completes successfully on a 16GB machine without linker memory exhaustion.
2. Verify that a panic in a development build still provides a backtrace with file names and line numbers.

TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback

Comment thread Cargo.toml
# Raise this back to full debug only with a plan for the link memory.
# ---------------------------------------------------------------------------
[profile.dev]
debug = "line-tables-only"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Reducing debug info helps, but the root cause of linker memory pressure in large projects is often the high number of parallel link jobs. Since every .rs file in the tests/ directory is linked as a standalone binary, Cargo attempts many simultaneous links of the 250+ object files mentioned. Consolidating these into a single integration test suite (e.g., tests/it/main.rs) would reduce the link operations to a single event, drastically lowering memory requirements and potentially allowing you to retain full debug symbols. Try running the following prompt in your coding agent: > Consolidate all integration tests into a single test binary by creating a tests/it/main.rs that includes the existing test files as modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant