Skip to content

chore: result grid benchmarking harness (legacy vs ResultGrid)#569

Draft
emrberk wants to merge 16 commits into
mainfrom
bench/result-grid-harness
Draft

chore: result grid benchmarking harness (legacy vs ResultGrid)#569
emrberk wants to merge 16 commits into
mainfrom
bench/result-grid-harness

Conversation

@emrberk

@emrberk emrberk commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Dev-only A/B benchmarking harness to compare the legacy grid.js with the new React ResultGrid under identical synthetic data, plus the latest results. Stacked on top of #568 (base: refactor/result-grid) since it exercises the new grid.

Everything is gated behind the mock.pagination localStorage flag.

What's in here

File Purpose
src/scenes/Result/benchmarkMock.ts Synthesizes any rows × cols result from one constant canned page at a fixed 10 ms latency (zero network). Cells are self-describing (r{row}c{col}).
src/scenes/Result/index.tsx (+26) Two isMockPagination()-guarded hooks: a paginationFn short-circuit and a window.__benchSeed(rows, cols) seeder. Shared by both grids.
e2e/benchmark/gridBench.js Grid-agnostic in-page runner. Drives the right DOM/key transport and asserts the end state of every step.
BENCHMARK.md How to run it + removal recipe.
BENCHMARK_RESULTS.md Full results, per-case tables + caveats.

The mock is shared by both grids because both go through the same paginationFn and gridRef.setData — no per-grid benchmarking code.

How a step is measured (every step is asserted)

The metric is input → fully repainted and correct. A step's timer stops only on the frame where all hold (else it waits up to a timeout and the step is counted as a failure):

  1. every visible cell shows the value for its own (row, col) — self-describing values catch blank, stale, and column-misaligned cells in one check;
  2. the rendered cells cover the viewport's content area (no half-painted scroll);
  3. for keyboard moves, the focused cell is at the exact expected (row, col) with the value for that position.

Across both grids and all 7 cases (~5,600 asserted steps): 0 failures. Every scroll/keystroke landed correctly, so the numbers are time-to-verified-repaint, not just "a frame went by."

How to reproduce

  1. Start the app against a QuestDB instance (yarn start, dev server on :9999).
  2. Pick the grid via URL (the flag persists, the param is stripped): new ResultGrid?useNewGrid=1; legacy grid.js?useNewGrid=0.
  3. In the console: localStorage.setItem("mock.pagination", "true"), reload, then run any query once (e.g. select 1) to mount the result pane — window.__benchSeed appears.
  4. Paste the contents of e2e/benchmark/gridBench.js into the DevTools console to define window.__gridBench.
  5. Run cases and record each row (check failures is 0):
    for (const k of window.__gridBench.cases) console.log(await window.__gridBench.run(k))
  6. Repeat steps 2–5 for the other grid; compare.

Cases

key drives data
vscroll_1m 100 randomized vertical scrolls 1,000,000 × 20
hscroll_10k 100 randomized horizontal scrolls 2,000 × 10,000
homeend_cols 100 End→Home combinations (200 presses) 2,000 × 10,000
pagedn_10k PageDown ×100 then PageUp ×100 10,000 × 20
corners_1m_10k bottom-right → top-left corner jumps ×100 1,000,000 × 10,000
arrow_right_1k 999 ArrowRight presses 2,000 × 1,000
arrow_down_1k 999 ArrowDown presses 1,000 × 20

Results

1600 × 900 window, grid viewport ≈ 1510 × 340–420 px. Median ms, lower is better; 0 failures everywhere.

Case Legacy ResultGrid
Vertical scroll — 1M rows (×100) 99.7 107.9 ≈ even (debounce-bound)
Horizontal scroll — 10k cols (×100) 114.0 18.8 ResultGrid 6.1×
Home/End across 10k cols (100 combos) 150.6 30.8 ResultGrid 4.9×
PageDown ×100 / PageUp ×100 — 10k rows 8.3 8.8 ≈ even
Corner jumps ×100 — 1M × 10k 263.0 132.0 ResultGrid 2.0×
Right arrow ×999 cols 17.8 8.3 ResultGrid 2.1×
Down arrow ×999 rows 8.3 8.3 ≈ even

Takeaway: vertical paging and single-row stepping are tied. Everything that touches columns — horizontal scroll, Home/End, corner jumps, even column-by-column arrow stepping — is markedly faster on ResultGrid, which virtualizes columns; the legacy grid renders the whole visible column band.

Per-case detail

All times in ms; fail = steps whose end-state assertion never held within the timeout.

1. Randomized vertical scroll — 1,000,000 rows × 20 (100 scrolls)

Grid steps median p95 min max total fail
legacy grid.js 100 99.7 104.3 7.3 107.2 9917.0 0
ResultGrid 100 107.9 110.6 14.1 111.3 10676.5 0

2. Randomized horizontal scroll — 2,000 rows × 10,000 cols (100 scrolls)

Grid steps median p95 min max total fail
legacy grid.js 100 114.0 150.0 98.7 179.1 11914.3 0
ResultGrid 100 18.8 22.9 16.6 29.1 1929.9 0

3. Home / End across 10,000 columns — 100 End→Home combinations (200 presses)

Grid steps median p95 min max total fail
legacy grid.js 200 150.6 177.9 128.9 221.8 30291.9 0
ResultGrid 200 30.8 35.2 23.9 62.7 6205.7 0

4. PageDown ×100 then PageUp ×100 — 10,000 rows × 20 (200 presses)

Grid steps median p95 min max total fail
legacy grid.js 200 8.3 11.1 5.1 19.5 1685.2 0
ResultGrid 200 8.8 11.1 4.9 18.6 1833.9 0

5. Corner jumps — bottom-right → top-left ×100 — 1,000,000 rows × 10,000 cols (200 presses)

Grid steps median p95 min max total fail
legacy grid.js 200 263.0 314.6 233.8 399.0 53497.2 0
ResultGrid 200 132.0 140.5 123.2 159.8 26526.5 0

6. Right arrow through 1,000 columns — 2,000 rows × 1,000 cols (999 presses)

Grid steps median p95 min max total fail
legacy grid.js 999 17.8 21.1 14.9 33.8 18104.8 0
ResultGrid 999 8.3 9.7 5.9 14.9 8322.3 0

7. Down arrow through 1,000 rows — 1,000 rows × 20 (999 presses)

Grid steps median p95 min max total fail
legacy grid.js 999 8.3 9.4 6.3 21.7 8318.5 0
ResultGrid 999 8.3 9.4 4.8 165.5 8604.6 0

BENCHMARK_RESULTS.md carries the same tables plus extended caveats.

Caveats

  • The wide-result deltas scale with how many columns are on screen. The self-describing values are short, so columns are narrow and ~20 are visible — which is what makes the horizontal cases stress so hard. With wider columns the gap shrinks (an earlier mixed-width run showed horizontal scroll 1.8× instead of 6.1×); the direction is the same.
  • Vertical-scroll / corner medians are mostly the ~75 ms load debounce + 10 ms mock latency, not paint.
  • Small result pane (~10–14 visible rows); a taller pane would widen the deltas.
  • Both grids cap the scroll canvas at 10,000,000 px (~333k rows) and leap to the tail when the scrollbar bottoms out; the harness asserts the focused cell in each grid's own coordinate space (new grid focus rows are virtual, legacy are absolute).

🤖 Generated with Claude Code

emrberk and others added 5 commits June 12, 2026 12:57
A dev-only A/B harness to compare the legacy grid.js with the React
ResultGrid under identical synthetic data. Inert unless localStorage
"mock.pagination" is set.

- benchmarkMock.ts: synthesizes any rows x cols result from one constant
  canned page served at a fixed 10ms latency (zero network), shared by both
  grids via paginationFn and window.__benchSeed.
- index.tsx: two isMockPagination()-guarded hooks (paginationFn short-circuit
  and the __benchSeed seeder).
- e2e/benchmark/gridBench.js: grid-agnostic in-page runner measuring
  input -> fully-repainted across 7 scroll/keyboard cases.
- BENCHMARK.md: how to run it; BENCHMARK_RESULTS.md: latest legacy vs
  ResultGrid numbers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make repetition counts explicit and comparable across both grids:
- Home/End: 100 End→Home combinations (200 presses)
- PageDown/PageUp: 100 down then 100 up (200 presses, within range)
- Corner jumps: 100 bottom-right→top-left cycles (200 presses)

Re-ran all three on both grids; updated BENCHMARK_RESULTS.md. Step counts
now match between grids for every case, so totals are directly comparable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each step now stops its timer only once the grid has settled into a
verified-correct state, not merely "a frame painted":
- every visible cell shows the value for its own (row, col),
- the rendered cells cover the viewport,
- for keyboard moves, the focused cell is at the exact expected (row, col)
  with the value for that position.

To make value assertions exact and grid-agnostic, the mock now seeds
self-describing cells "r{row}c{col}" (catches blank, stale, and
column-misaligned cells in one check). Each run reports a `failures` count.

Re-ran both grids end to end (~5,600 asserted steps, 0 failures) and
refreshed BENCHMARK_RESULTS.md. The narrower self-describing columns put
more columns on screen, so column virtualization shows harder: horizontal
scroll 4.9x, Home/End 5.2x, corner jumps 1.9x, right-arrow stepping 1.8x;
vertical paging and down-arrow stepping remain tied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The legacy grid does NOT scale proportionally — its M = yMax/h ratio is
computed but never used; it advances the virtual position 1:1 and leaps to
the tail when the canvas (capped at 10,000,000 px, same as the new grid)
bottoms out. So both grids cap + tail-leap; the real difference is the focus
coordinate space (new grid: virtual rows; legacy: absolute rows), which the
harness already accounts for.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@emrberk emrberk marked this pull request as draft June 12, 2026 11:44
emrberk and others added 11 commits June 12, 2026 15:15
Re-ran all 7 cases on both grids (1600x900, mock.pagination) against the
merged base; 0 failures across ~5,600 asserted steps. Column-bound cases
(horizontal scroll, Home/End, corner jumps, right-arrow) remain markedly
faster on ResultGrid.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Base automatically changed from refactor/result-grid to main June 19, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant