Add support for async workflow activities by seherv · Pull Request #1053 · dapr/python-sdk

seherv · 2026-05-25T22:13:08Z

Description

Workflow activities can now be async, and the runtime will automatically dispatch them to the event loop. Sync activities are still dispatched to the thread pool. The user-facing API remains exactly the same.

Also added a benchmark suite to verify performance locally.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #834 #897 #975

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Extended the documentation

seherv · 2026-05-25T22:15:29Z

-                        f"Activity '{req.name}#{req.taskId}' result is too large to deliver "
-                        f'(RESOURCE_EXHAUSTED). Failing the activity task: {rpc_error.details()}'
-                    )
-                    failure_res = pb.ActivityResponse(


This nesting got hard to follow, I needed to refactor this file to understand the logic better.

codecov · 2026-05-25T22:19:49Z

Codecov Report

❌ Patch coverage is 89.80392% with 78 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.18%. Comparing base (bffb749) to head (a2d4ad8).
⚠️ Report is 142 commits behind head on main.

Files with missing lines	Patch %	Lines
...-workflow/dapr/ext/workflow/_durabletask/worker.py	60.56%	28 Missing ⚠️
...r-ext-workflow/dapr/ext/workflow/_bench_harness.py	91.09%	22 Missing ⚠️
...workflow/tests/test_async_activity_registration.py	92.72%	12 Missing ⚠️
...kflow/dapr/ext/workflow/_durabletask/aio/client.py	50.00%	11 Missing ⚠️
...ext-workflow/dapr/ext/workflow/workflow_runtime.py	93.02%	3 Missing ⚠️
...rkflow/tests/durabletask/test_activity_executor.py	93.33%	1 Missing ⚠️
.../tests/durabletask/test_activity_executor_async.py	97.61%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1053      +/-   ##
==========================================
- Coverage   86.63%   83.18%   -3.45%     
==========================================
  Files          84      152      +68     
  Lines        4473    15536   +11063     
==========================================
+ Hits         3875    12924    +9049     
- Misses        598     2612    +2014

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

ext/dapr-ext-workflow/dapr/ext/workflow/_durabletask/worker.py:678

This adds an extra registry lookup (get_activity) and inspect.iscoroutinefunction check on the hot dispatch path, but _ActivityExecutor._resolve() will look up the activity again during execution. Consider plumbing activity_fn (and/or an is_async flag decided at registration time) through to the executor so each activity work item only does one lookup in total.

                                activity_handler,
                                work_item.activityRequest,
                                stub,
                                work_item.completionToken,
                            )

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 11 changed files in this pull request and generated 5 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 4 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 4 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 3 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 7 comments.

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv · 2026-06-02T17:27:53Z

Improving benchmarks and regression tests before submitting for review again

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

acroca · 2026-06-04T10:40:27Z

Thanks for working on this! I agree with the general direction, async activity support is something we've wanted for a while (#834, #897, #975), and keeping the user-facing API unchanged is the right approach.

I'd like to share my opinion on the benchmarking side, though. I think the sync-vs-async comparison is valuable in the context of this PR, to justify the why behind the change, but not as something we keep and maintain in the codebase. Concretely:

The benchmark suite adds ~1,100 lines (benchmarks/, _bench_harness.py), which is more code than the feature itself, and the comparison is essentially a one-off: once async dispatch is merged, it has served its purpose.
_bench_harness.py sits inside dapr/ext/workflow/, so it would ship to every user as part of the package.
The regression tests in test_async_dispatch_regression.py mostly assert sync-vs-async ratios, which again is the one-off comparison, and they depend on the harness.

My suggestion: run the benchmarks locally and paste the results + methodology into the PR description. Then the harness, the benchmarks/ directory, and the ratio-based tests can be dropped from the PR.

That said, I do think we should keep one simple throughput regression test so the async path can't silently regress, something like running 1000 activities that each take 1 second and asserting the batch completes in well under 10 seconds (ideally it's ~1s, so a 10x bound leaves plenty of headroom for CI noise). That single assertion catches the failure mode we actually care about, async activities accidentally getting serialized, without needing the full harness, and it can be written directly against the runtime in a few dozen lines.

Thoughts?

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

sicoyle

few comments so far. I'm almost half way through the files/changes :)

sicoyle · 2026-06-09T21:17:29Z

+    try:
+        return Path(path).read_text(encoding='utf-8', errors='ignore')
+    except OSError:
+        return ''


could you maybe add a warn log here

sicoyle · 2026-06-09T21:19:35Z

+"""Sync-vs-async activity benchmarks for ``dapr-ext-workflow``.
+
+Runs the same I/O-bound activity workload as ``def`` and ``async def`` through the
+production dispatch path against a mock sidecar stub. Scenarios: a fan-out burst, a
+fan-out shaped as many small workflows, and a sustained open-loop run.
+
+Run:
+
+    uv run python ext/dapr-ext-workflow/benchmarks/bench_async_activities.py
+
+``DAPR_BENCH_ACTIVITY_MS`` overrides the activity duration, ``DAPR_BENCH_SUSTAINED_SECONDS``
+the sustained run. Writes ``benchmarks/RESULTS.md`` and asserts pass-criteria budgets.
+"""


ideally this is not needed and is included in a readme pls

sicoyle · 2026-06-09T21:21:38Z

+    return rows
+
+
+async def run_sustained() -> tuple[SustainedMetrics, SustainedMetrics]:


what does this mean sustained? like constant throughput over the same duration?

sicoyle · 2026-06-09T21:22:29Z

+# Sync vs async activity benchmark
+
+Generated by `bench_async_activities.py`. Re-run with:
+
+```bash
+uv run python ext/dapr-ext-workflow/benchmarks/bench_async_activities.py


pls don't include how to run here bc this can easily become stale if we rename the file or move things.

sicoyle · 2026-06-09T21:25:09Z

+
+{_format_sustained_table(sustained_sync, sustained_async)}
+
+See `ext/dapr-ext-workflow/docs/concurrency.md` for sizing guidance.


is this a doc you plan to share? or can you rm this ref pls?

sicoyle · 2026-06-09T21:42:50Z

+    end_ts: dict[int, float],
+) -> tuple[Callable[..., object], Callable[..., object]]:
+    """Build the activity callable and pick the matching dispatch handler for ``kind``."""
+    if kind == 'async':


sicoyle · 2026-06-09T21:42:59Z

+            else _async_sleep_factory(latency_s, start_ts, end_ts)
+        )
+        return fn, worker._execute_activity_async
+    if kind == 'sync':


const and switch pls

sicoyle · 2026-06-09T21:43:21Z

+    )
+
+
+ActivityFactory = Callable[[dict[int, float], dict[int, float]], Callable[..., object]]


in general can you also split this file up with smaller appropriately named files pls?

sicoyle · 2026-06-09T21:43:57Z

+def _metrics(
+    *,
+    name: str,
+    n_items: int,
+    semaphore_cap: int,
+    thread_pool_workers: int,
+    server_latency_s: float,
+    wallclock_s: float,
+    e2e_samples: list[float],
+    sampler: _Sampler,
+    baseline_rss_kb: int,
+) -> ScenarioMetrics:
+    completed = len(e2e_samples) if e2e_samples else n_items
+    return ScenarioMetrics(
+        name=name,
+        n_items=n_items,
+        semaphore_cap=semaphore_cap,
+        thread_pool_workers=thread_pool_workers,
+        server_latency_s=server_latency_s,
+        wallclock_s=wallclock_s,
+        throughput_per_s=completed / wallclock_s if wallclock_s > 0 else 0.0,
+        latency=LatencyStats.from_samples(e2e_samples),
+        peak_tasks=sampler.peak_tasks,
+        peak_queue_depth=sampler.peak_queue_depth,
+        peak_rss_delta_mb=max(0.0, (sampler.peak_rss_kb - baseline_rss_kb) / 1024.0),
+    )
+
+
+def _make_activity_context(orchestration_id: str, task_id: int) -> task.ActivityContext:
+    return task.ActivityContext(orchestration_id, task_id, '', propagated_history=None)


is it really worth creating helpers that are one liners?

sicoyle · 2026-06-09T21:44:28Z

+    )
+
+
+async def _run_lite(


what is lite meaning here?

maybe docstrings would help on some of these funcs pls?

seherv commented May 25, 2026

View reviewed changes

Add support for async activities

872e75c

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv force-pushed the async-compat branch from f0870e0 to 872e75c Compare May 25, 2026 22:32

JoshVanL requested a review from Copilot May 26, 2026 10:45

Copilot AI reviewed May 26, 2026

Copilot started reviewing on behalf of JoshVanL May 26, 2026 12:21 View session

Merge branch 'main' into async-compat

413f16f

seherv requested a review from Copilot May 27, 2026 07:46

Copilot AI reviewed May 27, 2026

View reviewed changes

Address Copilot feedback (1)

b52139c

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv requested a review from Copilot June 1, 2026 09:12

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Address Copilot feedback (2)

0c64d1a

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv requested a review from Copilot June 1, 2026 11:43

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Address Copilot feedback (3)

69ea96e

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv requested a review from Copilot June 1, 2026 13:24

Copilot AI reviewed Jun 1, 2026

View reviewed changes

seherv added 3 commits June 1, 2026 16:11

Address Copilot feedback (4)

81fa323

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Fix linter

dadaa9a

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Merge branch 'main' into async-compat

8c4ce88

seherv requested a review from Copilot June 1, 2026 14:16

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Comment thread ext/dapr-ext-workflow/docs/concurrency.md

Comment thread ext/dapr-ext-workflow/AGENTS.md Outdated

Comment thread ext/dapr-ext-workflow/benchmarks/bench_async_activities.py Outdated

Comment thread ext/dapr-ext-workflow/dapr/ext/workflow/_durabletask/worker.py

Address Copilot feedback (5)

e8c4c05

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv requested a review from Copilot June 1, 2026 14:54

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Comment thread ext/dapr-ext-workflow/dapr/ext/workflow/_durabletask/internal/shared.py

Comment thread ext/dapr-ext-workflow/dapr/ext/workflow/_durabletask/worker.py

Comment thread ext/dapr-ext-workflow/docs/concurrency.md

Address Copilot feedback (6)

7ec820e

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv requested a review from Copilot June 2, 2026 09:34

Copilot AI reviewed Jun 2, 2026

View reviewed changes

seherv added 3 commits June 2, 2026 12:51

Merge branch 'main' into async-compat

9185482

Reword warning

73add2e

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Cleanup

5709bcd

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv marked this pull request as ready for review June 2, 2026 13:00

seherv requested review from a team as code owners June 2, 2026 13:00

Remove strands-agents-tools dependency

e68141c

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv marked this pull request as draft June 2, 2026 17:26

seherv added 3 commits June 2, 2026 23:14

Redo benchmarks and add performance regression tests

feb60db

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Merge branch 'main' into async-compat

70c6fad

Relax performance thresholds for CI

5fb88e6

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv force-pushed the async-compat branch from c9e325e to 5fb88e6 Compare June 2, 2026 21:43

seherv marked this pull request as ready for review June 2, 2026 21:52

seherv force-pushed the async-compat branch from 22ba468 to 4f7e85b Compare June 3, 2026 09:34

More async detection tests

6eb9ce0

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

seherv force-pushed the async-compat branch from 4f7e85b to 6eb9ce0 Compare June 3, 2026 09:45

seherv and others added 3 commits June 9, 2026 13:46

Create gRPC channel in the caller's event loop

8cf248b

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Silence gRPC error spam on EAGAIN

a73e994

Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>

Merge branch 'main' into async-compat

a2d4ad8

sicoyle reviewed Jun 9, 2026

View reviewed changes

		return rows


		async def run_sustained() -> tuple[SustainedMetrics, SustainedMetrics]:


		{_format_sustained_table(sustained_sync, sustained_async)}

		See `ext/dapr-ext-workflow/docs/concurrency.md` for sizing guidance.

		)


		ActivityFactory = Callable[[dict[int, float], dict[int, float]], Callable[..., object]]

Conversation

seherv commented May 25, 2026

Description

Issue reference

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seherv commented Jun 2, 2026

Uh oh!

acroca commented Jun 4, 2026

Uh oh!

codecov Bot commented May 25, 2026 •

edited

Loading