81e425b44f
* feat(durabletask): add workflow naming helpers (multi-workflow phase 0)
Foundation for hosting multiple workflows (and later sub-workflows) on one
durable task host. Adds a host-agnostic naming module that derives the stable
durable names a hosted workflow registers under.
- New `_workflows/naming.py`:
- `workflow_orchestrator_name(name)` -> `dafx-{name}` (orchestration name,
aligned byte-for-byte with .NET `WorkflowNamingHelper`).
- `workflow_name_from_orchestrator(name)` -> reverse, `None` when not prefixed.
- `validate_workflow_name(name)` -> rejects empty / malformed / auto-generated
`WorkflowBuilder-<uuid>` names (validate-and-reject rather than silently
sanitize, since the name becomes a durable identity and an HTTP route segment).
- `is_auto_generated_workflow_name(name)`, `DURABLE_NAME_PREFIX`.
- Export the helpers from the package public API.
- Mark `WORKFLOW_ORCHESTRATOR_NAME` deprecated in favor of per-workflow names
(kept functional; the single-workflow path still uses it until phase 1).
- 39 unit tests covering round-trips and validation.
Design: docs/design/durabletask-multiworkflow-and-subworkflows.md
* feat(durabletask): host multiple workflows per worker with scoped names (phase 1)
Enables hosting more than one MAF workflow on a single standalone Durable Task
worker, and aligns both hosts on workflow-scoped durable names so two co-hosted
workflows that reuse an executor id cannot collide.
Naming (shared, host-agnostic):
- orchestration: dafx-{workflowName} (matches .NET; the name DT tooling surfaces)
- non-agent activity / agent entity: dafx-{workflowName}-{executorId} (scoped)
- New naming helpers workflow_scoped_executor_id / workflow_executor_activity_name.
Standalone worker (agent-framework-durabletask):
- configure_workflow is now additive: stores workflows keyed by Workflow.name,
rejects duplicate / auto-generated (WorkflowBuilder-<uuid>) / invalid names,
registers one orchestrator per workflow plus its scoped activities/entities.
- The shared orchestrator dispatches scoped names derived from workflow.name.
- New registered_workflow_names property.
Client (DurableWorkflowClient):
- Optional default workflow_name on the client; start/run/stream accept a per-call
workflow_name and target dafx-{name}.
- Opt-in ownership validation on status/HITL methods: when a workflow name is
resolvable, an instance whose orchestration name does not match is treated as
not-found (status -> None, pending -> [], send_hitl_response / await -> raise),
mirroring the Azure Functions route-scoping check.
Azure Functions host (agent-framework-azurefunctions):
- Registration now uses the same scoped names so the shared orchestrator's
dispatch matches (single workflow per app for now; flat workflow/* routes kept).
- Workflow name is validated up front; workflow agents register under the scoped
entity id; _is_workflow_orchestration scopes to dafx-{workflow.name}.
Samples + tests:
- Durable Task and Azure Functions workflow samples now name their workflow.
- Unit tests cover multi-workflow registration, name validation, client targeting,
and ownership; integration tests target the named workflows.
WORKFLOW_ORCHESTRATOR_NAME remains exported (deprecated). This is a hard switch:
in-flight single-workflow instances created before upgrade (under the old
workflow_orchestrator name) will not resume.
Design: docs/design/durabletask-multiworkflow-and-subworkflows.md
* feat(azurefunctions): host multiple workflows per app with per-workflow routes (phase 2)
Completes multi-workflow hosting on the Azure Functions host, building on the
shared scoped-naming foundation from the worker phase.
AgentFunctionApp:
- New `workflows=` parameter accepting a list (keyed by each `Workflow.name`) or a
name->Workflow mapping; the existing `workflow=` is a single-workflow alias.
Both may be combined. Duplicate names and mapping-key/name mismatches are rejected.
- Each workflow registers its own `dafx-{name}` orchestration, workflow-scoped
activities/entities, and per-workflow HTTP routes:
`workflow/{name}/run`, `workflow/{name}/status/{instanceId}`,
`workflow/{name}/respond/{instanceId}/{requestId}`. Routes are always
per-workflow (even for a single workflow) so callers don't change URLs as an app
grows from one workflow to many.
- Route ownership check is per-workflow (`_is_owned_orchestration(status, name)`):
a leaked instance id for another orchestration -- or another workflow -- is
treated as not-found, extending the route-scoping defense.
- `get_agent(context, name, workflow_name=...)` resolves a workflow agent under its
scoped id; bare `agents=` registration keeps the standalone surface. New
`workflows` introspection property; `.workflow` now returns the sole workflow
(or None when several are hosted).
- Removed the now-unused flat-URL helper `_build_status_url` (handlers inline
per-workflow URLs).
Samples + tests:
- Azure Functions workflow samples (09-12) name their workflow; integration tests
target the per-workflow routes.
- Unit tests cover multi-workflow registration, duplicate/mapping/auto-name
rejection, and per-workflow ownership.
Note: sample README / demo.http route docs are updated in the docs phase.
Design: docs/design/durabletask-multiworkflow-and-subworkflows.md
* feat(durabletask): sub-workflows via durable child orchestrations (phase 3)
Run WorkflowExecutor nodes as durable child orchestrations on both hosts.
- Protocol: add call_sub_orchestrator to WorkflowOrchestrationContext, implemented by the durabletask and Azure Functions adapters.
- Registration: planner classifies WorkflowExecutor as subworkflow_executors; collect_hosted_workflows walks nested workflows (parent first, deduped by name). Both hosts recursively register every nested workflow's orchestration/agents/activities once; only top-level workflows get HTTP routes. Names validated up front before any registration side effects.
- Orchestrator: dispatch WorkflowExecutor nodes via call_sub_orchestrator(dafx-{innerName}) with deterministic child instance ids ({instanceId}::{executorId}::{counter}), a trusted-input marker carrying nesting depth (bounded at 25), and outputs routed as messages (default) or parent outputs (allow_direct_output).
- Tests: registration/collect, orchestrator prepare/process/unwrap, recursive registration on both hosts. Sample: 11_subworkflow.
* feat(durabletask): sub-workflow HITL via qualified request ids (phase 4)
Surface a nested sub-workflow's human-in-the-loop request behind the top-level instance (B2 single addressing surface).
- Orchestrator records dispatched sub-workflow child instance ids in its custom status (subworkflows map) before suspending in task_all, so the read side can reach a child's pending request while the parent is paused.
- Read side (durabletask client get_pending_hitl_requests; AF status route) recurses into nested child statuses, qualifying each nested request id as {executorId}::{requestId} (accumulated for deeper nesting).
- Write side (durabletask client send_hitl_response; AF respond route) splits a qualified id on '::', resolves the owning child orchestration via the parent's subworkflows map, and raises the event on the leaf child with the bare request id. Unknown/inactive sub-workflow -> error/404.
- Shared SUBWORKFLOW_REQUEST_SEPARATOR ('::') in naming so both hosts and the client agree. respondUrl/respond always targets the top-level instance.
- Tests: TestSubworkflowHitl (durabletask client, 7), TestAgentFunctionAppSubworkflowHitl (AF, 7). Sample: 12_subworkflow_hitl (HITL pause inside an embedded sub-workflow).
* docs(durabletask): ADR + sample route docs for multi-workflow and sub-workflows (phase 5)
- Add ADR-0030 capturing the multi-workflow and sub-workflow hosting decisions (naming, scoped inner names, per-workflow routes, child-orchestration sub-workflows, hard-switch migration, B2 sub-workflow HITL, scoped agent addressing) with considered alternatives; mark the design doc as implemented and link the ADR.
- Update Azure Functions workflow samples (09-12) README/demo.http to the per-workflow route shape (workflow/{name}/run|status|respond) introduced in phase 2.
- Extend the durabletask sample catalog with the workflow hosting patterns (08-12), including the new 11_subworkflow and 12_subworkflow_hitl samples.
* fix(durabletask): harden sub-workflow hosting + add sub-workflow integration tests
Post-review hardening of the multi-workflow / sub-workflow durable hosting:
- Trust boundary: strip the reserved sub-workflow envelope key from untrusted
client input at both host boundaries (DurableWorkflowClient.start_workflow and
the AF start route) so a forged envelope cannot reach the trusted pickle path.
- Nested HITL addressing: qualify nested pending requests by (executorId, ordinal)
using a '~' separator (was '::', which collided with core's auto::N functional
request ids); the parent status subworkflows map is now a per-executor list so
multiple children dispatched in one superstep stay independently addressable.
- Reject two different workflow instances that share a name (the same instance
reused by sibling nodes is still deduped); validate executor ids (separator-free,
length-bounded) when hosting durably.
- Remove the arbitrary sub-workflow nesting depth cap: a WorkflowExecutor wraps a
concrete Workflow so the nesting tree is finite at build time, and the durable
instance-id length limit is the natural ceiling (matches .NET, which has none).
Tests/samples:
- New durabletask integration tests for sub-workflow composition (11) and nested
sub-workflow HITL (12); new no-agent AF sub-workflow HITL sample (13) + test.
- Exempt no-agent samples from the model-credential gate in both integration
conftests so the nested-HITL plumbing is covered deterministically.
- Update durabletask sample 12 docs to the new qualified-id format.
Validated: 484 unit tests; durabletask integration 08/09/11/12 and AF 12/13 pass
against the live emulators; pyright 0 errors; ruff clean.
* fix(durabletask): address PR review feedback on naming, typing, and docs
- Unquote df.DurableOrchestrationClient annotations so pyupgrade passes.
- Narrow the split_subworkflow_request_id result before unpacking in a naming test so the strict type checkers pass.
- Correct the durabletask sample catalog to the {executor}~{ordinal}~{requestId} qualified id format.
- Reword the Azure Functions sub-workflow sample intro so it does not imply a difference from a same-numbered sample.
- Drop internal shorthand (B2, phase labels) from code comments.
* fix(durabletask): reject case-insensitive workflow name collisions
The route ownership guard compares the durable orchestration name with casefold(), but registration kept raw names as distinct keys. Hosting 'Orders' and 'orders' therefore succeeded while either workflow's status/respond route could operate on the other's instances. Reject case-insensitive name collisions at registration (within a composition via collect_hosted_workflows, and across registration calls via the case-folded _registered_orchestrations map and the top-level guard in both hosts) so the case-folded ownership boundary stays real. Single names of any case remain valid; only collisions are rejected.
* docs(durabletask): remove multiworkflow/subworkflow ADR and design docs
Drop the ADR and design exploration documents and the dangling docstring reference to them.
* refactor(durabletask): simplify workflow client status parsing and drop deprecated orchestrator-name symbols
Extract a shared _parse_custom_status helper in DurableWorkflowClient to remove duplicated custom-status JSON parsing across three call sites.
Drop the now-unused single-workflow compatibility shims WORKFLOW_ORCHESTRATOR_NAME and WorkflowRegistrationPlan.orchestrator_name, replaced by per-workflow workflow_orchestrator_name(name).
* fix(core): drop WORKFLOW_ORCHESTRATOR_NAME from agent_framework.azure re-exports
The constant was removed from agent-framework-durabletask, but the core azure lazy-loading namespace still re-exported it, breaking pyright in packages/core. Remove it from both the runtime _IMPORTS map and the .pyi stub.
* fix(durabletask): atomic multi-workflow registration and bubble sub-workflow events
Make configure_workflow / AgentFunctionApp registration atomic: check every cross-call name collision before mutating any state, so a colliding nested sub-workflow no longer leaves a host partially configured (with the top-level name stuck in the registry). Applied to both the standalone worker and the Functions app.
Bubble sub-workflow intermediate events: a workflow run as a child orchestration now returns a SUBWORKFLOW_RESULT_KEY envelope carrying its outputs plus event timeline, and the parent re-tags the child's intermediate events with the WorkflowExecutor node id and republishes them, matching the in-process WorkflowExecutor contract. Top-level runs still return a bare outputs list.
Adds cross-registration atomicity tests on both hosts and unit tests for the result envelope and event bubbling. Resolves review threads on _worker.py, orchestrator.py, and test coverage.
* fix(azurefunctions): widen workflow orchestrator wrapper return type
The shared run_workflow_orchestrator now returns list | dict (the sub-workflow result envelope), so the azurefunctions _workflow.py wrapper that delegates to it must widen its Generator return annotation to match. Caught by the package-level pyright in CI (Package Checks), which type-checks the whole package, not just the files changed in the previous commit.
79 lines
2.7 KiB
Python
79 lines
2.7 KiB
Python
# Copyright (c) Microsoft. All rights reserved.
|
|
|
|
"""Client that starts the composed workflow orchestration and prints the result.
|
|
|
|
The worker (``worker.py``) must be running first. Only the *outer* workflow is
|
|
started by the client; its embedded sub-workflow runs automatically as a durable
|
|
child orchestration when the outer workflow reaches the ``WorkflowExecutor`` node.
|
|
|
|
The workflow is started via ``DurableWorkflowClient.start_workflow`` - which
|
|
schedules the ``dafx-review_pipeline`` orchestration that
|
|
``DurableAIAgentWorker.configure_workflow`` auto-registers for the outer workflow.
|
|
|
|
Prerequisites:
|
|
- ``worker.py`` running and connected to the same Durable Task Scheduler.
|
|
- A Durable Task Scheduler reachable at ``ENDPOINT`` (default ``http://localhost:8080``).
|
|
"""
|
|
|
|
import asyncio
|
|
import logging
|
|
import os
|
|
|
|
from agent_framework.azure import DurableWorkflowClient
|
|
from azure.identity import AzureCliCredential
|
|
from dotenv import load_dotenv
|
|
from durabletask.azuremanaged.client import DurableTaskSchedulerClient
|
|
|
|
load_dotenv()
|
|
|
|
logging.basicConfig(level=logging.INFO)
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# The client targets the outer workflow; the sub-workflow runs as a child orchestration.
|
|
WORKFLOW_NAME = "review_pipeline"
|
|
|
|
|
|
def get_client(taskhub: str | None = None, endpoint: str | None = None) -> DurableTaskSchedulerClient:
|
|
"""Create a configured DurableTaskSchedulerClient."""
|
|
taskhub_name = taskhub or os.getenv("TASKHUB", "default")
|
|
endpoint_url = endpoint or os.getenv("ENDPOINT", "http://localhost:8080")
|
|
|
|
credential = None if endpoint_url == "http://localhost:8080" else AzureCliCredential()
|
|
|
|
return DurableTaskSchedulerClient(
|
|
host_address=endpoint_url,
|
|
secure_channel=endpoint_url != "http://localhost:8080",
|
|
taskhub=taskhub_name,
|
|
token_credential=credential,
|
|
)
|
|
|
|
|
|
def run_workflow(client: DurableWorkflowClient, review: str) -> None:
|
|
"""Start the outer workflow with a review and wait for the result."""
|
|
instance_id = client.start_workflow(input=review)
|
|
logger.info("Started workflow instance: %s", instance_id)
|
|
|
|
output = client.await_workflow_output(instance_id)
|
|
logger.info("Workflow output: %s", output)
|
|
|
|
|
|
async def main() -> None:
|
|
"""Run the composed workflow against a couple of product reviews."""
|
|
client = DurableWorkflowClient(get_client(), workflow_name=WORKFLOW_NAME)
|
|
|
|
logger.info("TEST 1: Positive review")
|
|
run_workflow(
|
|
client,
|
|
"Absolutely love this espresso machine - it heats up fast and the coffee is consistently great.",
|
|
)
|
|
|
|
logger.info("TEST 2: Negative review")
|
|
run_workflow(
|
|
client,
|
|
"Disappointed. The device stopped working after two weeks and support never replied.",
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
asyncio.run(main())
|