Python: Add hosting protocol helper surface#6891
Conversation
Introduce AgentFrameworkState and SessionStore for app-owned hosting routes, add Responses run conversion/rendering helpers, and update the local Responses sample to use native FastAPI routing with streaming support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
giles17
added
documentation
Python Test Coverage Report •
Python Unit Test Overview
|
||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Pull request overview
This PR introduces a helper-first Python hosting surface: agent-framework-hosting now provides shared execution/session state primitives for app-owned routes, and agent-framework-hosting-responses provides OpenAI Responses protocol conversion helpers (request → run args, run result → Responses JSON/SSE). It also updates the local Responses hosting sample to use native FastAPI routing to demonstrate the intended developer experience.
Changes:
- Added
AgentFrameworkState,SessionStore, and run-args TypedDicts (AgentRunArgs,WorkflowRunArgs) toagent-framework-hosting. - Added Responses helper functions (
responses_to_run,responses_from_run,responses_stream_events_from_run,responses_session_id,create_response_id) and expanded output rendering to map richer AF content into Responses output items. - Updated the
local_responsessample + docs to use FastAPI routes and the new helper/state surface (streaming via SSE and non-streaming JSON).
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| python/samples/04-hosting/af-hosting/README.md | Updates sample index text to reflect the helper-first + FastAPI route approach. |
| python/samples/04-hosting/af-hosting/local_responses/README.md | Reframes the sample around Responses helpers + app-owned FastAPI routes instead of channel run hooks. |
| python/samples/04-hosting/af-hosting/local_responses/pyproject.toml | Adds FastAPI dependency for the updated sample. |
| python/samples/04-hosting/af-hosting/local_responses/call_server_af.py | Updates the AF-backed client script to match the new (non-streaming) interaction pattern. |
| python/samples/04-hosting/af-hosting/local_responses/app.py | Replaces channel host usage with a native FastAPI /responses route using AgentFrameworkState + Responses helpers. |
| python/packages/hosting/tests/hosting/test_state.py | Adds unit tests for SessionStore and AgentFrameworkState behaviors. |
| python/packages/hosting/README.md | Updates package positioning and quickstart toward app-owned hosting with shared state helpers. |
| python/packages/hosting/agent_framework_hosting/_state.py | Introduces the new state/session primitives and TypedDict run-args surfaces. |
| python/packages/hosting/agent_framework_hosting/init.py | Exports the new state/session/run-args surface from the package. |
| python/packages/hosting-responses/tests/hosting_responses/test_parsing.py | Adds tests for new Responses helper functions and richer output-item mapping. |
| python/packages/hosting-responses/README.md | Updates docs from “channel” framing to “helpers for app-owned routes” with an example. |
| python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py | Adds helper APIs and full Responses output/SSE rendering backed by OpenAI SDK Pydantic models. |
| python/packages/hosting-responses/agent_framework_hosting_responses/init.py | Re-exports the new helper functions. |
| _RESPONSES_TRANSPORT_KEYS = frozenset({"input", "stream", "previous_response_id"}) | ||
| _RESPONSES_RUN_TRANSPORT_KEYS = frozenset({"input", "stream", "previous_response_id", "conversation_id"}) |
There was a problem hiding this comment.
Leaving this one alone for now — parse_responses_request/_RESPONSES_TRANSPORT_KEYS back the old ResponsesChannel/_channel.py path, which is being removed in a follow-up (alongside the Telegram work), so I don't want to touch it here. responses_to_run (the new helper this PR adds) already excludes conversation_id via _RESPONSES_RUN_TRANSPORT_KEYS, so the new code path is unaffected.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 85%
✓ Correctness
The PR adds hosting protocol helpers (AgentFrameworkState, SessionStore, and Responses conversion functions) with a clean separation between framework-owned state and app-owned routing. The implementation is correct: ResponseStream iteration semantics are properly handled (auto-finalization on consumption, safe re-call of get_final_response), AgentResponse.messages is always a list[Message] matching the Sequence check in result_to_output_items, and the session_id routing logic (conv prefix → conversation field, resp_ prefix → omitted) is consistent. No bugs, race conditions, or incorrect API usage found.
✓ Security Reliability
The PR introduces hosting protocol helpers and a shared state surface. The code is generally well-structured with appropriate input validation (empty session_id checks, type guards on body fields). One moderate reliability concern:
AgentFrameworkState.get_target()has a race condition for async target factories under concurrent access — the check-then-await-then-set sequence lacks synchronization, so concurrent calers can bypass the intended once-only initialization promise.
✓ Test Coverage
Test coverage for the new helper functions is reasonable for the core happy paths (text, reasoning, function_call, function_result, streaming) but leaves a substantial rendering surface untested. The ~600 lines of content-type rendering code in
_parsing.py(code_interpreter, image_generation, mcp, shell, approval, media) are duplicated from_channel.pyand NOT covered by the existing channel tests since those exercise the channel's own copy. TheAgentFrameworkStatetests cover the main paths but missreset_session()and documented error paths (ValueError, TypeError).
✓ Failure Modes
The new helper surface is well-structured and the rendering logic is thorough. The primary failure-mode concern is a TOCTOU race in
AgentFrameworkState.get_target()andget_session_store()when async target factories are used under concurrent requests: theawaitat line 145 of_state.pycreates a yield point between the cache check and cache write, which can lead to duplicate target creation and, more critically, splitSessionStoreinstances where sessions created in the first store are silently lost to subsequent requests that use the second (cached) store.
✗ Design Approach
I found two design-level gaps in the new helper-first surface. First, the sample and README wire
SessionStoredirectly toprevious_response_id, which only preserves continuity for the second turn and silently starts a freshAgentSessionon turn 3+ unless callers also invent a stableconversation_id. Second, the new streaming helper only emits text deltas, so streamed reasoning/tool-call updates are dropped until the finalresponse.completedevent even though the existing channel/client in this repo already support richer Responses event shapes.
Flagged Issues
- Session continuity in the helper-first sample breaks after turn 2 because
local_responses/app.py:111keysSessionStorebyresponses_session_id(...), and_parsing.py:177-196returns the changingprevious_response_idwhile_state.py:31-47only reuses sessions for identical keys. - The new SSE helper in
_parsing.py:906-914streams onlyresponse.output_text.delta, while the existing Responses channel already emits per-content events in_channel.py:368-456and447-756; streamed tool/reasoning updates are therefore silently suppressed until completion.
Automated review by eavanvalkenburg's agents
|
Flagged issue Session continuity in the helper-first sample breaks after turn 2 because Source: automated DevFlow PR review |
|
Flagged issue The new SSE helper in Source: automated DevFlow PR review |
- Fix constrained TargetT TypeVar in AgentFrameworkState: split __init__ into per-shape overloads (instance/sync factory/async factory/awaitable) since a bound TypeVar combined with one big Callable/Awaitable union parameter was unsolvable across pyright/pyrefly/ty/zuban. - Fix _FakeAgent test fixtures to structurally satisfy SupportsAgentRun (matching attribute types and overloaded run()), which the above surfaced. - Add SessionStore.put() to alias an additional session id to an already-resolved session, and use it in the local_responses sample to fix a real session-continuity bug: previous_response_id rotates every turn, so without aliasing the newly minted response id, turn 3+ of a conversation silently lost all prior history. Verified against a live Foundry model across a 3-turn conversation. - Fix responses_stream_events_from_run to report the real model instead of the "agent" fallback: AgentResponse.from_updates never carries a raw representation forward, so capture model from the individual streamed updates' raw representations instead. Verified live. - Add response_model=None to the sample's FastAPI route (it could not boot at all: FastAPI tried to build a Pydantic response model from the JSONResponse | StreamingResponse return annotation). - Map responses_to_run's ValueError to HTTP 400 instead of a 500. - Add HTTP round-trip integration tests (packages/hosting-responses) that exercise the same FastAPI + AgentFrameworkState + Responses helper wiring as the sample via httpx.ASGITransport, including a regression test for the session-continuity fix. - Add Workflow-target test coverage, SessionStore.put/reset_session tests, and TypeError-path coverage to packages/hosting/tests/hosting/test_state.py. - Extend call_server.py / call_server_af.py to a third conversation turn so they actually exercise the continuity chain (previous scripts stopped at turn 2, which would never have revealed the bug above). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation & Context
The Python hosting packages are moving toward a lighter helper-first shape where Agent Framework owns protocol-to-run conversion and shared execution state, while apps keep their native web framework routes, auth, middleware, background work, and response construction.
This implements the first part of that direction for the existing hosting and hosting-responses packages and updates the local Responses sample so reviewers can inspect the intended developer experience.
Description & Review Guide
AgentFrameworkState,SessionStore,AgentRunArgs, andWorkflowRunArgstoagent-framework-hosting.create_response_id,responses_session_id,responses_to_run,responses_from_run, andresponses_stream_events_from_run.local_responsesto use native FastAPI routing,AgentFrameworkState,SessionStore, non-streaming JSON, and streaming viaResponseStream.AgentFrameworkState,SessionStore, and the Responses helper functions.Related Issue
N/A — first implementation slice for the accepted-but-unreleased Python hosting helper direction. Related draft ADR PR: #6837. Checked for similar open PRs; existing hosting/channel PRs target the previous channel model or other protocols.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.