feat: add /v1/responses/compact endpoint - #137
Merged
Merged
Conversation
Implements the OpenAI/OpenResponses compaction endpoint: real model-driven summarization of the resolved conversation, returned as a Fernet-encrypted opaque blob (MSHIP_COMPACTION_KEY, with an ephemeral-per-process fallback) that decodes back into context on a later /v1/responses call. Shares history resolution with /v1/responses via a new field-based resolve_history_items, and factors the RayTaskError->4xx mapping out of _handle_response so both routes get the same error handling. Verified end-to-end against the OpenResponses compliance suite (compact-response, compact-missing-model) on both the vllm and llama_server loaders.
There was a problem hiding this comment.
Pull request overview
Adds first-class support for the OpenAI/OpenResponses “compaction” flow by introducing a new POST /v1/responses/compact route that summarizes prior conversation state and returns it as an opaque blob that can be replayed later as a compaction input item on /v1/responses.
Changes:
- Adds
/v1/responses/compactgateway route, sharing history-resolution logic with/v1/responsesand factoring initial Ray generator error mapping into a shared helper. - Introduces Fernet-based encryption/decryption for compaction blobs (
encrypted_content) and extends the Responses adapter to decodecompactioninput items back into messages. - Adds new protocol schemas for compaction request/response types and comprehensive unit tests for crypto, schemas, adapter decoding, and routing.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Adds cryptography dependency required for Fernet blob encryption/decryption. |
uv.lock |
Locks the new cryptography dependency. |
modelship/openai/compaction_crypto.py |
Implements Fernet encryption/decryption for encrypted_content blobs with an env-configured key and an ephemeral fallback. |
modelship/openai/utils/responses.py |
Adds compaction request builders and factors history resolution into a field-based helper shared by both routes. |
modelship/openai/protocol/responses/schemas.py |
Adds CompactRequest, CompactResource, and CompactionItem protocol models. |
modelship/openai/protocol/responses/adapter.py |
Adds support for decoding compaction input items by decrypting and splicing decoded items into chat messages. |
modelship/openai/api.py |
Implements the new /v1/responses/compact route and shares RayTaskError-to-4xx mapping via _await_first(). |
modelship/openai/protocol/responses/__init__.py |
Re-exports the new compaction protocol types. |
modelship/openai/protocol/__init__.py |
Re-exports the new compaction protocol types at the top-level protocol package. |
tests/test_responses_compaction.py |
Adds tests covering crypto behavior, schema serialization requirements, compaction builders, and the compact route. |
tests/test_responses_adapter.py |
Adds tests for decoding compaction input items and clean rejection on tampering/wrong key. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
messages_from_input's compaction decode branch claimed recursion was
"bounded" only because build_compaction happens to never encrypt another
compaction item — an invariant of our own code, not something the crypto
layer enforced. A forged or future-buggy blob that decrypts to another
{type: "compaction"} item would recurse without limit. Reject nesting
explicitly after decrypting.
No behavior change — condenses module/function docstrings and inline comments down to 1-2 lines each.
…I blobs - An invalid MSHIP_COMPACTION_KEY now fails fast with a clear error at resolution time instead of a cryptic ValueError surfacing mid-request. - decrypt_items() normalizes a non-ASCII blob's UnicodeEncodeError into InvalidToken, so callers map it to the same clean 400 as a tampered or wrong-key blob instead of leaking a raw 500.
CompactRequest exposed both fields but only model/input/previous_response_id affected behavior. instructions is now inserted as an additional system message during summarization; prompt_cache_key is dropped entirely since nothing in modelship hooks a cache key into anything (not even on /v1/responses) — extra="allow" keeps it a non-breaking removal for clients that still send it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the OpenAI/OpenResponses compaction endpoint: real model-driven summarization of the resolved conversation, returned as a Fernet-encrypted opaque blob (MSHIP_COMPACTION_KEY, with an ephemeral-per-process fallback) that decodes back into context on a later /v1/responses call. Shares history resolution with /v1/responses via a new field-based resolve_history_items, and factors the RayTaskError->4xx mapping out of _handle_response so both routes get the same error handling.
Verified end-to-end against the OpenResponses compliance suite (compact-response, compact-missing-model) on both the vllm and llama_server loaders.