fix(eve): surface terminal model-call failures from subagents to the parent#454
Open
enjoykumawat wants to merge 1 commit into
Open
fix(eve): surface terminal model-call failures from subagents to the parent#454enjoykumawat wants to merge 1 commit into
enjoykumawat wants to merge 1 commit into
Conversation
…parent
A terminal MODEL_CALL_FAILED classification (structural 4xx, invalid API
key, unresolvable model id, etc.) returned {done: true, output: ""} from
the tool loop regardless of run mode. For a subagent ("task" mode), this
was indistinguishable from success once rolled up into the parent's
subagent-result tool call, so the orchestrator reported the run as
completed even though the subagent's model call failed terminally.
The non-terminal (recoverable/retry-exhausted) failure path already
special-cased task mode to return {done: true, isError: true, output}
so the parent sees the failure. Apply the same mode check to the
terminal path.
Fixes vercel#412
Signed-off-by: Enjoy Kumawat <ekumawat54@gmail.com>
Contributor
|
@enjoykumawat is attempting to deploy a commit to the Vercel Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #412
Problem
A terminal
MODEL_CALL_FAILEDclassification (structural 4xx, invalid API key, unresolvable model id, etc.) madecreateToolLoopHarness's step function return{ next: { done: true, output: "" }, session }regardless of run mode.For a subagent (
config.mode === "task"), that result is whatfinalizeDoneinworkflow-entry.tsrolls up into the parent'ssubagent-resulttool call viacreateDelegatedSubagentSuccessResult/createDelegatedSubagentErrorResult, gated onaction.isError === true. Since the terminal branch never setisError, a subagent whose model call failed terminally (e.g. a 404 for an unresolvable model id, matching the repro in #412) was reported to the parent as a successful call with empty output. The orchestrator then continued and the top-level session reported success even though the subagent's model call failed.The non-terminal (recoverable / retry-exhausted) failure path a few lines below already special-cases
config.mode === "task"and returns{ done: true, isError: true, output: errorMessage }for exactly this reason — the terminal branch above it was just missing the same check.Fix
Apply the same
config.mode === "task"check to the terminal branch, so a terminal model-call failure in a subagent surfaces as{ done: true, isError: true, output: errorMessage }instead of silently swallowing the failure. Non-task-mode behavior ({ done: true, output: "" }, tearing down the conversation session) is unchanged.Testing
Added a regression test in
tool-loop.test.tsmirroring the existing "structural 4xx" terminal-cascade test, but intaskmode, assertingresult.nextcarriesisError: trueand the error message instead of an empty success output. Confirmed it fails onmainand passes with the fix.Ran the full
packages/eveunit suite before/after — no new failures (a handful of pre-existing Windows-path-separator failures in unrelated CLI/nitro-host tests are present onmainas well, unrelated to this change).Added a changeset (patch) for the published
evepackage.