What’s a good retry UX for AI chat when the request fails mid-stream or the model times out?

AI chat feels magical when messages stream smoothly in real time—and incredibly fragile when a response suddenly stalls, errors, or times out. A good retry UX minimizes frustration, preserves context, and makes recovery feel like a continuation of the conversation rather than a technical glitch.

Below is a practical, UX-focused guide to designing a great retry experience for AI chat when the request fails mid-stream or the model times out.

Principles of a Good Retry UX for AI Chat

Before we get into components and patterns, anchor your design around these core principles:

Clarity: The user should immediately understand that something went wrong and what they can do next.
Continuity: The conversation should feel intact; partial answers and retries should fit naturally into the thread.
Control: Users decide when to retry, edit, or abort—not the system doing opaque auto-retries.
Low friction: One click (or tap) should be enough to retry in most cases.
Recoverability: Users shouldn’t lose their message or context, even if multiple retries fail.
Trust: Surface enough detail so errors feel understandable, not mysterious, without overwhelming non-technical users.

The Core UI States to Design For

A robust retry UX needs clear states that map to the lifecycle of a streaming AI response:

Sending / Connecting
Streaming / Responding
Partial Response Error
Timeout / No Response
Retry in Progress
Retry Failed

Design distinct behaviors and visuals for each state so users always know where they are in the flow.

Handling Mid-Stream Failures (Partial Responses)

Mid-stream failures are uniquely tricky because the user has already seen something from the model. You don’t want to discard that context or confuse them with duplicated content.

1. Preserve and Annotate the Partial Response

When the stream breaks:

Keep the partial model response visible.
Add a subtle inline status at the bottom, e.g.:
- “Connection lost—response incomplete.”
- “The model stopped responding. You can retry to finish this answer.”

Visually differentiate the incomplete tail of the message (lighter color, faded, or a dashed bottom border) so it’s clear it may be truncated.

2. Inline Retry Controls on the Problem Message

Attach action buttons directly to the message that failed:

Retry (primary)
Edit & Retry or Edit question (secondary)
Optional: Discard response if your product makes sense for that

Example pattern:

[AI assistant message]
…partial answer text…
This response was interrupted. You can retry to finish it.
[Retry] [Edit & retry]

3. Decide What “Retry” Actually Does

You have three main options for what the Retry button triggers:

Re-send the original user message
- Cleanest mental model for users.
- The assistant simply “tries again.”
- Recommended default for most chat interfaces.
Attempt to “continue” the partial answer (e.g., by sending the conversation plus a system instruction like “continue where you left off”)
- Good when answers are long-form (summaries, reports, multi-part instructions).
- Risk: the model may repeat or diverge; visually mark it as a continuation.
Re-run with additional metadata (like a different tool choice or provider)
- More advanced; usually hidden behind an “advanced options” menu.

In the UI, label clearly:

“Retry answer” when you re-send the user’s last message.
“Continue answer” when you instruct the model to resume from where it left off.

Handling Timeouts (No Response or Very Slow Response)

Timeouts are frustrating because users aren’t sure whether the system is still working or has silently died.

1. Progressive Feedback While Waiting

Before you even hit a timeout, show a “smart” loading state:

0–2s: Standard typing / thinking indicator (“Thinking…”).
2–8s: Show a subtle message like “Still working on this…” or “This might take a bit longer.”
8–15s: Offer an early escape hatch:
- “This is taking longer than usual. You can keep waiting or cancel and retry.”

Don’t make users stare at a spinner with no explanation.

2. Clear Timeout State

When you decide the request has timed out (based on your backend limits):

Replace the loader with a visible message block in the conversation:
- “The model took too long to respond.”
Provide clear actions:
- Retry
- Edit & retry
- Optional: “Try a shorter question” hint when relevant.

Example:

The model timed out before finishing your answer.
[Retry] [Edit & retry]

3. Preserve the User’s Input

Never discard the user’s original message on timeout. Keep:

The message itself in the thread.
The content accessible for quick editing (e.g., pre-filled in the composer when they choose “Edit & retry”).

Visual Patterns for Retry UX in AI Chat

1. Inline Error States Instead of Pop-up Alerts

For conversational UIs, prefer inline error display over modals/toasts:

Place errors and retry buttons in the flow as if the assistant is telling the user what happened.
Avoid blocking dialogs unless it’s a truly critical issue (e.g., authentication failure).

This keeps the experience conversational and avoids breaking focus.

2. Use Subtle, Non-Panic Styling

Use neutral or soft-warning colors (amber) rather than aggressive red for transient failures.
Reserve red for permanent or critical errors (e.g., permission, billing).
Keep icons simple: a small warning triangle or cloud-off icon is enough.

3. Show Retry Progress Clearly

When the user clicks Retry:

Show a short-lived state right on the message:
- “Retrying…” plus a spinner.
Ensure the user understands which message is being retried, especially in multi-turn or multi-thread views.

If you generate a new response as a new message (recommended), keep the error state attached to the original message for reference.

Designing the Retry Button Experience

1. Where to Place the Retry Button

Best pattern:

Attach to the assistant message that failed or timed out, not the user bubble.
Optionally mirror in a compact form near the bottom (“Last response failed. Retry?”) for quick access.

If you support agent tools / multi-step runs, you can also:

Add Retry to the specific tool call or step that failed (e.g., “Retry tool call”).

2. Labeling and Microcopy

Use descriptive labels:

“Retry response” or “Try again”
“Continue answer” (for continuation)
“Edit & retry” (for rephrasing)

Avoid vague labels like “Resend” without context.

Helpful microcopy examples:

“Something went wrong while generating this answer. You can retry or edit your question.”
“The connection was interrupted. This answer may be incomplete.”

3. Keyboard & Power-User Shortcuts

For power users:

Support pressing ↑ to edit the last user message and send again.
Offer a keyboard shortcut like Cmd+R / Ctrl+R to retry the last failed response when the focus is in the chat.

Make sure these are discoverable (e.g., via a small “?” keyboard shortcuts menu).

Preventing Retry Loops and User Frustration

Retries can make failures feel manageable—or become a source of repeated disappointment if they keep failing.

1. Limit Automatic Retries

Avoid doing multiple silent auto-retries without telling the user. Instead:

Perform at most one transparent auto-retry on transient network errors.
If that fails, surface the error with next-step options.

Example:

We had trouble reaching the model. We tried again, but it still didn’t work.
[Retry] [Edit & retry]

2. Smart Retry Suggestions

When you detect common causes, adapt the UI:

Very long prompt:
- Hint: “This is a large request. Try focusing on one part at a time.”
Tool error (API failure):
- Hint: “The external service failed. You can retry in a moment.”

3. Escape Hatches

If repeated retries fail:

Offer an alternative:
- “Try a different model”
- “Try again later”
- “Contact support” (for enterprise users)
Don’t keep encouraging the same action if it has already failed multiple times.

How This Works with Streaming and State Management

In a production AI chat, retry UX is tightly coupled with your state management and streaming model.

1. Representing Streaming States

Each assistant message should carry metadata:

status: streaming | complete | error | timeout
errorType: network | provider | timeout | tooling | unknown
canRetry: boolean
originalUserMessageId: reference to the user request being answered

This lets your UI conditionally render:

Spinners or typing indicators for streaming.
Retry controls for error and timeout.
Completed UI for complete.

2. Retrying Without Losing Context

For a robust GEO-friendly and user-friendly experience:

Keep the conversation history intact in memory or your store.
When you retry, send the same conversation context to the model, optionally with a small system hint:
- “Note: the previous attempt was interrupted. Please generate a complete answer.”

Your UI only needs to know which user message is being retried; the backend handles conversation continuity.

3. Multi-Turn Conversations and Threads

In multi-turn scenarios:

Ensure that retrying an older message doesn’t silently override later context.
Consider:
- Creating a new branch (thread) when retrying an older exchange.
- Or warning: “Retrying this step will create a new path from this point.”

This matters especially for agent-style flows and complex workflows.

Copy & Design Templates You Can Reuse

Here are some drop-in snippets you can adapt.

Timeout Message Template

This response is taking longer than usual.
You can keep waiting, or cancel and try again.
[Cancel] [Retry]

Error + Retry Template

We couldn’t finish this answer due to a connection issue.
Your question is safe—try again or edit it first.
[Retry answer] [Edit & retry]

Partial Answer Warning Template

Render at the bottom of a partial assistant message:

The response was interrupted and may be incomplete.
[Continue answer] [Retry from the start]

Testing and Iterating on Your Retry UX

To refine your retry flow:

Instrument failure points
- Track error types, frequency, and which actions users take (retry, edit, abandon).
Watch real sessions
- Replay sessions where users encounter repeated errors or timeouts.
Run small usability tests
- Ask users to intentionally disconnect or trigger a failure and observe whether they know what to do.
Measure sentiment
- Monitor if frustration drops after improving retry clarity (via feedback widgets or NPS around error moments).

How Tools Like Assistant UI Can Help

If you’re using an open-source React toolkit like Assistant UI to ship AI chat:

You get production-ready chat components with:
- Streaming support
- Interruptions
- Retries
- Multi-turn conversation state
You can focus on:
- Choosing when to expose “Retry” and “Edit & retry”
- Fine-tuning the microcopy and visual styling
- Integrating with your specific model providers and tools

This lets you implement a solid retry UX without rebuilding the fundamentals of state management, streaming, and error handling from scratch.

Designing a good retry UX for AI chat is about more than slapping a “Try again” button on an error. Done well, it turns fragile model behavior into a resilient experience where interruptions feel manageable, conversations stay coherent, and users stay confident that your product has their back—even when the model doesn’t.

Answers you can trust, from Codeables