Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the UserIdleProcessor docs for the retry callback #108

Merged
merged 1 commit into from
Jan 30, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 94 additions & 31 deletions server/utilities/user-idle-processor.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,19 @@ title: "UserIdleProcessor"
description: "A processor that monitors user inactivity and triggers callbacks after specified timeout periods"
---

The `UserIdleProcessor` is a specialized frame processor that monitors user activity in a conversation and executes callbacks when the user becomes idle. It's particularly useful for maintaining engagement by detecting periods of user inactivity.
The `UserIdleProcessor` is a specialized frame processor that monitors user activity in a conversation and executes callbacks when the user becomes idle. It's particularly useful for maintaining engagement by detecting periods of user inactivity and providing escalating responses to inactivity.

## Constructor Parameters

<ParamField path="callback" type="Callable" required>
An async function that will be called when user inactivity is detected. The
function should accept the processor instance as its argument.
<ParamField path="callback" type="Union[BasicCallback, RetryCallback]" required>
An async function that will be called when user inactivity is detected. Can be
either:

- Basic callback: `async def(processor: UserIdleProcessor) -> None`

- Retry callback: `async def(processor: UserIdleProcessor, retry_count: int) ->
bool` where returning `False` stops idle monitoring

</ParamField>

<ParamField path="timeout" type="float" required>
Expand All @@ -24,39 +30,95 @@ The processor starts monitoring for inactivity only after the first conversation
- Pauses idle monitoring while user is speaking
- Resets idle timer when bot is speaking
- Stops monitoring on conversation end or cancellation
- Manages a retry count for the retry callback
- Stops monitoring when retry callback returns `False`

## Properties

<ParamField path="retry_count" type="int">
The current number of retry attempts made to engage the user.
</ParamField>

## Example Implementation

Here's a complete example showing how to use the UserIdleProcessor in a pipeline:

```python
async def user_idle_callback(user_idle: UserIdleProcessor):
messages.append({
"role": "system",
"content": "Ask the user if they are still there and try to prompt for some input."
})
await user_idle.push_frame(LLMMessagesFrame(messages))

# Create the processor
user_idle = UserIdleProcessor(
callback=user_idle_callback,
timeout=5.0
)

# Add to pipeline
pipeline = Pipeline([
transport.input(),
user_idle, # Add the processor to monitor user activity
context_aggregator.user(),
# ... rest of pipeline
])
```
## Example Implementations

Here are two example showing how to use the `UserIdleProcessor`: one with the basic callback and one with the retry callback:

<Tabs>
<Tab title="Basic Callback">
```python
async def handle_idle(user_idle: UserIdleProcessor) -> None:
messages.append({
"role": "system",
"content": "Ask the user if they are still there and try to prompt for some input."
})
await user_idle.push_frame(LLMMessagesFrame(messages))

# Create the processor
user_idle = UserIdleProcessor(
callback=handle_idle,
timeout=5.0
)

# Add to pipeline
pipeline = Pipeline([
transport.input(),
user_idle, # Add the processor to monitor user activity
context_aggregator.user(),
# ... rest of pipeline
])
```

</Tab>
<Tab title="Retry Callback">
```python
async def handle_user_idle(user_idle: UserIdleProcessor, retry_count: int) -> bool:
if retry_count == 1:
# First attempt: Gentle reminder
messages.append({
"role": "system",
"content": "The user has been quiet. Politely and briefly ask if they're still there."
})
await user_idle.push_frame(LLMMessagesFrame(messages))
return True
elif retry_count == 2:
# Second attempt: Direct prompt
messages.append({
"role": "system",
"content": "The user is still inactive. Ask if they'd like to continue our conversation."
})
await user_idle.push_frame(LLMMessagesFrame(messages))
return True
else:
# Third attempt: End conversation
await user_idle.push_frame(
TTSSpeakFrame("It seems like you're busy right now. Have a nice day!")
)
await task.queue_frame(EndFrame())
return False # Stop monitoring

# Create the processor
user_idle = UserIdleProcessor(
callback=handle_user_idle,
timeout=5.0
)

# Add to pipeline
pipeline = Pipeline([
transport.input(),
user_idle, # Add the processor to monitor user activity
context_aggregator.user(),
# ... rest of pipeline
])
```

</Tab>
</Tabs>

## Frame Handling

The processor handles the following frame types:

- `UserStartedSpeakingFrame`: Marks user as active and resets idle timer
- `UserStartedSpeakingFrame`: Marks user as active, resets idle timer and retry count
- `UserStoppedSpeakingFrame`: Starts idle monitoring
- `BotSpeakingFrame`: Resets idle timer
- `EndFrame` / `CancelFrame`: Stops idle monitoring
Expand All @@ -65,3 +127,4 @@ The processor handles the following frame types:

- The idle callback won't be triggered while the user or bot is actively speaking
- The processor automatically cleans up its resources when the pipeline ends
- Basic callbacks are supported for backward compatibility