Managing Agents

Once your agents are live, you'll want to monitor performance, make updates, and optimize their behavior.

Admin Chat

Admin Chat lets you talk to your agent in real-time. Use it to:

Update configuration on the fly
Ask it to send emails
Check recent activity
Test behavior before updating live

How to Use Admin Chat

Go to agent details page
Click "Admin Chat" tab
Type a message
Agent responds in real-time

Example Commands

Update configuration:

Change your tone to friendly
Update your instructions to also handle billing questions
Set autonomy mode to semi-auto

Ask it to send emails:

Send an email to john@example.com saying we have a new feature
Send a follow-up to all prospects who haven't responded

Check status:

What's your current configuration?
How many emails have you processed today?
Show me recent activity

Test behavior:

How would you respond to a customer asking for a refund?
What would you do if someone mentioned legal action?

Monitoring Performance

Activity Feed

The Activity Feed shows everything your agent does:

Emails received and classified
Responses sent
Escalations
Tool usage
Integration actions
Errors

Review weekly:

Go to Activity Feed
Look for patterns
Check escalation rate
Review quality of responses

Key Metrics

Escalation Rate

Target: 5-15%
Too high (>30%): Agent needs better training
Too low (<5%): You might be missing issues

Response Time

Typical: 5-30 seconds
If slow: Disable unnecessary tools/integrations

Error Rate

Should be <1%
If high: Check instructions and integrations

Tool Usage

Web Search: How often used?
PDF Generation: Working correctly?
Integrations: Being used as expected?

Updating Agents

Update Instructions

Go to agent settings
Click "Configuration"
Update "Instructions" field
Click "Save Changes"
Changes take effect immediately

When to update:

After reviewing escalated emails
When you notice patterns
When adding new capabilities
When refining behavior

Update Escalation Rules

Go to agent settings
Click "Configuration"
Update "Escalation Rules" field
Click "Save Changes"

When to update:

Too many escalations? Narrow rules
Too few escalations? Broaden rules
New business rules? Add them

Update Tone

Go to agent settings
Click "Configuration"
Select new tone
Click "Save Changes"

Change Autonomy Mode

Go to agent settings
Click "Configuration"
Select new autonomy mode
Click "Save Changes"

Progression:

Week 1: Human in the Loop
Week 2: Semi-Autonomous
Week 3+: Full Auto (if appropriate)

Enable/Disable Tools

Go to agent settings
Click "Configuration"
Toggle tools on/off
Click "Save Changes"

Best practice: Only enable tools the agent actually needs.

Optimization Workflow

Weekly Review (15 minutes)

Check metrics
- Escalation rate
- Response time
- Error rate
Review escalations
- Read 5-10 escalated emails
- Look for patterns
- Note common issues
Update if needed
- Update instructions based on patterns
- Adjust escalation rules
- Enable/disable tools
Document changes
- Note what you changed
- Why you changed it
- Expected impact

Monthly Review (30 minutes)

Analyze trends
- Escalation rate trend
- Response quality trend
- Tool usage patterns
Gather feedback
- Ask team about agent performance
- Check customer feedback
- Review satisfaction metrics
Plan improvements
- What's working well?
- What needs improvement?
- What new capabilities needed?
Implement changes
- Update instructions
- Adjust rules
- Add new tools/integrations

Common Issues & Solutions

Too Many Escalations (>30%)

Symptoms:

Most emails are escalated
Agent seems unsure
Takes too long to process

Solutions:

Review escalation rules - are they too broad?
Update instructions - be more specific
Add examples to instructions
Enable Web Search for more knowledge
Provide knowledge base documents

Example:

Before: "Escalate if customer seems confused"
After: "Escalate if customer asks about features we don't have"

Too Few Escalations (<5%)

Symptoms:

Almost nothing escalates
Missing important issues
Customer complaints about responses

Solutions:

Broaden escalation rules
Add more conditions
Review what should escalate
Test with edge cases

Example:

Before: "Escalate if customer mentions legal action"
After: "Escalate if customer mentions legal action, lawsuit, or refund"

Poor Response Quality

Symptoms:

Responses are generic
Missing important details
Customers confused

Solutions:

Review instructions - are they clear?
Add specific examples
Enable Web Search
Provide knowledge base
Adjust tone

Example:

Before: "Answer customer questions"
After: "Answer questions about pricing, features, and billing. 
For technical issues, explain what you know and escalate. 
Always be friendly and helpful."

Slow Responses

Symptoms:

Takes >1 minute to respond
Customers waiting too long

Solutions:

Disable unnecessary tools
Disable unnecessary integrations
Simplify instructions
Check for errors in logs

Agent Not Using Tools

Symptoms:

Tools enabled but not used
Agent could use them but doesn't

Solutions:

Mention tools in instructions
Give examples of when to use
Check if agent understands
Test in Admin Chat

Example:

Before: "Answer customer questions"
After: "Answer customer questions. Use web search to find 
current information. Generate PDFs for invoices."

Team Management

Currently, only the creator can manage an agent. We're working on team features.

Workaround:

Share agent email with team
Share escalations via email
Document instructions in shared doc

Documenting Agents

Create a document for each agent:

Agent: Support Bot
Role: Customer Support Specialist
Tone: Friendly
Autonomy: Semi-Auto

Instructions:
[Full instructions]

Escalation Rules:
[Full rules]

Tools: Web Search, Send Email, Generate PDF
Integrations: Google Contacts

Owner: Sarah
Last Updated: 2026-03-29
Notes: [Any special notes]