claude-frontend-control
How to build web frontends where Claude can trigger actions and control UI state.
Purpose
Enable Claude to act as an agent that can manipulate your frontend application - navigating pages, showing dialogs, updating state, triggering animations, and more.
Key Concepts
Agentic UI
An “agentic UI” is a frontend where an AI agent has the ability to:
- Execute actions on behalf of the user
- Manipulate UI state
- Navigate between views
- Trigger visual feedback
- Interact with components programmatically
Why Not ACP?
The Agent Client Protocol (ACP) from Zed is designed for code editors communicating with AI coding agents via stdio. It’s not suitable for web frontends because:
- Uses stdio (not available in browsers)
- Agent processes are child processes of the editor
- Designed specifically for code editing workflows
For web frontends, Tool Use / Function Calling is the right approach.
Architecture Patterns
Pattern 1: Tool Use via Backend Proxy
The most robust approach. Your backend mediates between Claude API and your frontend.
┌─────────────┐ WebSocket ┌─────────────┐ Claude API ┌─────────┐│ Frontend │ ←───────────────→ │ Backend │ ←──────────────→ │ Claude ││ (React/ │ │ (Node/ │ │ API ││ Vue/etc) │ │ Python) │ │ │└─────────────┘ └─────────────┘ └─────────┘ ↑ │ │ │ └───── Tool execution results ──────┘Flow:
- User sends message through your UI
- Backend calls Claude API with tool definitions
- Claude responds with
tool_useblocks - Backend forwards tool calls to frontend via WebSocket
- Frontend executes action, returns result
- Backend sends result back to Claude
- Claude continues (may call more tools or respond)
Pattern 2: Direct API from Frontend
Simpler but exposes API key. Only for prototypes or with proper key rotation.
┌─────────────┐ Claude API ┌─────────┐│ Frontend │ ←───────────────→ │ Claude ││ │ (tool_use) │ API │└─────────────┘ └─────────┘Pattern 3: Streaming + Real-time Control
For interactive experiences where Claude’s actions appear immediately as it “thinks.”
Uses Claude’s streaming API to receive tool calls as they’re generated, executing them in real-time.
Tool Definitions
Define tools that map to frontend capabilities:
Navigation Tools
{ "name": "navigate", "description": "Navigate to a different page or route in the application", "input_schema": { "type": "object", "properties": { "path": { "type": "string", "description": "The route path, e.g., '/dashboard', '/settings/profile'" } }, "required": ["path"] }}UI State Tools
{ "name": "update_ui_state", "description": "Update a piece of UI state in the application", "input_schema": { "type": "object", "properties": { "key": { "type": "string", "description": "State key path, e.g., 'sidebar.open', 'theme'" }, "value": { "description": "New value for the state" } }, "required": ["key", "value"] }}Dialog/Modal Tools
{ "name": "show_dialog", "description": "Display a dialog or modal to the user", "input_schema": { "type": "object", "properties": { "type": { "type": "string", "enum": ["info", "warning", "error", "confirm"], "description": "Type of dialog" }, "title": { "type": "string", "description": "Dialog title" }, "content": { "type": "string", "description": "Dialog message or content" }, "actions": { "type": "array", "items": { "type": "object", "properties": { "label": { "type": "string" }, "action": { "type": "string" } } }, "description": "Optional action buttons" } }, "required": ["type", "title", "content"] }}Form Tools
{ "name": "fill_form", "description": "Fill in form fields with specified values", "input_schema": { "type": "object", "properties": { "form_id": { "type": "string", "description": "Identifier for the form" }, "fields": { "type": "object", "description": "Key-value pairs of field names and values" } }, "required": ["form_id", "fields"] }}Visual Feedback Tools
{ "name": "highlight_element", "description": "Visually highlight an element to draw user attention", "input_schema": { "type": "object", "properties": { "selector": { "type": "string", "description": "CSS selector or element ID" }, "duration_ms": { "type": "number", "description": "How long to highlight in milliseconds" }, "style": { "type": "string", "enum": ["pulse", "glow", "outline", "shake"], "description": "Highlight animation style" } }, "required": ["selector"] }}Frontend Implementation
Tool Executor Pattern
// Frontend tool executorclass ToolExecutor { private handlers: Map<string, (input: any) => Promise<any>> = new Map();
register(name: string, handler: (input: any) => Promise<any>) { this.handlers.set(name, handler); }
async execute(toolName: string, input: any): Promise<any> { const handler = this.handlers.get(toolName); if (!handler) { throw new Error(`Unknown tool: ${toolName}`); } return handler(input); }}
// Register handlersconst executor = new ToolExecutor();
executor.register('navigate', async ({ path }) => { router.push(path); return { success: true, navigated_to: path };});
executor.register('show_dialog', async ({ type, title, content }) => { const result = await dialogService.show({ type, title, content }); return { success: true, user_action: result };});
executor.register('update_ui_state', async ({ key, value }) => { store.set(key, value); return { success: true, updated: key };});WebSocket Message Protocol
// Messages from backend to frontendinterface ToolCallMessage { type: 'tool_call'; call_id: string; tool_name: string; input: any;}
// Messages from frontend to backendinterface ToolResultMessage { type: 'tool_result'; call_id: string; result: any; error?: string;}Security Considerations
Tool Allowlisting
Only enable tools appropriate for your application. Don’t expose:
- Arbitrary code execution
- Direct DOM manipulation
- Unrestricted API calls
User Confirmation
For sensitive actions, require user confirmation:
executor.register('delete_item', async ({ item_id }) => { const confirmed = await confirmDialog(`Delete item ${item_id}?`); if (!confirmed) { return { success: false, reason: 'User cancelled' }; } await api.deleteItem(item_id); return { success: true };});Rate Limiting
Prevent runaway tool calls:
const rateLimiter = new RateLimiter({ maxCalls: 10, windowMs: 1000 });
async function executeWithLimit(tool: string, input: any) { if (!rateLimiter.allow()) { return { error: 'Rate limited', retry_after_ms: rateLimiter.retryAfter() }; } return executor.execute(tool, input);}Comparison: ACP vs Tool Use
| Aspect | ACP (Zed) | Tool Use (Claude API) |
|---|---|---|
| Transport | stdio | HTTP/WebSocket |
| Environment | Desktop editors | Web browsers |
| Agent location | Child process | Cloud API |
| Use case | Code editing | General UI control |
| Protocol | JSON-RPC | Claude Messages API |
| Web compatible | No | Yes |
Related Protocols
MCP (Model Context Protocol)
Anthropic’s protocol for connecting Claude to external data sources and tools. Could be used alongside tool use for:
- Accessing databases
- File system operations
- External API integrations
Computer Use
Claude can directly see and interact with screens via screenshots and mouse/keyboard control. Heavier but useful for:
- Testing applications
- Automating existing UIs without modification
- Demo/tutorial generation
Sources
- Claude Function Calling - Composio
- Advanced Tool Use - Anthropic
- Computer Use Tool - Claude Docs
- Claude API Integration Guide 2025
- Zed ACP - For comparison
- ACP GitHub
Created: 2025-12-11