feat (ui): add onToolCall callback to useChat (#1729)

Author: lgrammel

.changeset/swift-lamps-juggle.md

@@ -0,0 +1,5 @@
+---
+'ai': patch
+---
+
+chore (ui): deprecate old function/tool call handling

.changeset/three-foxes-dream.md

@@ -0,0 +1,5 @@
+---
+'ai': patch
+---
+
+feat (ui): add onToolCall handler to useChat

content/docs/05-ai-sdk-ui/02-chatbot-with-tool-calling.mdx

@@ -1,27 +1,37 @@
 ---
-title: Chatbot with Tool Calling
-description: Learn how to use tool calling with the useChat hook.
+title: Chatbot with Tools
+description: Learn how to use tools with the useChat hook.
 ---
 
-# Chatbot with Tool Calling
+# Chatbot with Tools
 
 <Note type="warning">
-  The tool calling functionality described here is experimental. It is currently
-  only available for React.
+  The tool calling functionality described here currently only available for
+  React.
 </Note>
 
 With `useChat` and `streamText`, you can use tools in your chatbot application.
-The Vercel AI SDK supports both client and server side tool execution.
+The Vercel AI SDK supports three types of tools in this context:
+
+1. Automatically executed server-side tools
+2. Automatically executed client-side tools
+3. Tools that require user interaction, such as confirmation dialogs
 
 The flow is as follows:
 
+1. The user enters a message in the chat UI.
+1. The message is sent to the API route.
+1. The messages from the client are converted to AI SDK Core messages using `convertToCoreMessages`.
 1. In your server side route, the language model generates tool calls during the `streamText` call.
-   The messages from the client are converted to AI SDK Core messages using `convertToCoreMessages`.
-2. All tool calls are forwarded to the client.
-3. Server-side tools are executed using their `execute` method and their results are sent back to the client.
-4. The client-side tools are executed on the client.
-   The results of client side tool calls can be appended to last assigned message using `experimental_addToolResult`.
-5. When all tool calls are resolved, the client sends the updated messages with the tool results back to the server, triggering another `streamText` call.
+1. All tool calls are forwarded to the client.
+1. Server-side tools are executed using their `execute` method and their results are forwarded to the client.
+1. Client-side tools that should be automatically executed are handled with the `onToolCall` callback.
+   You can return the tool result from the callback.
+1. Client-side tool that require user interactions can be displayed in the UI.
+   The tool calls and results are available in the `toolInvocations` property of the last assistant message.
+1. When the user interaction is done, `experimental_addToolResult` can be used to add the tool result to the chat.
+1. When there are tool calls in the last assistant message and all tool results are available, the client sends the updated messages back to the server.
+   This triggers another iteration of this flow.
 
 The tool call and tool executions are integrated into the assistant message as `toolInvocations`.
 A tool invocation is at first a tool call, and then it becomes a tool result when the tool is executed.
@@ -34,16 +44,15 @@ The tool result contains all information about the tool call as well as the resu
   for backward compatibility.
 </Note>
 
-## Example: Client-Side Tool Execution
+## Example
 
-In this example, we'll define two tools.
-The client-side tool is a confirmation dialog that asks the user to confirm the execution of the server-side tool.
-The server-side tool is a simple fake tool that restarts an engine.
+In this example, we'll use three tools:
 
-### Server-side route
+- `getWeatherInformation`: An automatically executed server-side tool that returns the weather in a given city.
+- `askForConfirmation`: A user-interaction client-side tool that asks the user for confirmation.
+- `getLocation`: An automatically executed client-side tool that returns a random city.
 
-Please note that only the `restartEngine` tool has an `execute` method and is executed on the server side.
-The `askForConfirmation` tool is executed on the client side.
+### API route
 
 ```tsx filename='app/api/chat/route.ts'
 import { openai } from '@ai-sdk/openai';
@@ -60,19 +69,30 @@ export async function POST(req: Request) {
     model: openai('gpt-4-turbo'),
     messages: convertToCoreMessages(messages),
     tools: {
-      restartEngine: {
-        description:
-          'Restarts the engine. ' +
-          'Always ask for confirmation before using this tool.',
-        parameters: z.object({}),
-        execute: async () => 'Engine restarted.',
+      // server-side tool with execute function:
+      getWeatherInformation: {
+        description: 'show the weather in a given city to the user',
+        parameters: z.object({ city: z.string() }),
+        execute: async ({}: { city: string }) => {
+          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
+          return weatherOptions[
+            Math.floor(Math.random() * weatherOptions.length)
+          ];
+        },
       },
+      // client-side tool that starts user interaction:
       askForConfirmation: {
         description: 'Ask the user for confirmation.',
         parameters: z.object({
           message: z.string().describe('The message to ask for confirmation.'),
         }),
       },
+      // client-side tool that is automatically executed on the client:
+      getLocation: {
+        description:
+          'Get the user location. Always ask for confirmation before using this tool.',
+        parameters: z.object({}),
+      },
     },
   });
 
@@ -85,24 +105,43 @@ export async function POST(req: Request) {
 The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming.
 Tool invocations are displayed in the chat UI.
 
-If the tool is a confirmation dialog, the user can confirm or deny the execution of the server-side tool.
-Once the user confirms the execution, the tool result is appended to the assistant message using `experimental_addToolResult`,
-and the server route is called again to execute the server-side tool.
+There are three things worth mentioning:
+
+1. The `onToolCall` callback is used to handle client-side tools that should be automatically executed.
+   In this example, the `getLocation` tool is a client-side tool that returns a random city.
+
+1. The `toolInvocations` property of the last assistant message contains all tool calls and results.
+   The client-side tool `askForConfirmation` is displayed in the UI.
+   It asks the user for confirmation and displays the result once the user confirms or denies the execution.
+   The result is added to the chat using `experimental_addToolResult`.
+
+1. The `experimental_maxAutomaticRoundtrips` option is set to 5.
+   This enables several tool use iterations between the client and the server.
 
 ```tsx filename='app/page.tsx'
-'use client';
+'use client'
 
-import { ToolInvocation } from 'ai';
-import { Message, useChat } from 'ai/react';
+import { ToolInvocation } from 'ai'
+import { Message, useChat } from 'ai/react'
 
 export default function Chat() {
   const {
     messages,
     input,
     handleInputChange,
     handleSubmit,
-    experimental_addToolResult,
-  } = useChat();
+    experimental_addToolResult
+  } = useChat({
+    experimental_maxAutomaticRoundtrips: 5,
+
+    // run client-side tools that are automatically executed:
+    async onToolCall({ toolCall }) {
+      if (toolCall.toolName === 'getLocation') {
+        const cities = ['New York', 'Los Angeles', 'Chicago', 'San Francisco']
+        return cities[Math.floor(Math.random() * cities.length)]
+      }
+    }
+  })
 
   return (
     <div>
@@ -111,9 +150,9 @@ export default function Chat() {
           <strong>{`${m.role}: `}</strong>
           {m.content}
           {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
-            const toolCallId = toolInvocation.toolCallId;
+            const toolCallId = toolInvocation.toolCallId
 
-            // render confirmation tool
+            // render confirmation tool (client-side tool with user interaction)
             if (toolInvocation.toolName === 'askForConfirmation') {
               return (
                 <div key={toolCallId}>
@@ -127,7 +166,7 @@ export default function Chat() {
                           onClick={() =>
                             experimental_addToolResult({
                               toolCallId,
-                              result: 'Yes, confirmed.',
+                              result: 'Yes, confirmed.'
                             })
                           }
                         >
@@ -137,7 +176,7 @@ export default function Chat() {
                           onClick={() =>
                             experimental_addToolResult({
                               toolCallId,
-                              result: 'No, denied',
+                              result: 'No, denied'
                             })
                           }
                         >
@@ -147,123 +186,34 @@ export default function Chat() {
                     )}
                   </div>
                 </div>
-              );
+              )
             }
 
             // other tools:
             return 'result' in toolInvocation ? (
-              <div key={toolCallId}>
-                <strong>{`${toolInvocation.toolName}:`}</strong>
+              <div key={toolCallId}
+                Tool call {`${toolInvocation.toolName}: `}
                 {toolInvocation.result}
               </div>
             ) : (
-              <div key={toolCallId}>Calling {toolInvocation.toolName}...</div>
-            );
+              <div key={toolCallId}
+                Calling {toolInvocation.toolName}...
+              </div>
+            )
           })}
           <br />
           <br />
         </div>
       ))}
 
       <form onSubmit={handleSubmit}>
-        <input value={input} onChange={handleInputChange} />
-      </form>
-    </div>
-  );
-}
-```
-
-## Example: Server-Side Tool Execution with Roundtrips
-
-In this example, we'll define a weather tool that shows the weather in a given city.
-
-When asked about the weather, the assistant will call the weather tool to get the weather information.
-The server will then respond with the weather information tool results.
-
-Once the client receives all tool results, it will send the updated messages back to the server for another roundtrip.
-The server will then generate a streaming text response that uses the information from the weather tool results.
-
-### Server-side route
-
-The server-side route defines a weather tool that returns the weather in a given city.
-
-```tsx filename='app/api/chat/route.ts'
-import { openai } from '@ai-sdk/openai';
-import { convertToCoreMessages, streamText } from 'ai';
-import { z } from 'zod';
-
-export const dynamic = 'force-dynamic';
-export const maxDuration = 60;
-
-export async function POST(req: Request) {
-  const { messages } = await req.json();
-
-  const result = await streamText({
-    model: openai('gpt-4-turbo'),
-    system:
-      'You are a weather bot that can use the weather tool ' +
-      'to get the weather in a given city. ' +
-      'Respond to the user with weather information in a friendly ' +
-      'and helpful manner.',
-    messages: convertToCoreMessages(messages),
-    tools: {
-      weather: {
-        description: 'show the weather in a given city to the user',
-        parameters: z.object({ city: z.string() }),
-        execute: async ({}: { city: string }) => {
-          // Random weather:
-          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
-          return weatherOptions[
-            Math.floor(Math.random() * weatherOptions.length)
-          ];
-        },
-      },
-    },
-  });
-
-  return result.toAIStreamResponse();
-}
-```
-
-### Client-side page
-
-The page uses the `useChat` hook to create a chatbot application with real-time message streaming.
-We set `experimental_maxAutomaticRoundtrips` to 2 to automatically
-send another request to the server when all server-side tool results are received.
-
-<Note>
-  The `experimental_maxAutomaticRoundtrips` option is disabled by default for
-  backward compatibility. It also limits the number of automatic roundtrips to
-  prevent infinite client-server call loops.
-</Note>
-
-```tsx filename='app/page.tsx'
-'use client';
-
-import { useChat } from 'ai/react';
-
-export default function Chat() {
-  const { messages, input, handleInputChange, handleSubmit } = useChat({
-    experimental_maxAutomaticRoundtrips: 2,
-  });
-
-  return (
-    <div>
-      {messages
-        .filter(m => m.content) // filter out empty messages
-        .map(m => (
-          <div key={m.id}>
-            <strong>{`${m.role}: `}</strong>
-            {m.content}
-            <br />
-            <br />
-          </div>
-        ))}
-
-      <form onSubmit={handleSubmit}>
-        <input value={input} onChange={handleInputChange} />
+        <input
+          value={input}
+          placeholder="Say something..."
+          onChange={handleInputChange}
+        />
       </form>
     </div>
-  );
+  )
 }
 ```