Skip to content

Commit e220b22

Browse files
jph00jph00
authored
Merge pull request #65 from AnswerDotAI/extended-thinking
add extended thinking
2 parents c55b6b1 + 9384545 commit e220b22

File tree

7 files changed

+1001
-368
lines changed

7 files changed

+1001
-368
lines changed

‎00_core.ipynb

Lines changed: 295 additions & 17 deletions
Large diffs are not rendered by default.

‎02_async.ipynb

Lines changed: 244 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@
146146
{
147147
"cell_type": "code",
148148
"execution_count": null,
149-
"id": "6c464f8b",
149+
"id": "6ca62759",
150150
"metadata": {},
151151
"outputs": [
152152
{
@@ -250,7 +250,6 @@
250250
"metadata": {},
251251
"outputs": [],
252252
"source": [
253-
"#| exports\n",
254253
"@patch\n",
255254
"@delegates(Client)\n",
256255
"async def __call__(self:AsyncClient,\n",
@@ -781,7 +780,6 @@
781780
"metadata": {},
782781
"outputs": [],
783782
"source": [
784-
"#| exports\n",
785783
"@patch\n",
786784
"async def __call__(self:AsyncChat,\n",
787785
" pr=None, # Prompt / message\n",
@@ -1098,6 +1096,247 @@
10981096
"chat.use"
10991097
]
11001098
},
1099+
{
1100+
"cell_type": "markdown",
1101+
"id": "3d162fca",
1102+
"metadata": {},
1103+
"source": [
1104+
"## Extended Thinking"
1105+
]
1106+
},
1107+
{
1108+
"cell_type": "markdown",
1109+
"id": "11327638",
1110+
"metadata": {},
1111+
"source": [
1112+
"Claude 3.7 Sonnet has enhanced reasoning capabilities for complex tasks. See [docs](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking) for more info.\n",
1113+
"\n",
1114+
"We can enable extended thinking by passing a `thinking` param with the following structure.\n",
1115+
"\n",
1116+
"```js\n",
1117+
"thinking={\n",
1118+
" \"type\": \"enabled\",\n",
1119+
" \"budget_tokens\": 16000\n",
1120+
"}\n",
1121+
"```\n",
1122+
"\n",
1123+
"When extended thinking is enabled a thinking block is included in the response as shown below.\n",
1124+
"\n",
1125+
"```js\n",
1126+
"{\n",
1127+
" \"content\": [\n",
1128+
" {\n",
1129+
" \"type\": \"thinking\",\n",
1130+
" \"thinking\": \"To approach this, let's think about...\",\n",
1131+
" \"signature\": \"Imtakcjsu38219c0.eyJoYXNoIjoiYWJjM0NTY3fQ....\"\n",
1132+
" },\n",
1133+
" {\n",
1134+
" \"type\": \"text\",\n",
1135+
" \"text\": \"Yes, there are infinitely many prime numbers such that...\"\n",
1136+
" }\n",
1137+
" ]\n",
1138+
"}\n",
1139+
"```"
1140+
]
1141+
},
1142+
{
1143+
"cell_type": "markdown",
1144+
"id": "503cae9b",
1145+
"metadata": {},
1146+
"source": [
1147+
"Let's add a `maxthinktok` param to the `AsyncClient` and `AsyncChat` call methods. When this value is not 0, we'll pass a thinking param to Claude `{\"type\":\"enabled\", \"budget_tokens\":maxthinktok}`.\n",
1148+
"\n",
1149+
"*Note: When thinking is [enabled](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking) `prefill` must be empty and the `temp` must be 1.*"
1150+
]
1151+
},
1152+
{
1153+
"cell_type": "code",
1154+
"execution_count": null,
1155+
"id": "89fade2f",
1156+
"metadata": {},
1157+
"outputs": [],
1158+
"source": [
1159+
"#| exports\n",
1160+
"@patch\n",
1161+
"@delegates(Client)\n",
1162+
"async def __call__(self:AsyncClient,\n",
1163+
" msgs:list, # List of messages in the dialog\n",
1164+
" sp='', # The system prompt\n",
1165+
" temp=0, # Temperature\n",
1166+
" maxtok=4096, # Maximum tokens\n",
1167+
" maxthinktok=0, # Maximum thinking tokens\n",
1168+
" prefill='', # Optional prefill to pass to Claude as start of its response\n",
1169+
" stream:bool=False, # Stream response?\n",
1170+
" stop=None, # Stop sequence\n",
1171+
" tools:Optional[list]=None, # List of tools to make available to Claude\n",
1172+
" tool_choice:Optional[dict]=None, # Optionally force use of some tool\n",
1173+
" **kwargs):\n",
1174+
" \"Make an async call to Claude.\"\n",
1175+
" if tools: kwargs['tools'] = [get_schema(o) for o in listify(tools)]\n",
1176+
" if tool_choice: kwargs['tool_choice'] = mk_tool_choice(tool_choice)\n",
1177+
" if maxthinktok: \n",
1178+
" kwargs['thinking']={'type':'enabled', 'budget_tokens':maxthinktok} \n",
1179+
" temp=1; prefill=''\n",
1180+
" msgs = self._precall(msgs, prefill, stop, kwargs)\n",
1181+
" if any(t == 'image' for t in get_types(msgs)): assert not self.text_only, f\"Images are not supported by the current model type: {self.model}\"\n",
1182+
" if stream: return self._stream(msgs, prefill=prefill, max_tokens=maxtok, system=sp, temperature=temp, **kwargs)\n",
1183+
" res = await self.c.messages.create(\n",
1184+
" model=self.model, messages=msgs, max_tokens=maxtok, system=sp, temperature=temp, **kwargs)\n",
1185+
" return self._log(res, prefill, msgs, maxtok, sp, temp, stream=stream, stop=stop, **kwargs)"
1186+
]
1187+
},
1188+
{
1189+
"cell_type": "code",
1190+
"execution_count": null,
1191+
"id": "9875abd9",
1192+
"metadata": {},
1193+
"outputs": [],
1194+
"source": [
1195+
"#| exports\n",
1196+
"@patch\n",
1197+
"async def __call__(self:AsyncChat,\n",
1198+
" pr=None, # Prompt / message\n",
1199+
" temp=None, # Temperature\n",
1200+
" maxtok=4096, # Maximum tokens\n",
1201+
" maxthinktok=0, # Maximum thinking tokens\n",
1202+
" stream=False, # Stream response?\n",
1203+
" prefill='', # Optional prefill to pass to Claude as start of its response\n",
1204+
" tool_choice:Optional[Union[str,bool,dict]]=None, # Optionally force use of some tool\n",
1205+
" **kw):\n",
1206+
" if temp is None: temp=self.temp\n",
1207+
" await self._append_pr(pr)\n",
1208+
" res = await self.c(self.h, stream=stream, prefill=prefill, sp=self.sp, temp=temp, maxtok=maxtok, maxthinktok=maxthinktok, tools=self.tools, tool_choice=tool_choice,**kw)\n",
1209+
" if stream: return self._stream(res)\n",
1210+
" self.h += mk_toolres(self.c.result, ns=mk_ns(*listify(self.tools)))\n",
1211+
" return res"
1212+
]
1213+
},
1214+
{
1215+
"cell_type": "markdown",
1216+
"id": "4e66607b",
1217+
"metadata": {},
1218+
"source": [
1219+
"Let's call the model without extended thinking enabled. "
1220+
]
1221+
},
1222+
{
1223+
"cell_type": "code",
1224+
"execution_count": null,
1225+
"id": "26b5f9c7",
1226+
"metadata": {},
1227+
"outputs": [],
1228+
"source": [
1229+
"tk_model = first(has_extended_thinking_models)"
1230+
]
1231+
},
1232+
{
1233+
"cell_type": "code",
1234+
"execution_count": null,
1235+
"id": "adc3e86f",
1236+
"metadata": {},
1237+
"outputs": [],
1238+
"source": [
1239+
"chat = AsyncChat(tk_model)"
1240+
]
1241+
},
1242+
{
1243+
"cell_type": "code",
1244+
"execution_count": null,
1245+
"id": "b7911771",
1246+
"metadata": {},
1247+
"outputs": [
1248+
{
1249+
"data": {
1250+
"text/markdown": [
1251+
"Python is a versatile programming language known for its readable syntax and wide application in fields ranging from web development to data science and artificial intelligence.\n",
1252+
"\n",
1253+
"<details>\n",
1254+
"\n",
1255+
"- id: `msg_01WDDwTK4S6kdm5LKbT9GVDG`\n",
1256+
"- content: `[{'citations': None, 'text': 'Python is a versatile programming language known for its readable syntax and wide application in fields ranging from web development to data science and artificial intelligence.', 'type': 'text'}]`\n",
1257+
"- model: `claude-3-7-sonnet-20250219`\n",
1258+
"- role: `assistant`\n",
1259+
"- stop_reason: `end_turn`\n",
1260+
"- stop_sequence: `None`\n",
1261+
"- type: `message`\n",
1262+
"- usage: `{'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 13, 'output_tokens': 31}`\n",
1263+
"\n",
1264+
"</details>"
1265+
],
1266+
"text/plain": [
1267+
"Message(id='msg_01WDDwTK4S6kdm5LKbT9GVDG', content=[TextBlock(citations=None, text='Python is a versatile programming language known for its readable syntax and wide application in fields ranging from web development to data science and artificial intelligence.', type='text')], model='claude-3-7-sonnet-20250219', role='assistant', stop_reason='end_turn', stop_sequence=None, type='message', usage=In: 13; Out: 31; Cache create: 0; Cache read: 0; Total: 44)"
1268+
]
1269+
},
1270+
"execution_count": null,
1271+
"metadata": {},
1272+
"output_type": "execute_result"
1273+
}
1274+
],
1275+
"source": [
1276+
"await chat(\"Write a sentence about Python!\")"
1277+
]
1278+
},
1279+
{
1280+
"cell_type": "markdown",
1281+
"id": "e9aab9e0",
1282+
"metadata": {},
1283+
"source": [
1284+
"Now, let's call the model with extended thinking enabled."
1285+
]
1286+
},
1287+
{
1288+
"cell_type": "code",
1289+
"execution_count": null,
1290+
"id": "0aec5ad6",
1291+
"metadata": {},
1292+
"outputs": [
1293+
{
1294+
"data": {
1295+
"text/markdown": [
1296+
"\n",
1297+
"With its extensive library ecosystem and intuitive design philosophy of \"there should be one—and preferably only one—obvious way to do it,\" Python empowers developers to write clear, concise code for projects of any scale.\n",
1298+
"\n",
1299+
"<details>\n",
1300+
"<summary>Thinking</summary>\n",
1301+
"I've been asked to write another sentence about Python. Since I already wrote one in my previous response, I should write a different sentence that highlights other aspects of Python to avoid repetition.\n",
1302+
"\n",
1303+
"Some other aspects of Python I could mention:\n",
1304+
"- Its large ecosystem of libraries and frameworks\n",
1305+
"- Its beginner-friendly nature\n",
1306+
"- Its interpreted nature\n",
1307+
"- Its object-oriented features\n",
1308+
"- Its community support\n",
1309+
"- Its cross-platform compatibility\n",
1310+
"- Its use in specific domains like machine learning or automation\n",
1311+
"</details>\n",
1312+
"\n",
1313+
"\n",
1314+
"<details>\n",
1315+
"\n",
1316+
"- id: `msg_019zC9dyvrTAh3U4rvdLfV2K`\n",
1317+
"- content: `[{'signature': 'ErUBCkYIARgCIkC5glBpf2cj1azao0wX9aW2WRlDwTIkccMttnanL/KrYv6BUKoOgDGjGqo/cmUuEID9EkMKXHphKczvfU40SLTFEgz/LvmZfnfyqSuVX3waDBNofBaC+QczWomd7CIw+dglNoRIEGUT+WMx/ZSnLMQUonX2ggTESUd6qkyvchdyAw/yxqP3+tBlH+VFe7okKh1VxqpF7ecOJzFHU2RSevAESrdADPrAQKGs2GXW/g==', 'thinking': \"I've been asked to write another sentence about Python. Since I already wrote one in my previous response, I should write a different sentence that highlights other aspects of Python to avoid repetition.\\n\\nSome other aspects of Python I could mention:\\n- Its large ecosystem of libraries and frameworks\\n- Its beginner-friendly nature\\n- Its interpreted nature\\n- Its object-oriented features\\n- Its community support\\n- Its cross-platform compatibility\\n- Its use in specific domains like machine learning or automation\", 'type': 'thinking'}, {'citations': None, 'text': 'With its extensive library ecosystem and intuitive design philosophy of \"there should be one—and preferably only one—obvious way to do it,\" Python empowers developers to write clear, concise code for projects of any scale.', 'type': 'text'}]`\n",
1318+
"- model: `claude-3-7-sonnet-20250219`\n",
1319+
"- role: `assistant`\n",
1320+
"- stop_reason: `end_turn`\n",
1321+
"- stop_sequence: `None`\n",
1322+
"- type: `message`\n",
1323+
"- usage: `{'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0, 'input_tokens': 81, 'output_tokens': 159}`\n",
1324+
"\n",
1325+
"</details>"
1326+
],
1327+
"text/plain": [
1328+
"Message(id='msg_019zC9dyvrTAh3U4rvdLfV2K', content=[ThinkingBlock(signature='ErUBCkYIARgCIkC5glBpf2cj1azao0wX9aW2WRlDwTIkccMttnanL/KrYv6BUKoOgDGjGqo/cmUuEID9EkMKXHphKczvfU40SLTFEgz/LvmZfnfyqSuVX3waDBNofBaC+QczWomd7CIw+dglNoRIEGUT+WMx/ZSnLMQUonX2ggTESUd6qkyvchdyAw/yxqP3+tBlH+VFe7okKh1VxqpF7ecOJzFHU2RSevAESrdADPrAQKGs2GXW/g==', thinking=\"I've been asked to write another sentence about Python. Since I already wrote one in my previous response, I should write a different sentence that highlights other aspects of Python to avoid repetition.\\n\\nSome other aspects of Python I could mention:\\n- Its large ecosystem of libraries and frameworks\\n- Its beginner-friendly nature\\n- Its interpreted nature\\n- Its object-oriented features\\n- Its community support\\n- Its cross-platform compatibility\\n- Its use in specific domains like machine learning or automation\", type='thinking'), TextBlock(citations=None, text='With its extensive library ecosystem and intuitive design philosophy of \"there should be one—and preferably only one—obvious way to do it,\" Python empowers developers to write clear, concise code for projects of any scale.', type='text')], model='claude-3-7-sonnet-20250219', role='assistant', stop_reason='end_turn', stop_sequence=None, type='message', usage=In: 81; Out: 159; Cache create: 0; Cache read: 0; Total: 240)"
1329+
]
1330+
},
1331+
"execution_count": null,
1332+
"metadata": {},
1333+
"output_type": "execute_result"
1334+
}
1335+
],
1336+
"source": [
1337+
"await chat(\"Write a sentence about Python!\", maxthinktok=1024)"
1338+
]
1339+
},
11011340
{
11021341
"cell_type": "markdown",
11031342
"id": "94ec4289",
@@ -1130,9 +1369,9 @@
11301369
],
11311370
"metadata": {
11321371
"kernelspec": {
1133-
"display_name": "python3",
1372+
"display_name": "python",
11341373
"language": "python",
1135-
"name": "python3"
1374+
"name": "python"
11361375
}
11371376
},
11381377
"nbformat": 4,

0 commit comments

Comments
 (0)