vidtheque.

read-only demo

Videos uiP88SpCi1Q

Your Agent Is Wasting Tokens and You Don't Know It - Erik Hanchett, AWS

index_state ready data_status ok

AI Engineer· published 2026-06-28· 0:05:55· en· indexed 2026-08-11 06:00

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:09, 1 of 1 keyframes kept
  2. Shot 1, 0:09 to 0:18, 1 of 1 keyframes kept
  3. Shot 2, 0:18 to 0:24, 1 of 1 keyframes kept
  4. Shot 3, 0:24 to 1:00, 1 of 1 keyframes kept
  5. Shot 4, 1:00 to 1:09, 1 of 1 keyframes kept
  6. Shot 5, 1:09 to 1:51, 1 of 1 keyframes kept
  7. Shot 6, 1:51 to 1:57, 1 of 1 keyframes kept
  8. Shot 7, 1:57 to 2:47, 1 of 1 keyframes kept
  9. Shot 8, 2:47 to 2:50, 1 of 1 keyframes kept
  10. Shot 9, 2:50 to 3:36, 1 of 1 keyframes kept
  11. Shot 10, 3:36 to 3:42, 1 of 1 keyframes kept
  12. Shot 11, 3:42 to 4:16, 1 of 1 keyframes kept
  13. Shot 12, 4:16 to 4:50, 1 of 1 keyframes kept
  14. Shot 13, 4:50 to 5:37, 1 of 1 keyframes kept
  15. Shot 14, 5:37 to 5:54, 1 of 1 keyframes kept
  16. Shot 15, 5:54 to 5:55, 0 of 1 keyframes kept

16 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
63
whisperx 63
chunks
10
from 63 cues
keyframes
15
kept of 16 captured
frames with text
15
95 lines read
chapters
0
from the source metadata
keyframe bytes
1.5 MB
word timings on 63 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-11 05:59 1m 21s
stt done — 2026-08-11 06:00 6s
chunk done — 2026-08-11 06:00 0s
text_embed done — 2026-08-11 06:00 1s
keyframe done — 2026-08-11 06:00 19s
ocr done — 2026-08-11 06:00 3s
frame_embed done — 2026-08-11 06:00 3s

Frames, and what the machine read

  • 0:03 #0 done6 line(s)

    shot 0·sharpness 1725.3

    1. Your Agent Is1.00
    2. Wasting1.00
    3. Tokens1.00
    4. Cutting Al Agent Token Costs0.98
    5. Erik Hanchett1.00
    6. Developer Advocate at AWS1.00

  • 0:12 #1 done3 line(s)

    shot 1·sharpness 1175.0

    1. Costing Too0.99
    2. 3X0.86
    3. Much1.00

  • 0:23 #2 done2 line(s)

    shot 2·sharpness 774.7

    1. Cache System1.00
    2. Prompt1.00

  • 0:35 #3 done10 line(s)

    shot 3·sharpness 940.7

    1. FIX 010.93
    2. Cache the System Prompt1.00
    3. agent = Agent(0.99
    4. model=BedrockModel(1.00
    5. model_id="claude-sonnet",1.00
    6. cache_prompt="default",1.00
    7. cache static prefix1.00
    8. ),0.75
    9. system_prompt=BIG_SYSTEM_PROMPT,1.00
    10. U50.73

  • 1:05 #4 done2 line(s)

    shot 4·sharpness 946.5

    1. Route By1.00
    2. Difficulty0.99

  • 1:34 #5 done9 line(s)

    shot 5·sharpness 706.7

    1. FIX 021.00
    2. Route by Difficulty1.00
    3. def pick_model(task):1.00
    4. if task.is_simple:1.00
    5. return "claude-haiku"1.00
    6. #1.00
    7. cheap1.00
    8. return "claude-sonnet"1.00
    9. expensive reasoning1.00

  • 1:53 #6 done2 line(s)

    shot 6·sharpness 1491.9

    1. Offload Tool1.00
    2. Results1.00

  • 2:27 #7 done12 line(s)

    shot 7·sharpness 1501.6

    1. FIX 031.00
    2. Offload Big Tool Results0.98
    3. @tool1.00
    4. def fetch_report(id: str) → str:0.99
    5. data = api.get(id)1.00
    6. # 10k tokens0.98
    7. key = store.put(data)1.00
    8. # offload0.97
    9. return summarize(data, key)0.99
    10. # summary + ref1.00
    11. *only the summarized tool result gets added1.00
    12. not the full tool result1.00

  • 2:48 #8 done2 line(s)

    shot 8·sharpness 1012.5

    1. Cap Your Tool0.98
    2. Loops1.00

  • 3:26 #9 done6 line(s)

    shot 9·sharpness 483.5

    1. FIX 040.98
    2. Cap Your Tool Loops1.00
    3. agent = Agent(1.00
    4. tools=[...],0.99
    5. max_iterations=8,1.00
    6. # stop runaway loops0.99

  • 3:39 #10 done1 line(s)

    shot 10·sharpness 711.4

    1. Trim the History1.00

  • 3:46 #11 done8 line(s)

    shot 11·sharpness 1448.1

    1. FIX 050.99
    2. Trim the History0.99
    3. from strands.agent.conversation_manager import (0.99
    4. SlidingWindowConversationManager,0.99
    5. agent = Agent(0.98
    6. conversation_manager=SlidingWindowConversationManager(1.00
    7. window_size=10,1.00
    8. *multi-turn conversation agent1.00

  • 4:26 #12 done7 line(s)

    shot 12·sharpness 1176.7

    1. FIX 050.99
    2. Trim the History0.99
    3. from strands.agent.conversation_manager import (0.99
    4. SlidingWindowConversationManager,1.00
    5. agent = Agent(0.96
    6. conversation_manager=SlidingWindowConversationManager(1.00
    7. window_size=10,1.00

  • 5:00 #13 done18 line(s)

    shot 13·sharpness 1489.7

    1. RECAP1.00
    2. // FIVE FIXES1.00
    3. Same Agent. Smaller Bill.1.00
    4. 011.00
    5. Cache the System Prompt1.00
    6. stop resending the static prompt every turn1.00
    7. 021.00
    8. Route by Difficulty1.00
    9. cheap model for simple calls1.00
    10. 031.00
    11. Offload Big Tool Results1.00
    12. keep 10k-token blobs out of context1.00
    13. 041.00
    14. Cap Your Tool Loops0.99
    15. max_iterations, no runaways1.00
    16. 051.00
    17. Trim the History1.00
    18. window or summarize old turns1.00

  • 5:50 #14 done7 line(s)

    shot 14·sharpness 1153.4

    1. Thanks!1.00
    2. Go Deeper0.99
    3. Erik Hanchett0.97
    4. Social Media1.00
    5. ErikCH1.00
    6. Site1.00
    7. programwitherik.com1.00

  • 5:54 #15 skipped

    shot 15·duplicate of #14

Transcript

63 cues· 991 words· 5,160 chars

  1. 0:00 Hey everyone.
  2. 0:01 My name is Eric Hanchett.
  3. 0:02 I am a senior developer advocate at AWS, and I'm going to talk to you about how you can save on token costs.
  4. 0:09 Now I'm going to show you five ways that you can reduce your token costs while using and creating agents.
  5. 0:18 So the first way you can do that is to cache your system prompt.
  6. 0:24 Let me show you some code.
  7. 0:25 Now I'm using AWS's strands agents.
  8. 0:29 This works with all different providers.
  9. 0:33 This is a little bit of pseudo code, but the idea is that you can add cache prompt equals default.
  10. 0:39 And what that'll do is on the first call of your agent,
  11. 0:43 it will send the full system prompt over.
  12. 0:47 And then on every subsequent call, it will have a much reduced system prompt being sent over.
  13. 0:54 So it'll be cached.
  14. 0:55 You can also cache the tool prompts and messages as well.
  15. 1:01 This may sound obvious, but you want to look into routing your different messages based on the difficulty.
  16. 1:09 So here's a code example.
  17. 1:11 Let's imagine that you have a task that's very difficult.
  18. 1:15 You may want to use one of the newer Frontier models.
  19. 1:18 However, if it's something simpler, you want to use a cheaper model.
  20. 1:23 In this case, maybe we use Cloud Haiku for cheap.
  21. 1:26 something cheap and then use cloud sonnet for something a little bit more difficult and then you can use an if statement you can even have another model that's very cheap decide which model to use so you can play around with this but i highly recommend don't use the most expensive model for everything you're doing you want to use multiple different models based on the use case and then try to route to it inside your agent
  22. 1:51 Another good tip is to offload the tool result.
  23. 1:55 Let me show you some code on here.
  24. 1:56 Once again, I'm using strands agents.
  25. 1:58 This is a manual way to do it.
  26. 2:00 There is some additional APIs that the strands agents offers to do this.
  27. 2:06 If you have a large tool result that's coming back, you can store it locally or in the cloud and then use some kind of summarization that saves on tokens.
  28. 2:18 So that way, when it's being called over and over again, the tool result isn't added into the context every time the tool loops or every time the agent loops.
  29. 2:30 so if you can find any way that where you have this tool result that you don't necessarily send it on every single call back to the large language model that will save a lot of tokens for you and like i said there's a few apis to do this but essentially you can do this summarization technique
  30. 2:47 You can also cap your tool loops.
  31. 2:50 So when you're dealing with the agent loop and it decides to do a tool call, I've had this happen often where it calls the tool over and over and over again.
  32. 3:00 And if you don't cap that tool call, then it might run 10, 20 times.
  33. 3:06 It might get into an infinite loop, which would be very bad for your token usage.
  34. 3:10 So always set a max iterations of how many times it will loop.
  35. 3:16 A good thing you can do before you deploy your agent is to run some observability tools and take a look at the tool call use for every single tool and then see how long each one of them is running and how many times they're looping.
  36. 3:31 And that way you can get an idea of how efficient the tool call is.
  37. 3:37 Last but not least, we can trim the history.
  38. 3:42 So if we're using a multi-turn agent and we are talking back and forth, you will find at times that the conversation history will get very large.
  39. 3:52 On every single call, that whole conversation history will be sent back to the large language model.
  40. 3:59 And this can eat through hundreds, if not thousands, of tokens.
  41. 4:03 In strands agents, we have something called sliding window conversation manager, which this does is it looks back at the last 10 messages and only sends those back.
  42. 4:14 And you can set this to whatever you want.
  43. 4:16 And that way you're not sending these huge message histories back to the agent every single time a new message comes in.
  44. 4:25 The downfall of this, or the trade-off of this, I should say, is that you will lose the message history from the beginning.
  45. 4:32 The way you want to deal with that is you can use some sort of summarization of the history and then put that into the context window.
  46. 4:40 So rather than sending all of it, you may send a small amount once you hit this sliding window.
  47. 4:47 This will save you a lot of tokens.
  48. 4:50 So in conclusion, we have five things.
  49. 4:53 Cache the system prompt.
  50. 4:56 And if you can, maybe the tool prompt to messages.

Open at this second