Videos vJukHCIv7Ck
Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS
Scene timeline
443 shot(s).
keyframes kept every frame deduplicated
What was stored
- cues
- 538
- whisperx 538
- chunks
- 101
- from 538 cues
- keyframes
- 384
- kept of 443 captured
- frames with text
- 369
- 16,992 lines read
- chapters
- 0
- from the source metadata
- keyframe bytes
- 70.4 MB
- word timings on 538 cues
Provenance
| stage | state | model | started | took |
|---|---|---|---|---|
fetch |
done | — | 2026-08-11 03:42 | 1m 45s |
stt |
done | — | 2026-08-11 03:44 | 51s |
chunk |
done | — | 2026-08-11 03:44 | 0s |
text_embed |
done | — | 2026-08-11 03:44 | 1s |
keyframe |
done | — | 2026-08-11 03:44 | 5m 00s |
ocr |
done | — | 2026-08-11 03:49 | 8m 48s |
frame_embed |
done | — | 2026-08-11 03:58 | 1m 13s |
Frames, and what the machine read
-
- Stop Al Agent Hallucinations1.00
- 5 Techniques + Production Patterns1.00
- Five techniques beyond the prompt — each one a code change, not a prompt1.00
- change1.00
- Demos built with Strands Agents · Deployable on Amazon Bedrock AgentCore1.00
- aws0.87
- © 2026, Amazon Web Services, Inc. or its affiliates. All rights reserved.0.99
-
- Five Techniques Beyond the Prompt0.99
- Each fixes a different failure. None of them is a prompt problem.0.99
- 21.00
- 31.00
- 41.00
- 51.00
- Semantic1.00
- Multi-Agent1.00
- Neurosymbolic1.00
- Graph-RAG1.00
- Runtime Steering1.00
- Tool Selection1.00
- Validation1.00
- Guardrails1.00
- Filter tools into1.00
- Compute precise1.00
- A second agent checks1.00
- Rules in code, not in1.00
- Self-correct instead of0.98
- context on every call1.00
- answers, don't guess1.00
- every response1.00
- the prompt1.00
- blocking1.00
- ...then ship all five to production with Amazon Bedrock AgentCore0.99
- aws0.99
- © 2026, Amazon Web Services, Inc. or its affiliates. All rights reserved.0.99
-
- aws0.99
- Elizabeth Fuentes Leone1.00
- Developer Advocate @ AWS0.99
- elifuentes.tech1.00
- Resources1.00
- bit.ly/4oN2LVu1.00
- aws1.00
- ©2026, Amazon Web Services, Inc. or its affiliates. All rights reserved.0.99
-
- 1 Semantic Tool Selection0.99
- First: what a tool actually looks like to the model1.00
- You write a @tool1.00
- Strands generates a schema1.00
- Into context on every call0.99
- @tool1.00
- def search_hotels(query: str) ->0.98
- "name": ...0.91
- ~70–100 tokens per schema0.99
- str:1.00
- "description": ...0.95
- """Search hotels by1.00
- city or location."""0.98
- "parameters": ..0.93
- × dozens of tools1.00
- = thousands of tokens1.00
- every call, before your message1.00
- name·docstring·typed params1.00
- the model reads this, not your code1.00
- If your agent has memory, that grows too — every turn adds context that ships with every message.0.99
- aws1.00
- © 2026, Amazon Web Services, Inc. or its affiliates. All rights reserved.0.99
-
- EXPLORER1.00
- 0000.76
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- test_semantic_tools_hallucinations.ipynb1.00
- 0000.77
- WHY-AGENTS-FAIL-SAMPLE-...0.99
- -tools-demo>token_efficiency_analysis.ipynb >M↓ Semantic Tool Selection: Token Efficiency & Accuracy Analysis > M↓ Build Semantic Index>build_index(ALL_TOOLS)0.98
- Q0.99
- > .claude0.89
- Generate + Code + MarkdownRun All Restart Clear All OutputsJupyter Variables Outline0.97
- 0000.97
- Python 3.12.131.00
- > 01-faq-graphrag-demo0.97
- 02-semantic-tools-demo1.00
- < Back to Main README | Demo README0.97
- __pycache_0.90
- > images0.92
- enhanced_tools.py1.00
- Semantic Tool Selection: Token Efficiency & Accuracy Analysis1.00
- registry.py1.00
- ① README.md0.96
- 业0.92
- Based on: Internal Representations as Indicators of Hallucinations in Agent Tool Selection0.99
- requirements.txt0.98
- test_semantic_tools_hallucinations.ipynb1.00
- What This Demo Measures1.00
- token_comparison_app.py1.00
- 00.89
- token_efficiency_analysis.ipynb1.00
- > 03-multiagent-demo0.97
- This notebook compares Traditional (all 31 tools) vs Semantic (top-3 filtered tools) approaches across two critical metrics:1.00
- 30.55
- > 04-neurosymbolic-demo0.98
- 1. Token Consumption: How many tokens are used per query?0.99
- > 05-agent-control-demo0.97
- 2. Tool Selection Accuracy: Does the agent pick the correct tool?1.00
- aws1.00
- 07-context-graph-integration1.00
- 06-agentcore-production-demo0.99
- The Dual Problem0.99
- > artifacts0.97
- > blog-es0.98
- • Token Waste: Sending 31 tool descriptions = ~4,500 tokens per query0.98
- > images0.93
- Hallucination Risk: More tools = more confusion = wrong tool selection0.99
- langgraph-demos1.00
- .gitignore1.00
- The Solution1.00
- CLAUDE.md1.00
- CODE_OF_CONDUCT.md1.00
- User Query → FAISS Search → Top 3 Tools → Agent → Correct Selection + Fewer Tokens0.99
- LICENSE0.97
- ③ README.md0.93
- test_nova_embeddings.py1.00
- !pip install -q -r requirements.txt0.95
- [13]1.00
- 2.3s1.00
- 0000.95
- [notice] A new release of pip is available: 25.0.1 -> 26.1.20.98
- [notice] To update, run: pip install --upgrade pip0.99
- Configure API Key1.00
- OUTLINE0.96
- > APPLICATION BUILDER0.96
-
- EXPLORER1.00
- 0000.74
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb0.99
- test_semantic_tools_hallucinations.ipynb1.00
- WHY-AGENTS-FAIL-SAMPLE-...0.99
- -tools-demo>token_efficiency_analysis.ipynb> M↓ Semantic Tool Selection: Token Efficiency & Accuracy Analysis > M↓ What This Demo Measures > M↓ The Solution >0.97
- Q0.99
- > .claude0.90
- Generate + Code + MarkdownRun All Restart Clear All OutputsJupyter Variables Outline0.97
- 0001.00
- Python 3.12.130.99
- > 01-faq-graphrag-demo0.97
- 02-semantic-tools-demo1.00
- This notebook compares Traditional (all 31 tools) vs Semantic (top-3 filtered tools) approaches across two critical metrics:1.00
- >__pycache__0.85
- 1. Token Consumption: How many tokens are used per query?0.99
- 好0.65
- > images0.94
- 2. Tool Selection Accuracy: Does the agent pick the correct tool?1.00
- enhanced_tools.py1.00
- ①README.md0.98
- The Dual Problem1.00
- registry.py1.00
- test_semantic_tools_hsliucinations.ipynb0.98
- requirements.txt0.98
- • Token Waste: Sending 31 tool descriptions = ~4,500 tokens per query0.98
- • Hallucination Risk: More tools = more confusion = wrong tool selection0.99
- token_comparison_app.p ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcore/02-0.99
- 00.86
- token_efficiency_analysis semantic-tools-demo/test_semantic_tools_hallucinations.ipynb1.00
- > 03-multiagent-demo0.97
- >04-neurosymbolic-demo0.99
- User Query → FAISS Search → Top 3 Tools → Agent → Correct Selection + Fewer Tokens0.99
- > 05-agent-control-demo0.97
- θ0.62
- aws1.00
- >07-context-graph-integration0.99
- 06-agentcore-production-demo1.00
- !pipinstall -q -r requirements.txt0.96
- > artifacts0.96
- Python1.00
- > blog-es0.96
- > images0.94
- Configure API Key1.00
- langgraph-demos1.00
- .gitignore1.00
- CLAUDE.md1.00
- LICENSE0.99
- CODE_OF_CONDUCT.md1.00
- from dotenv import load_dotenv0.99
- import os1.00
- ①README.md0.94
- load_dotenv()1.00
- test_nova_embeddings.py1.00
- U1.00
- # Uncomment and set your key if not using a .env file:0.99
- # os.environ['OPENAI_API_KEY'] = 'your-key-here'0.97
- assert os.getenv('OPENAI_API_KEY'), 'Set OPENAI_API_KEY in .env file or uncomment line above'1.00
- []0.99
- Setup1.00
- > OUTLINE0.91
- APPLICATION BUILDER0.99
-
- EXPLORER1.00
- 0000.78
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- requirements.txt ×0.96
- test_semantic_tools_hallucinations.ipynb1.00
- ①0.99
- 0000.63
- WHY-AGENTS-FAIL-SAMPLE-FOR-AMAZON-AG...1.00
- 02-semantic-tools-demo requirements.txt0.99
- Q0.99
- .claude0.97
- > 01-faq-graphrag-demo0.96
- Search files by name (append : to go to line or @ to go to symbol)0.99
- 990.70
- 02-semantic-tools-demo1.00
- Go to File1.00
- 960.99
- __pycache__0.91
- Show and Run Commands >0.99
- > images0.93
- Search for Text %1.00
- enhanced_tools.py1.00
- ① README.md0.95
- Open Quick Chat0.99
- registry.py1.00
- Go to Symbol in Editor @1.00
- test_semantic_tools_hallucinations.ipynb1.00
- requirements.txt0.98
- Run Task task0.96
- Start Debugging debug1.00
- token_comparison_app.py1.00
- 00.57
- token_efficiency_analysis.ipynb1.00
- More ?0.92
- I0.87
- > 03-multiagent-demo0.96
- requirements.txt 02-semantic-tools-demo0.98
- recently opened1.00
- 30.53
- > 04-neurosymbolic-demo0.97
- token_efficiency_analysis.ipynb 02-semantic-tools-demo0.99
- > 05-agent-control-demo0.96
- aws1.00
- > 06-agentcore-production-demo0.98
- test_semantic_tools_hallucinations.ipynb 02-semantic-tools-demo1.00
- >07-context-graph-integration0.98
- test_graphrag.ipynb 01-faq-graphrag-demo0.99
- > artifacts0.95
- .env 01-faq-graphrag-demo1.00
- > blog-es0.98
- > images0.91
- langgraph-demos1.00
- .gitignore1.00
- CLAUDE.md1.00
- CODE_OF_CONDUCT.md1.00
- LICENSE0.98
- README.md1.00
- test_nova_embeddings.py1.00
- U0.99
- OUTLINE0.95
- APPLICATION BUILDER0.99
-
- EXPLORER1.00
- 0000.87
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- requirements.txt X0.96
- test_semantic_tools_hallucinations.ipynb1.00
- 0000.68
- WHY-AGENTS-FAIL-SAMPLE-FOR-AMAZON-AG...1.00
- 02-semantic-tools-demo> requirements.txt0.98
- .claude0.98
- strands-agents[openai]>=1.27.01.00
- > 01-faq-graphrag-demo0.97
- sentence-transformers>=3.4.01.00
- 990.72
- 02-semantic-tools-demo1.00
- faiss-cpu>=1.9.01.00
- neo4j>=5.28.01.00
- __pycache_0.90
- python-dotenv>=1.0.11.00
- > images0.94
- enhanced_tools.py1.00
- ① README.md0.94
- registry.py1.00
- test_semantic_tools_hallucinations.ipynb1.00
- requirements.txt0.98
- token_comparison_app.py1.00
- token_efficiency_analysis.ipynb1.00
- I0.92
- >03-multiagent-demo0.98
- > 04-neurosymbolic-demo0.98
- > 05-agent-control-demo0.97
- aws1.00
- > 06-agentcore-production-demo0.98
- > 07-context-graph-integration0.98
- artifacts0.98
- > blog-es0.92
- > images0.92
- langgraph-demos1.00
- .gitignore1.00
- CLAUDE.md1.00
- CODE_OF_CONDUCT.md1.00
- LICENSE0.97
- README.md1.00
- test_nova_embeddings.py1.00
- OUTLINE0.95
- APPLICATION BUILDER0.97
-
- EXPLORER1.00
- 0000.82
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- requirements.txt0.97
- test_semantic_tools_hallucinations.ipynb1.00
- 00。0.64
- WHY-AGENTS-FAIL-SAMPLE-FOR-AMAZON-AG...0.99
- 02-semantic-tools-demo >test_semantic_tools_hallucinations.ipynb > M↓ Semantic Tool Discovery: Reducing Tool Selection Hallucinations0.98
- .claude0.97
- Generate + Code + MarkdownRun All Restart Clear All Outputs0.97
- Go ToJupyter Variables Outline0.99
- 0000.97
- Python 3.10.201.00
- > 01-faq-graphrag-demo0.97
- 02-semantic-tools-demo1.00
- e0.56
- import io1.00
- import sys0.99
- __pycache__0.90
- import re0.97
- > images0.90
- e0.64
- enhanced_tools.py1.00
- from strands import Agent1.00
- registry.py1.00
- ①README.md0.99
- from registry import build_index, search_tools, get_scores1.00
- from enhanced_tools import ALL_ToOLS0.97
- test_semantic_tools_hallucinations.ipynb1.00
- requirements.txt0.99
- [3]1.00
- print(f" Loaded {len(ALL_ToOLS)} tools")0.97
- 1.9s1.00
- Python1.00
- token_comparison_app.py1.00
- 00.88
- token_efficiency_analysis.ipynb0.99
- 0000.97
- ModuleNotFoundError1.00
- Traceback (most recent call last)1.00
- > 03-multiagent-demo0.96
- Cell In[3], line 70.97
- > 04-neurosymbolic-demo0.97
- 5 from strands import Agent0.97
- > 05-agent-control-demo0.97
- 7 from registry import build_index, search_tools, get_scores0.99
- 6 from enhanced_tools import ALL_T0OLS0.97
- aws0.89
- > 06-agentcore-production-demo0.98
- 9 print(f" Loaded {len(ALL_T00LS)} tools")0.96
- > 07-context-graph-integration0.98
- > artifacts0.94
- File ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcore/02-semantic-tools-demo/registry-py:41.00
- > blog-es0.97
- 10.84
- > images0.93
- 2 Tool Registry - FAISS semantic search over tools0.99
- 3 "……0.63
- langgraph-demos0.97
- --> 4 import faiss0.97
- .gitignore1.00
- 5 from sentence_transformers import SentenceTransformer0.99
- CLAUDE.md1.00
- 6 from typing import List, Callable0.97
- CODE_OF_CONDUCT.md1.00
- LICENSE0.97
- ModuleNotFoundError: No module named 'faiss'1.00
- ③ README.md0.91
- test_nova_embeddings.py1.00
- U0.99
- Build Semantic Index1.00
- For each tool, we concatenatename: docstringand encode it withall-MiniLM-L6-v2 (384-dim vectors). FAISs0.99
- L2 nearest-neighbor search. The score shown is 1/(1+distance)- closer to 1.0 means better match.0.98
- build_index(ALL_T00LS)0.98
- OUTLINE1.00
- # Test semantic search0.98
- > APPLICATION BUILDER0.95
- []0.97
-
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- requirements.txt1.00
- test_semantic_tools_hallucinations.ipynb1.00
- ①0.98
- 00。0.64
- 02-semantic-tools-demo >test_semantic_tools_hallucinations.ipynb > M↓ Semantic Tool Discovery: Reducing Tool Selection Hallucinations0.98
- Q0.99
- Generate + Code + Markdown Run All Restart Clear All Outputs Go To Jupyter Variables ≡Outline0.96
- Python 3.10.201.00
- D0.98
- import sys1.00
- I0.71
- import io1.00
- import re1.00
- 好0.59
- from strands import Agent0.99
- from enhanced_tools import ALL_TOOLS0.97
- from registry import build_index, search_tools, get_scores0.99
- print(f" Loaded {len(ALL_ToOLS)} tools")0.97
- [3]1.00
- 1.9s1.00
- Python1.00
- 00.94
- 0000.82
- ModuleNotFoundError1.00
- Traceback (most recent call last)1.00
- Cell In[3], line 70.98
- 5 from strands import Agent1.00
- 6 from enhanced_tools import ALL_ToOLS0.97
- 7 from registry import build_index, search_tools, get_scores1.00
- aws1.00
- 9 print(f" Loaded {len(ALL_To0LS)} tools")0.97
- File ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcore/02-semantic-tools-demo/registry.py:40.99
- 1 "……0.70
- 2 Tool Registry - FAISS semantic search over tools0.99
- 3 "……0.66
- --> 4 import faiss0.94
- 5 from sentence_transformers import SentenceTransformer1.00
- 6 from typing import List, Callable0.98
- ModuleNotFoundError: No module named 'faiss'1.00
- Build Semantic Index1.00
- For each tool, we concatenate name: docstring and encode it with all-MiniLM-L6-v2 (384-dim vectors). FAISS stores these vectors and performs L2 nearest0.99
- is1/(1+distance) - closer to 1.0 means better match.0.98
- build_index(ALL_T00LS)0.97
- # Test semantic search0.98
- []0.98
-
- why-agents-fail-sample-for-amazon-agentcore1.00
- U80.83
- oken_efficiency_analysis.ipynb1.00
- Erequirements.txt0.97
- test_semantic_tools_hallucinations.ipynb1.00
- o0o0.59
- 02-semantic-tools-demo >0.98
- test_semantic_tools_hallucinations.ipynb > M↓ Semantic Tool Discovery: Reducing Tool Select0.98
- Generate1.00
- + Code0.94
- + Markdown0.96
- Run All0.95
- GRestart0.91
- Clear All Outputs1.00
- 0000.98
- Python 3.10.201.00
- import sys0.99
- import io1.00
- 961.00
- import re1.00
- 深0.51
- from strands import Agent1.00
- from enhanced_tools import ALL_To0LS0.97
- 田0.61
- from registry import build_index, search_tools, get_scores0.99
- print(f" Loaded {len(ALL_TOOLS)} tools")0.97
- [3]1.00
- 1.9s1.00
- Python1.00
- 0000.97
- 0oo0.56
- ModuleNotFoundError1.00
- Traceback (most recent call last)1.00
- Cell In[3], line 70.95
- 5 from strands import Agent0.99
- 6 from enhanced_tools import ALL_ToOLS0.97
- 7 from registry import build_index, search_tools, get_scores0.99
- 9 print(f" Loaded {len(ALL_TOOLS)} tools")0.97
- File ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcc1.00
-
- test_graphrag.ipynb1.00
- token_efficiency_analysis.ipynb1.00
- requirements.txt1.00
- test_semantic_tools_hallucinations.ipynb0.99
- 0000.95
- 2-semantic-tools-demo >test_semantic_tools_hallucinations.ipynb> M↓ Semantic Tool Discovery: Reducing Tool Selection Hallucinations0.99
- Generate + Code + Markdown0.97
- Run All Restart Clear All Outputs0.97
- Go ToJupyter Variables Outline0.99
- 0000.86
- Python 3.10.201.00
- import sys1.00
- 290.70
- import io1.00
- 961.00
- import re0.94
- from strands import Agent1.00
- from enhanced_tools import ALL_ToOLS0.98
- 田0.66
- from registry import build_index, search_tools, get_scores0.98
- print(f" Loaded {len(ALL_ToOLS)} tools")0.96
- [3]1.00
- 区0.63
- 1.9s0.91
- Python1.00
- 0001.00
- ModuleNotFoundError1.00
- Traceback (most recent call last)0.99
- Cell In[3], line 70.97
- 5 from strands import Agent1.00
- 6 from enhanced_tools import ALL_ToOLS0.97
- 7 from registry import build_index, search_tools, get_scores0.99
- aws1.00
- 9 print(f" Loaded {len(ALL_TO0LS)} tools")0.96
- File ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcore/02-semantic-tools-demo/registry.py:41.00
- 1"…"0.85
- 2 Tool Registry - FAISS semantic search over tools0.98
- 3 "……0.78
- 4 import faiss1.00
- 5 from sentence_transformers import SentenceTransformer0.99
- 6 from typing import List, Callable1.00
- ModuleNotFoundError: No module named 'faiss'0.98
- Build Semantic Index0.98
-
- EXPLORER1.00
- 0000.99
- token_efficiency_analysis.ipynb1.00
- requirements.txt1.00
- test_semantic_tools_hallucinations.ipynb1.00
- 0000.92
- WHY-AGENTS-FAIL-SAMPLE-...0.99
- 02-semantic-tools-demo >test_semantic_tools_hallucinations.ipynb >M↓ Semantic Tool Discovery: Reducing Tool Selection Halluci0.98
- .claude1.00
- Generate + Code + MarkdownRun All Restart Clear All Outputs0.97
- Go To1.00
- 0000.93
- Python 3.10.201.00
- >01-faq-graphrag-demo0.98
- import sys1.00
- 290.83
- 02-semantic-tools-demo1.00
- import io1.00
- 961.00
- __pycache__0.88
- import re1.00
- images1.00
- enhanced_tools.py1.00
- from strands import Agent1.00
- 60.51
- README.md1.00
- from registry import build_index, search_tools, get_scores1.00
- from enhanced_tools import ALL_ToOLS0.97
- registry.py0.98
- requirements.txt0.97
- print(f" Loaded {len(ALL_ToOLS)} tools")0.96
- test_semantic_tools_hallucinations.ipynb1.00
- [3]1.00
- X0.60
- 1.9s1.00
- Python1.00
- token_comparison_app.py1.00
- token_efficiency_analysis.ipynb1.00
- 0000.99
- ModuleNotFoundError1.00
- Traceback (most recent call last)1.00
- >03-multiagent-demo0.98
- Cell In[3], line 70.97
- > 04-neurosymbolic-demo0.97
- 5 from strands import Agent0.99
- > 05-agent-control-demo0.97
- 6 from enhanced_tools import ALL_T0OLS0.96
- 7 from registry import build_index, search_tools, get_scores0.99
- aws1.00
- > 06-agentcore-production-demo0.97
- 9 print(f" Loaded {len(ALL_TO0LS)} tools")0.95
- > 07-context-graph-integration0.98
- > artifacts0.97
- File ~/Documents/repositories/eli-cosas/why-agents-fail-sample-for-amazon-agentcore/02-semantic-tools-demo/t1.00
- > blog-es0.93
- 1 "…"0.73
- > images0.96
- 2 Tool Registry - FAISS semantic search over tools0.98
- 3 "…"0.74
- > langgraph-demos0.95
- 4 import faiss0.97
- .gitignore1.00
- 5 from sentence_transformers import SentenceTransformer0.99
- CLAUDE.md1.00
- 6 from typing import List, Callable0.99
- CODE_OF_CONDUCT.md1.00
- LICENSE1.00
- ModuleNotFoundError: No module named 'faiss'0.98
- ①README.md0.95
- > OUTLINE0.89
- test_nova_embeddings.py1.00
- Build Semantic Index0.99
- APPLICATION BUILDER0.97
-
- EXPLORER1.00
- 0000.99
- token_efficiency_analysis.ipynb1.00
- requirements.txt1.00
- test_semantic_tools_hallucinations.ipynb0.99
- 0000.92
- WHY-AGENTS-FAIL-SAMPLE-FOR-AMAZON-AG...1.00
- 02-semantic-tools-demo>token_efficiency_analysis.ipynb >M↓ Semantic Tool Selection: Token Efficiency & Accuracy Analysis >MJ0.98
- .claude1.00
- Generate +Code +MarkdownRun All Restart Clear All Outputs0.97
- 0000.96
- Python 3.12.131.00
- 01-faq-graphrag-demo1.00
- This notebook compares Traditional (all 31 tools) vs Semantic (top-3 filtered tools) approaches across two critical1.00
- 290.82
- 02-semantic-tools-demo1.00
- metrics:1.00
- 961.00
- __pycache__0.86
- 1. Token Consumption: How many tokens are used per query?1.00
The page's on-screen-text budget of 600
lines is spent, so the last cards in this grid list fewer lines than they
hold. Narrow the page with ?frames= to read them.
Transcript
538 cues· 6,909 words· 36,062 chars
- 0:00 Hi, today we are going to talk about how to stop AI agents' hallucinations with five techniques beyond the prom.
- 0:11 Each one is a code change, not a prom change.
- 0:16 Let's see.
- 0:18 Every time your AI agent responds, you are paying for the words going in and the words coming out.
- 0:26 in your build you will see those calling tokens yeah and the more tokens you send in the more you pay and if what you send is not quite right too much or missing something important you're again start to hallucinate there are five techniques
- 0:50 to help reduce tokens' waste, improve accuracy, and catch failures before you succeed them.
- 0:59 And each one is a code change, not a prompt change at all.
- 1:05 So let's see it.
- 1:08 First, we have semantic tool selection.
- 1:12 You filter
- 1:14 which still go into context on every call.
- 1:18 The model only see what is needs for that specific query.
- 1:23 Second, we have GraphRack for persist queries like aggregation, cons, multi-hop reasoning.
- 1:32 and you replace the text retrieval with a structural graph query.
- 1:38 The model gets a compute verifiable answer, not a sample, as RAC do.
- 1:47 Three, multi-agent validation.
- 1:50 A second agent can check every response before it rejects the users.
- 1:57 And four, neurosymbolic audience.
- 2:01 you rule life in python not in the prom and the model cannot skip them so five runtime guardians because if you don't want to block you can steer and you don't you need to block everything and when a rule fires the engine self correct and complete the task
- 2:26 no hard stop no user retries so for each technique i will show you the engine without it and then with it so you can compare and all the demos uh i'm using a travel agent that i build using strands again and strands again is an open source am framework that we maintain on edul yes
- 2:56 And I am Elizabeth Fuentes Leone.
- 2:59 I am a developer advocate for AWS.
- 3:02 I'm focused on agentics application.
- 3:05 And here in this QR code, you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment.
- 3:21 So let's get into it.
- 3:26 Let's get into it.
- 3:31 So semantic tool selection or travel agent has 29 tools, flights, hotels, payments, weather, cancellations,
- 3:44 All our dummies tools are not like a travel agent for real, but every time that user sends a message, all the 29 tools descriptions go into the context windows.
- 4:00 The model reads all of them before deciding what to do.
- 4:07 And if your agent has memory, the growth to every conversation adds more context that gets sent with every single message.
- 4:21 you pay for every single one of those tokens, whether the model ends up using the tool or not.
- 4:30 To understand where those tokens come from, you need to see what a tool actually looks like to the model.
- 4:39 In strands, you write a function with the tool decorator, that's the tool, and a name, a description, and a dog's treats
- 4:50 typed parameters, then strands take that and generate a schema with name and description parameters and the schema is what goes into the context windows on every call.
- 5:07 Each tool schema is about
- 5:10 17 or 200 tokens depending on how many parameters it has.
- 5:17 If our travel agent has 29 tools, that adds up to somewhere around 3000 tokens per call, just for the tool description.
- 5:31 before your messenger, before the response, every single call.
- 5:38 By creating a tool database, we can filter the tools that the AMA needs before the ANC log.
- 5:48 With this filter, the model sees only three most relevant tools.
- 5:55 tokens usage drops from 1000 to fewer than 300.
- 6:03 Let me show you in the code.
- 6:05 Here in my Kero IDE, this is the, let me clear all the outputs.
- 6:12 So first we need to install the requirements.
- 6:16 Here is the requirement file.
- 6:18 This is a Jupyter notebook because it's more simple to show everything, but it is one application that you can run if it is more comfortable for you.
- 6:27 So here are the requirements.
- 6:29 I have the strans agent and because I'm using OpenAI as a model invocation, I'm using the API from OpenAI and I need a strans agent for OpenAI.
- 6:41 You can use OpenStrands with almost all the model providers.
- 6:47 And with Amazon Vetro, of course, because we as AWS, we are the maintaining of this framework.
loading