Videos esY99nYXxR4

How we solved Context Management in Agents — Sally-Ann Delucia

index_state ready data_status ok

AI Engineer· published 2026-05-10· 0:16:16· en-US· indexed 2026-08-10 19:51

Open on YouTube

47 shot(s).

keyframes kept every frame deduplicated

What was stored

cues: 225; whisperx 225
chunks: 28; from 225 cues
keyframes: 35; kept of 47 captured
frames with text: 35; 427 lines read
chapters: 13; from the source metadata
keyframe bytes: 4.4 MB; word timings on 225 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage	state	model	started	took
`fetch`	done	—	2026-08-10 14:15	1m 43s
`stt`	done	—	2026-08-10 14:17	19s
`chunk`	done	—	2026-08-10 14:17	0s
`text_embed`	done	—	2026-08-10 19:50	1s
`keyframe`	done	—	2026-08-10 14:17	1m 22s
`ocr`	done	—	2026-08-10 14:18	10s
`frame_embed`	done	—	2026-08-10 19:51	6s

Frames, and what the machine read

0:03 #0 done2 line(s)

shot 0·sharpness 658.4
1. AlEngineer0.98
2. EUROPE1.00
0:08 #1 done2 line(s)

shot 1·sharpness 827.9
1. PRESENTINGSPONSOR1.00
2. Google DeepMind1.00
0:13 #2 done3 line(s)

shot 2·sharpness 907.7
1. PLATINUM SPONSORS0.98
2. # Braintrust0.96
3. WorkOS OpenAI0.95
0:25 #3 done7 line(s)

shot 3·sharpness 373.2
1. AlEngineer0.95
2. Escaping the Co0.99
3. EUROPE1.00
4. Lessons from Buildin1.00
5. EU/ACO0.98
6. AlEngineer0.97
7. EUROPE1.00
0:33 #4 done10 line(s)

shot 4·sharpness 1648.9
1. arize1.00
2. Escaping the Context Window1.00
3. AIE1.00
4. Lessons from Building Alyx1.00
5. ★1.00
6. ★1.00
7. AlEngineer0.97
8. AlEngineer0.97
9. 20260.93
10. EUROPE1.00
0:42 #5 done18 line(s)

shot 5·sharpness 3033.4
1. Hi, I'm SallyAnn!1.00
2. Head of Product at Arize0.98
3. Technical background in data1.00
4. science → now building products1.00
5. AIE1.00
6. for teams1.00
7. ★1.00
8. ★1.00
9. ★1.00
10. Hands-on: core contributor to our1.00
11. own agent Alyx, so I know the pain0.99
12. of building firsthand0.98
13. My job: turn that pain into tools1.00
14. that actually help1.00
15. AlEngineer0.96
16. AlEngineer0.98
17. 20260.92
18. EUROPE1.00
1:25 #6 done32 line(s)

shot 6·sharpness 2602.1
1. What Is Alyx?0.99
2. Ask Alyx1.00
3. Identifying Common Person...0.99
4. Evaluator Recommendation1.00
5. Al Harness to help you build your Al1.00
6. Applications1.00
7. Advanced Planning1.00
8. 40+ Skills0.99
9. Core Workflows: Prompt Opt, Data1.00
10. Make the most out of Alyx1.00
11. AIE1.00
12. ★1.00
13. ★1.00
14. Annotations, etc1.00
15. Generation and Augmentation,1.00
16. Find critical issues, categorize errors, apply annotations, and build evals0.99
17. Summarize my evaluations0.99
18. How can I use Arize?0.96
19. ★1.00
20. Categorize errors and apply annotations1.00
21. ★1.00
22. ★1.00
23. ★1.00
24. Find critical issues and create an eval0.99
25. Ask Alyx a question, type @ for context0.99
26. AlClaude Sonnet 4.50.99
27. arize We Make Al Work0.94
28. © All Rights Reserved0.97
29. AlEngineer0.96
30. AlEngineer1.00
31. 20260.95
32. EUROPE1.00
1:38 #7 done21 line(s)

shot 7·sharpness 1965.3
1. Agenda1.00
2. /011.00
3. /021.00
4. /030.99
5. The Problem1.00
6. The Vicious Loop0.99
7. Escaping the Loop0.99
8. AIE1.00
9. ★1.00
10. ★1.00
11. /041.00
12. /051.00
13. /060.99
14. Long Conversations Break1.00
15. Sub-Agents: Moving1.00
16. What Still Doesn't Work1.00
17. Agents1.00
18. Heavy Context Out0.99
19. Engineering the future of Al1.00
20. AlEngineer0.99
21. 20260.99
1:55 #8 done24 line(s)

shot 8·sharpness 3968.2
1. Andrej Karpathy0.99
2. @karpathy1.00
3. The stack is0.97
4. +1 for "context engineering" over "prompt1.00
5. changing.1.00
6. AIE1.00
7. ★1.00
8. engineering".1.00
9. .. Too little or of the wrong form and the LLM0.99
10. Context is the0.98
11. ★1.00
12. ★1.00
13. doesn't have the right context for optimal1.00
14. performance. Too much or too irrelevant and the1.00
15. new engineering1.00
16. LLM costs might go up and performance might1.00
17. problem.1.00
18. come down. Doing this well is highly non-trivial.1.00
19. And art because of the guiding intuition around0.98
20. LLM psychology of people spirits...0.99
21. Tweet link1.00
22. Engineering the future of Al1.00
23. AlEngineer0.99
24. 20260.99
2:19 #9 done18 line(s)

shot 9·sharpness 3486.6
1. iru0.99
2. 00.52
3. Updates1.00
4. 4:591.00
5. Update Available0.99
6. Google Chrome1.00
7. Update Now0.99
8. PERSPECTIVE1.00
9. Delay 1 Hour0.95
10. AIE1.00
11. The best context strategy is the one that0.99
12. ★1.00
13. ★1.00
14. lets your agent remember what matters1.00
15. and forget what doesn't.0.99
16. Engineering the future of Al0.99
17. AlEngineer0.97
18. 20260.97
2:53 #10 done14 line(s)

shot 10·sharpness 2649.3
1. Why Context Management Matters1.00
2. ★0.54
3. AIE1.00
4. ★0.99
5. Context engineering = choosing what the model sees1.00
6. ★1.00
7. ★1.00
8. Not just staying under a token limit.1.00
9. arize We Make Al Work0.96
10. All Rights Reserved0.95
11. AlEngineer0.97
12. AlEngineer1.00
13. 20260.99
14. EUROPE1.00
3:27 #11 done7 line(s)

shot 11·sharpness 400.1
1. ality1.00
2. AlEngineer1.00
3. EUROPE1.00
4. EU/ACC0.97
5. Being strategic about contex1.00
6. AlEngineer0.99
7. EUROPE1.00
3:36 #12 done11 line(s)

shot 12·sharpness 2574.0
1. Alyx Reality0.98
2. One Trace0.98
3. wser context0.92
4. AIE1.00
5. ★1.00
6. ★1.00
7. this multiptes..0.94
8. Being strategic about context was non-negotiable.0.99
9. Engineering the future of Al0.97
10. AlEngineer1.00
11. 20260.97
3:41 #13 done7 line(s)

shot 13·sharpness 2397.8
1. AIE1.00
2. Context management is a product + UX problem0.99
3. ★1.00
4. Not just an engineering one.1.00
5. Engineering the future of Al0.98
6. AlEngineer1.00
7. 20260.90
4:23 #14 skipped

shot 14·duplicate of #12
4:48 #15 done9 line(s)

shot 15·sharpness 2360.8
1. AIE1.00
2. The system analyzing the data was constrained1.00
3. ★1.00
4. ★1.00
5. by the data.1.00
6. AlEngineer0.97
7. AlEngineer1.00
8. 20260.99
9. EUROPE1.00
4:58 #16 done17 line(s)

shot 16·sharpness 1910.7
1. 11.00
2. Control Context1.00
3. Escaping the1.00
4. AIE1.00
5. ★1.00
6. Loop1.00
7. 21.00
8. Separate context from memory0.99
9. ★1.00
10. ★1.00
11. ★1.00
12. 31.00
13. Move heavy work out1.00
14. AlEngineer0.96
15. AlEngineer0.99
16. 20260.99
17. EUROPE1.00
5:13 #17 done10 line(s)

shot 17·sharpness 367.6
1. AlEngineer0.99
2. Contr1.00
3. EUROPE1.00
4. Escaping the1.00
5. Loop1.00
6. Separ1.00
7. EU/ACC0.98
8. Move1.00
9. AlEngineer0.99
10. EUROPE1.00
5:21 #18 done9 line(s)

shot 18·sharpness 365.3
1. runcation0.99
2. AlEngineer0.99
3. EUROPE1.00
4. NAIVE1.00
5. First 100 chars0.96
6. rest dropped1.00
7. EU/ACC0.91
8. AlEngineer0.99
9. EUROPE1.00
5:43 #19 done13 line(s)

shot 19·sharpness 2402.4
1. Naive Truncation0.97
2. Result:1.00
3. NAIVE1.00
4. AIE1.00
5. The agent forgot everything.1.00
6. ★1.00
7. First 100 chars0.98
8. ... rest dropped0.90
9. Follow-ups looked like new1.00
10. conversations.1.00
11. Engineering the future of Al1.00
12. AlEngineer1.00
13. 20260.98
6:39 #20 done13 line(s)

shot 20·sharpness 2601.9
1. LLM summarization as compression1.00
2. Sounded like the obvious solution.1.00
3. But:1.00
4. AIE1.00
5. too inconsistent1.00
6. ★1.00
7. ★1.00
8. No control over what was important1.00
9. unreliable1.00
10. arizeWe Make Al Work1.00
11. Google DeepMind0.99
12. AlEngineer1.00
13. 20260.89
6:54 #21 done22 line(s)

shot 21·sharpness 3230.1
1. Solution: Smart Truncation + Memory0.98
2. SMART1.00
3. Head1.00
4. ..truncated0.90
5. Tail1.00
6. ID: x (retrievable)0.98
7. AIE1.00
8. ★1.00
9. ★1.00
10. ★1.00
11. Memory1.00
12. Better context strategies1.00
13. Retrieve by ID1.00
14. remove duplicate messages/1.00
15. tool calls0.97
16. keep latest result1.00
17. don't reset system prompt0.98
18. truncate middle (keep head +1.00
19. tail)1.00
20. Engineering the future of Al1.00
21. AlEngineer0.97
22. 20260.96
7:27 #22 skipped

shot 22·duplicate of #11
7:55 #23 done8 line(s)

shot 23·sharpness 2475.3
1. AIE1.00
2. Context decides what the model sees.1.00
3. ★1.00
4. Memory decides what survives.1.00
5. AlEngineer0.97
6. AlEngineer1.00
7. 20260.92
8. EUROPE1.00

Transcript

225 cues· 3,442 words· 18,400 chars

chunk 0 · 0:15–1:00 · 170 words · 886 chars
0:15 All right, welcome.
0:15 Thanks so much for coming today.
0:17 I'm here to talk a little bit about context windows, and I'm really excited because I get to talk about something that my team and I have been building for, honestly, close to a year now, which is RIA Agent Alex.
0:27 So I'm gonna talk a little bit about some of the lessons we learned about context management and escaping the context window.
0:34 So, who am I?
0:35 I'm Sally Ann.
0:36 I'm the head of product at Arise.
0:38 I have a technical background.
0:39 I started out in data science, and now I build products for teams.
0:42 I'm hands-on.
0:43 I'm a core contributor of Alex.
chunk 1 · 0:45–1:27 · 157 words · 835 chars
0:45 I'm not only a PM, but I also function a little bit as a part-time AI engineer as well.
0:50 So I know the pain of building these products firsthand, and it is not easy to build a successful agent.
0:55 My job today really is to turn those pains into tools that may actually help AI engine AI PMs.
1:01 I'm gonna talk a little bit about Alex.
1:03 I don't wanna spend a lot of time on Alex.
1:04 If you wanna know more about what we built, come find me in the booth downstairs.
1:07 I'll give you a demo, but basically what Alex is is an AI harness.
1:11 It's here to help you build your AI applications.
1:13 We have advanced planning, 40 plus skills built into it, core workflows across prompt engineering, like prompt optimization, data gen, data augmentation, annotations, et cetera.
chunk 2 · 1:24–2:08 · 164 words · 927 chars
1:24 That's just a screenshot from our product, but yeah, come find me if you'd like a demo.
1:29 For today's talk, I'm gonna talk a little bit about the problem of context engineering, context management, tell you a little bit about a vicious loop that we got stuck in, how we escaped that loop, and then how long conversations can break agents, a little bit about what we learned about sub-agents, and then I'll tell you a little bit about what we're still working on, because we certainly haven't figured everything out.
1:47 So, the problem.
1:48 I think like mid last year, this term context engineering started to become more and more popular.
1:52 This is an X from Andre Caparthi about plus one in context engineering over prompt engineering.
chunk 3 · 1:58–2:43 · 163 words · 901 chars
1:58 I think very early on, everybody was really, really focused on the prompts, but we started to realize that the context is what really made an agent fail or succeed.
2:06 And so the stack has really changed.
2:08 We're no longer really focused just on the prompts, we're focused on the new engineering problem, which is context.
2:13 So,
2:14 My little perspective is the best context strategy is one that lets your agents remember what it needs to and forget what it doesn't.
2:22 And so we're gonna talk a little bit about how you do that, but first let's talk about why context management even matters.
2:28 So I think a lot of folks think, um, like context management is just like what fits in the window, but context engineering is really choosing strategically what the model sees.
chunk 4 · 2:37–3:22 · 187 words · 984 chars
2:37 It's really important that you think about what the data is that is most important and not just think about, Oh, I only have X amount of tokens.
2:43 Let's shove as much as I can in there and see how it does.
2:45 So it's not just saying under that token limit,
2:49 It's being strategic about it.
2:50 And that's why it really matters.
2:52 All these different applications, a lot of times, it's running on top of your context.
2:55 And so what you choose to let the model see really matters.
2:58 It can make or break the experience there.
3:00 And so our reality with Alex is Alex is built on top of Arise, which is our observability platform.
3:05 So we have to deal with all of the traces that come with AI agents.
chunk 5 · 3:09–3:45 · 141 words · 756 chars
3:09 And so we have one trace we are getting
3:11 the input from the user, there's prompts, there's all of this metadata, then the user is interacting with Alex.
3:16 And so it becomes really large.
3:18 And that's just when we're talking about one trace, but what happens when they wanna see patterns across all of their traces?
3:22 Well, this just continues to multiply and multiply and multiply.
3:26 So being strategic about context was a non-negotiable for us.
3:29 We really had to figure out, okay, what was most important for Alex to see?
3:32 And how do we handle when it needs to kind of see everything?

Chapters

0:00 Introduction and speaker background
1:02 Overview of the AI agent, Alyx
1:29 The problem: Context engineering vs. prompt engineering
4:06 The vicious loop of data growth in AI agents
5:16 Why naive truncation failed
6:14 Why summarization proved unreliable
6:46 The solution: Smart truncation and memory stores
8:02 Handling long session challenges
9:23 Offloading tasks to sub-agents
11:19 Ongoing challenges and future work
12:57 Findings from the Claude Code source release
13:44 Final key takeaways on context management
14:58 Q&A session