Videos u6jJcIFDLE4

Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates

index_state ready data_status ok

AI Engineer· published 2026-07-23· 0:15:00· en-US· indexed 2026-08-10 19:46

Open on YouTube

34 shot(s).

keyframes kept every frame deduplicated

What was stored

cues: 240; whisperx 240
chunks: 27; from 240 cues
keyframes: 22; kept of 34 captured
frames with text: 22; 330 lines read
chapters: 9; from the source metadata
keyframe bytes: 3.3 MB; word timings on 240 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage	state	model	started	took
`fetch`	done	—	2026-08-10 03:33	1m 50s
`stt`	done	—	2026-08-10 03:35	19s
`chunk`	done	—	2026-08-10 03:35	0s
`text_embed`	done	—	2026-08-10 19:46	0s
`keyframe`	done	—	2026-08-10 03:36	1m 23s
`ocr`	done	—	2026-08-10 03:37	11s
`frame_embed`	done	—	2026-08-10 19:46	4s

Frames, and what the machine read

0:02 #0 done2 line(s)

shot 0·sharpness 452.4
1. AlEngineer0.96
2. World's Fair1.00
0:03 #1 done2 line(s)

shot 1·sharpness 661.8
1. AlEngineer0.95
2. World's Fair0.99
0:10 #2 done24 line(s)

shot 2·sharpness 2734.4
1. LAB & PLATINUM SPONSORS0.99
2. Amazon AGI Lab0.98
3. ANTHROP\C1.00
4. Google DeepMind1.00
5. MINIMAX0.94
6. OpenAl0.92
7. Akamai1.00
8. arize1.00
9. aws1.00
10. Braintrust bright data0.98
11. B1.00
12. Browserbase1.00
13. docker1.00
14. :neo4j0.93
15. ORACLE1.00
16. PayPal1.00
17. qodo1.00
18. reducto1.00
19. Sonar1.00
20. Makers of0.99
21. togetherai1.00
22. Unblocked1.00
23. WorkOS1.00
24. SonarQube1.00
0:17 #3 done2 line(s)

shot 3·sharpness 388.3
1. AlEngineer1.00
2. World's Fair0.99
0:31 #4 done13 line(s)

shot 4·sharpness 1595.5
1. AlEngineer0.98
2. World'sFair1.00
3. Why We Killed Our1.00
4. PRESENTED BY1.00
5. Multi-Agent Pipeline1.00
6. Microsoft1.00
7. Lessons from building Al for pharma commercial teams0.99
8. Subbiah Sethuraman1.00
9. Abhilash Asokan1.00
10. ZS Associates0.97
11. Al Engineer World's Fair 20261.00
12. World'sFair0.88
13. Engineering the future of Al0.98
0:59 #5 done14 line(s)

shot 5·sharpness 1449.3
1. AlEngineer0.98
2. World'sFair1.00
3. Why We Killed Our1.00
4. PRESENTED BY0.97
5. Multi-Agent Pipeline1.00
6. Microsoft1.00
7. Lessons from building Al for pharma commercial teams0.99
8. Subbiah Sethuraman1.00
9. Abhilash Asokan1.00
10. ZS Associates0.97
11. Al Engineer World's Fair 20260.99
12. Worid'sFair0.97
13. TRACK 5· JULY 2,20260.96
14. Graphs1.00
1:27 #6 done18 line(s)

shot 6·sharpness 1028.7
1. AlEngineer0.99
2. How an analyst works1.00
3. World's Fair0.97
4. Four questions, answered in order.0.99
5. 21.00
6. 31.00
7. 41.00
8. Signal1.00
9. Why1.00
10. Action1.00
11. Outlook1.00
12. Is the change real?0.99
13. What's causing it?0.98
14. What should we do?1.00
15. What happens next?1.00
16. World's Fair0.99
17. TRACK 5• JULY 2, 20260.96
18. Graphs1.00
2:12 #7 skipped

shot 7·duplicate of #6
2:47 #8 done20 line(s)

shot 8·sharpness 1442.2
1. AlEngineer0.99
2. We built an agentic system to mimic that1.00
3. World's Fair0.98
4. One agent per step, an orchestrator running the team.1.00
5. Orchestrator1.00
6. runs the team1.00
7. SIGNAL1.00
8. WHY IT'S HAPPENING0.99
9. ACTION + OUTLOOK0.99
10. Signal1.00
11. Source1.00
12. Driver1.00
13. Detection1.00
14. Localization1.00
15. Attribution1.00
16. Synthesis1.00
17. two agents to answer one question1.00
18. Worid's Fair0.93
19. TRACK 5· JULY 2, 20260.95
20. Graphs1.00
3:00 #9 skipped

shot 9·duplicate of #8
3:56 #10 done15 line(s)

shot 10·sharpness 2669.4
1. AlEngineer0.96
2. The packet it generated1.00
3. World'sFair1.00
4. SIGNAL1.00
5. Brand A's new prescriptions dropped 18% in Territory X over four weeks.1.00
6. WHY1.00
7. Almost all of the drop is in one insurer, Payer Y. They moved Brand A to a worse coverage tier, so0.99
8. patients now pay more — and many walk away at the pharmacy counter.1.00
9. ACTION1.00
10. Send sales reps to visit more doctors in Territory X to push prescriptions back up.1.00
11. OUTLOOK0.96
12. The overall outlook for the brand is healthy heading into next quarter.0.99
13. World'sFair0.90
14. TRACK 5· JULY 2, 20260.95
15. Graphs1.00
4:13 #11 done16 line(s)

shot 11·sharpness 2027.6
1. AlEngineer0.97
2. It looks correct. Until you read it closely.1.00
3. World'sFair1.00
4. THE CAUSE1.00
5. Patients can't afford the drug after an insurance change0.99
6. THE ACTION1.00
7. Reps should visit more doctors — which says nothing about insurance0.99
8. THE OUTLOOK1.00
9. "Recoverable if rep activity improves"0.97
10. And here's the unsettling part1.00
11. The action doesn't match the cause.0.99
12. Nothing was made up. Every fact is real — but no single agent ever saw the whole0.98
13. picture.1.00
14. World'sFair0.89
15. TRACK 5· JULY 2,20260.95
16. Graphs1.00
4:54 #12 done17 line(s)

shot 12·sharpness 2062.5
1. AlEngineer0.96
2. Why it failed0.97
3. World'sFair1.00
4. Not the model. The way we split the work.0.99
5. A language model was deciding the signal1.00
6. We asked an LLM whether an 18% move was a real signal or normal noise — a question statistics should0.98
7. answer, not a model.0.97
8. 21.00
9. Context was lost at every handoff0.99
10. Each agent passed on its conclusion, not its reasoning. The clue that explained the drop never reached the0.99
11. agent writing the advice.1.00
12. No shared understanding of the business0.98
13. Each agent inferred how pharma metrics relate from raw tables. Nothing told them what actually drives0.99
14. prescriptions, or in which direction.1.00
15. World'sFair0.85
16. TRACK 5· JULY 2, 20260.95
17. Graphs1.00
5:25 #13 done18 line(s)

shot 13·sharpness 2193.7
1. AlEngineer0.97
2. Why it failed0.96
3. World'sFair0.93
4. Not the model. The way we split the work.0.99
5. A language model was deciding the signal1.00
6. PRESENTED BY1.00
7. We asked an LLM whether an 18% move was a real signal or normal noise — a question statistics should0.98
8. answer, not a model.0.96
9. Microsoft1.00
10. Context was lost at every handoff0.99
11. Each agent passed on its conclusion, not its reasoning. The clue that explained the drop never reached the0.99
12. agent writing the advice.0.99
13. No shared understanding of the business0.98
14. Each agent inferred how pharma metrics relate from raw tables. Nothing told them what actually drives0.99
15. prescriptions, or in which direction.1.00
16. Word'sFair0.97
17. TRACK 5· JULY 2,20260.95
18. Graphs1.00
5:57 #14 skipped

shot 14·duplicate of #13
6:06 #15 done14 line(s)

shot 15·sharpness 1384.5
1. AlEngineer0.97
2. World'sFair1.00
3. So we stoppeddesigningthe architecture — and0.99
4. started deriving it.0.98
5. PRESENTED BY0.96
6. Microsoft1.00
7. A bare agent.0.98
8. Bash and the database1.00
9. One real KPI decline.0.96
10. Then we watched.0.99
11. What the agent touches, we derived from traces. Where the math lives, we decided up front.1.00
12. World'sFair0.95
13. TRACK 5· JULY 2,20260.95
14. Graphs1.00
6:43 #16 done12 line(s)

shot 16·sharpness 1250.6
1. AlEngineer0.97
2. World'sFair1.00
3. So we stoppeddesigningthe architecture — and0.99
4. started deriving it.0.98
5. A bare agent.0.99
6. Bash and the database1.00
7. One real KPI decline.0.97
8. Then we watched.0.99
9. What the agent touches, we derived from traces. Where the math lives, we decided up front.1.00
10. World'sFair0.92
11. TRACK 5· JULY 2,20260.95
12. Graphs1.00
6:58 #17 done15 line(s)

shot 17·sharpness 1694.2
1. FIX 1 · Language model determined the signal0.97
2. AlEngineer0.97
3. World'sFair1.00
4. Statistics finds the signal. The agent explains it.1.00
5. STATISTICS1.00
6. THEAGENT1.00
7. Find the signal.1.00
8. Explains it.0.99
9. Is an 18% drop real, or just noise? Deterministic tests0.99
10. Once the signal is real, the agent investigates, reasons1.00
11. and guardrails decide — before the agent starts.1.00
12. about cause, and writes the recommendation.1.00
13. A language model is a reasoning engine, not a statistics engine.0.99
14. TRACK 5· JULY 2,20260.95
15. Graphs1.00
7:45 #18 skipped

shot 18·duplicate of #17
8:04 #19 done15 line(s)

shot 19·sharpness 1380.3
1. FIX 2 · context was lost at every handoff0.98
2. AlEngineer0.97
3. What replaced the pipeline.1.00
4. World'sFair1.00
5. Knowledge graph1.00
6. Sub-agent1.00
7. Signal Detection1.00
8. Analyst agent1.00
9. Sub-agent1.00
10. launched for one isolated task — e.g. verify a driver0.99
11. branch: did rep activity really fall?1.00
12. One agent owns the whole investigation. Sub-agents investigate. Knowledge Graph provides pharma context0.99
13. World'sFair0.97
14. TRACK 5· JULY 2,20260.95
15. Graphs1.00
8:40 #20 skipped

shot 20·duplicate of #19
9:13 #21 skipped

shot 21·duplicate of #19
9:48 #22 done29 line(s)

shot 22·sharpness 1819.0
1. FIX 3 · No shared understanding of the business0.98
2. AlEngineer0.99
3. We gave the agent the business domain — as a graph.0.99
4. World'sFair1.00
5. WHERE· localize first0.97
6. WHY· only after WHERE0.99
7. National1.00
8. Brand A1.00
9. Payer Y0.99
10. CONTAINS1.00
11. SETS1.00
12. Region1.00
13. OF1.00
14. Coverage tier ↓0.99
15. Territory X0.97
16. measured af0.96
17. NRx↓18%1.00
18. Patient cost ↑1.00
19. + raises0.98
20. Account1.00
21. + small0.95
22. - suppresses0.97
23. Abandoned fills ↑0.97
24. HCP1.00
25. Rep reach small0.96
26. geography entity driver signal0.97
27. The graph didn't just add context. It structured the investigation.1.00
28. TRACK 5· JULY 2,20260.96
29. Graphs1.00
10:23 #23 done31 line(s)

shot 23·sharpness 1945.7
1. FIX 3 · No shared understanding of the business0.98
2. AlEngineer0.99
3. We gave the agent the business domain — as a graph.0.99
4. World'sFair1.00
5. WHERE·localize first0.99
6. WHY· only after WHERE0.99
7. National1.00
8. Brand A1.00
9. Payer Y1.00
10. CONTAINS1.00
11. SETS1.00
12. Region1.00
13. OF1.00
14. PRESENTED BY1.00
15. Coverage tier ↓0.99
16. Microsoft1.00
17. Territory X1.00
18. measured at0.96
19. NRx↓18%1.00
20. Patient cost ↑0.99
21. + raises0.97
22. Account1.00
23. + small0.94
24. - suppresses0.98
25. Abandoned fills ↑0.99
26. HCP1.00
27. Rep reach ↓ small0.93
28. geography entity driver signal0.98
29. The graph didn't just add context. It structured the investigation.1.00
30. TRACK 5· JULY 2,20260.97
31. Graphs1.00

Transcript

240 cues· 2,567 words· 13,967 chars

chunk 0 · 0:13–0:58 · 149 words · 819 chars
0:13 Okay, so I'm Subbaya.
0:14 I head AI engineering at ZS.
0:17 I'm Abhilash.
0:17 I'm director of AI engineering at ZS.
0:20 So ZS, we are a tech firm.
0:21 We work with many of the top companies in the world, including a lot of the top pharmas, actually.
0:26 So today's talk, I think we wanted to, as I think already introduced, right?
0:29 We wanted to talk from our experience, right?
0:31 Building multi-agent pipelines.
0:33 What are the mistakes we did, right?
0:35 And what did we learn and how did we fix them?
0:38 So I'm going to orient it more on pharma, commercial domain, and for people in the room probably who are not aware quickly, pharma has two main functions.
chunk 1 · 0:46–1:31 · 140 words · 812 chars
0:46 One is R&D, the drug discovery, and the clinical trials part.
0:50 And then there's the commercial.
0:51 How do you take a drug to a patient, basically?
0:55 And within commercial, there are different functions.
0:58 Like once you create a drug,
1:00 What is the performance of a brand?
1:01 How is the drug performing in different markets?
1:05 Then there are things like your field force, your reps. How effectively are they engaging with everyone?
1:10 There are things around patient journey, how patients are adopting a drug.
1:14 I think if there is any therapy switch which is happening.
chunk 2 · 1:16–1:59 · 148 words · 802 chars
1:16 So as you can think about, there is a lot of analytics which really happens in a commercial domain.
1:22 And how do typically analysts work?
1:25 So there are four steps, right, what analysts do, right?
1:28 So first, there is always something called a signal detection, right?
1:31 So signal can be something like, okay, the prescriptions what a doctor is writing, maybe is there a drop in the prescription?
1:37 So that's a signal.
1:39 So once you got a signal, the second thing what an analyst does is why is this signal really failing?
1:44 What is the reason for it?
1:45 So is it like there is a competitor drug which has come in?
chunk 3 · 1:48–2:32 · 147 words · 757 chars
1:48 Because of that, is it reducing?
1:50 Is it because maybe a payer coverage for the drug has reduced?
1:53 Or maybe the reps on the ground, there's no proper, they're actually not taking the benefits to the doctors.
2:01 And once you arrive at the reason, the next step becomes, okay, what is the action do you take?
2:05 So if reps, suppose if reps, the coverage is not good in a particular region, do we have to increase that?
2:12 And once you do that, what is the outlook?
2:15 Is my brand, is my sales performance, is it going to improve?
chunk 4 · 2:18–3:02 · 156 words · 821 chars
2:18 So these are the four things which happens.
2:21 Now, for some of the top farmers, what we have done is how do we, in an agentic way, right?
2:26 I think how do we actually mimic this analytics work?
2:29 So what we did, we built agents for every step, right?
2:33 Signal detection, we said, okay, we'll have an agent for signal detection.
2:36 It'll identify the signals for me.
2:39 Second, what are the root cause, right, for the signals, right?
2:42 So in this case, we have two agents.
2:43 One, we call it a source localization.
2:45 So for example, if my sales is dropping at a national level,
chunk 5 · 2:48–3:33 · 162 words · 905 chars
2:48 Is it because it's dropping at say a particular region or is it dropping for a payer?
2:53 So we need to understand that.

Chapters

0:00 Pharma commercial analytics and the analyst's four steps
2:33 V1: an agent for every step
3:26 Why the output was incoherent
4:32 Why it failed: signals, handoffs, and missing domain
5:57 The rebuild: watching Claude Code in an empty directory
7:01 Deterministic signal detection before the agent
8:05 Consolidating to a single agent
9:22 The knowledge graph as a control plane
11:04 Every edge a hypothesis, and the result