vidtheque.

read-only demo

Videos 2czYyrTzILg

From Chaos to Choreography: Multi-Agent Orchestration Patterns That Actually Work — Sandipan Bhaumik

index_state ready data_status ok

AI Engineer· published 2026-04-08· 0:26:28· en-US· indexed 2026-08-10 19:50

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:26, 1 of 1 keyframes kept
  2. Shot 1, 0:26 to 0:53, 0 of 1 keyframes kept
  3. Shot 2, 0:53 to 1:32, 1 of 1 keyframes kept
  4. Shot 3, 1:32 to 1:57, 1 of 1 keyframes kept
  5. Shot 4, 1:57 to 2:34, 1 of 1 keyframes kept
  6. Shot 5, 2:34 to 3:08, 1 of 1 keyframes kept
  7. Shot 6, 3:08 to 3:38, 1 of 1 keyframes kept
  8. Shot 7, 3:38 to 4:08, 0 of 1 keyframes kept
  9. Shot 8, 4:08 to 4:14, 1 of 1 keyframes kept
  10. Shot 9, 4:14 to 4:42, 1 of 1 keyframes kept
  11. Shot 10, 4:42 to 5:23, 1 of 1 keyframes kept
  12. Shot 11, 5:23 to 5:28, 1 of 1 keyframes kept
  13. Shot 12, 5:28 to 5:54, 1 of 1 keyframes kept
  14. Shot 13, 5:54 to 6:22, 1 of 1 keyframes kept
  15. Shot 14, 6:22 to 6:49, 1 of 1 keyframes kept
  16. Shot 15, 6:49 to 7:16, 0 of 1 keyframes kept
  17. Shot 16, 7:16 to 7:42, 0 of 1 keyframes kept
  18. Shot 17, 7:42 to 8:21, 1 of 1 keyframes kept
  19. Shot 18, 8:21 to 8:56, 1 of 1 keyframes kept
  20. Shot 19, 8:56 to 9:32, 1 of 1 keyframes kept
  21. Shot 20, 9:32 to 10:16, 1 of 1 keyframes kept
  22. Shot 21, 10:16 to 10:48, 1 of 1 keyframes kept
  23. Shot 22, 10:48 to 11:20, 0 of 1 keyframes kept
  24. Shot 23, 11:20 to 11:38, 1 of 1 keyframes kept
  25. Shot 24, 11:38 to 12:11, 1 of 1 keyframes kept
  26. Shot 25, 12:11 to 12:43, 0 of 1 keyframes kept
  27. Shot 26, 12:43 to 13:11, 1 of 1 keyframes kept
  28. Shot 27, 13:11 to 13:39, 1 of 1 keyframes kept
  29. Shot 28, 13:39 to 14:07, 0 of 1 keyframes kept
  30. Shot 29, 14:07 to 14:43, 1 of 1 keyframes kept
  31. Shot 30, 14:43 to 15:19, 1 of 1 keyframes kept
  32. Shot 31, 15:19 to 15:55, 1 of 1 keyframes kept
  33. Shot 32, 15:55 to 16:31, 0 of 1 keyframes kept
  34. Shot 33, 16:31 to 16:51, 1 of 1 keyframes kept
  35. Shot 34, 16:51 to 17:19, 1 of 1 keyframes kept
  36. Shot 35, 17:19 to 17:48, 0 of 1 keyframes kept
  37. Shot 36, 17:48 to 18:16, 1 of 1 keyframes kept
  38. Shot 37, 18:16 to 19:00, 1 of 1 keyframes kept
  39. Shot 38, 19:00 to 19:33, 1 of 1 keyframes kept
  40. Shot 39, 19:33 to 20:06, 0 of 1 keyframes kept
  41. Shot 40, 20:06 to 20:56, 1 of 1 keyframes kept
  42. Shot 41, 20:56 to 21:29, 1 of 1 keyframes kept
  43. Shot 42, 21:29 to 22:02, 0 of 1 keyframes kept
  44. Shot 43, 22:02 to 22:28, 1 of 1 keyframes kept
  45. Shot 44, 22:28 to 22:53, 0 of 1 keyframes kept
  46. Shot 45, 22:53 to 23:18, 0 of 1 keyframes kept
  47. Shot 46, 23:18 to 23:43, 0 of 1 keyframes kept
  48. Shot 47, 23:43 to 24:08, 1 of 1 keyframes kept
  49. Shot 48, 24:08 to 24:33, 0 of 1 keyframes kept
  50. Shot 49, 24:33 to 24:58, 0 of 1 keyframes kept
  51. Shot 50, 24:58 to 25:28, 1 of 1 keyframes kept
  52. Shot 51, 25:28 to 25:58, 1 of 1 keyframes kept
  53. Shot 52, 25:58 to 26:28, 0 of 1 keyframes kept

53 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
252
whisperx 252
chunks
50
from 252 cues
keyframes
36
kept of 53 captured
frames with text
36
546 lines read
chapters
0
from the source metadata
keyframe bytes
4.7 MB
word timings on 252 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-10 13:59 1m 23s
stt done — 2026-08-10 14:00 28s
chunk done — 2026-08-10 14:01 0s
text_embed done — 2026-08-10 19:50 1s
keyframe done — 2026-08-10 14:01 1m 12s
ocr done — 2026-08-10 14:02 11s
frame_embed done — 2026-08-10 19:50 6s

Frames, and what the machine read

  • 0:18 #0 done7 line(s)

    shot 0·sharpness 951.9

    1. MULTI-AGENT1.00
    2. ORCHESTRATION1.00
    3. PATTERNS1.00
    4. Sandipan Bhaumik0.98
    5. Data & Al Tech Lead0.98
    6. Databricks0.98
    7. April 20260.99

  • 0:40 #1 skipped

    shot 1·duplicate of #0

  • 0:58 #2 done10 line(s)

    shot 2·sharpness 1213.8

    1. • PROBLEM0.97
    2. War story, why complexity explodes0.99
    3. • PATTERNS0.97
    4. AGENDA1.00
    5. Coordination1.00
    6. State management (immutable)1.00
    7. Failure recovery1.00
    8. • IMPLEMENTATION0.98
    9. Production architecture1.00
    10. April 20260.95

  • 1:54 #3 done5 line(s)

    shot 3·sharpness 536.8

    1. ONE AGENT0.98
    2. WORKS1.00
    3. Input1.00
    4. Output1.00
    5. One agent is a feature1.00

  • 2:19 #4 done10 line(s)

    shot 4·sharpness 673.2

    1. FIVE AGENTS1.00
    2. A1.00
    3. ARE CHAOS1.00
    4. C0.99
    5. B1.00
    6. F1.00
    7. Five agents is a1.00
    8. distributed1.00
    9. E0.99
    10. systems problem1.00

  • 3:04 #5 done15 line(s)

    shot 5·sharpness 927.6

    1. WAR STORY0.96
    2. THE RACE CONDITION1.00
    3. Incorrect1.00
    4. Risk1.00
    5. Ratings1.00
    6. Incorrectly1.00
    7. flagged1.00
    8. customers1.00
    9. Week 2:1.00
    10. Week 3 - 5:0.92
    11. 5 Agents Live in1.00
    12. Firefighting1.00
    13. Production1.00
    14. Delayed1.00
    15. Value1.00

  • 3:26 #6 done16 line(s)

    shot 6·sharpness 906.8

    1. WAR STORY1.00
    2. THE BUG0.96
    3. 6801.00
    4. write score1.00
    5. read score1.00
    6. (T= 0ms)0.95
    7. (T=500ms)1.00
    8. A1.00
    9. B1.00
    10. stale data0.97
    11. Credit Score1.00
    12. Database1.00
    13. Cache1.00
    14. Risk Assessment1.00
    15. Calculation Agent1.00
    16. Agent1.00

  • 3:50 #7 skipped

    shot 7·duplicate of #6

  • 4:11 #8 done25 line(s)

    shot 8·sharpness 1154.0

    1. WAR STORY1.00
    2. THE BUG1.00
    3. Agent B is1.00
    4. reading1.00
    5. stale1.00
    6. 6801.00
    7. version of0.97
    8. write score1.00
    9. X1.00
    10. read score1.00
    11. (T= 0ms)0.98
    12. (T=500ms)1.00
    13. A1.00
    14. B1.00
    15. the data -0.99
    16. stale data0.99
    17. that leads1.00
    18. Credit Score1.00
    19. Database1.00
    20. Cache1.00
    21. Risk Assessment1.00
    22. to wrong1.00
    23. Calculation Agent1.00
    24. Agent1.00
    25. decisions1.00

  • 4:25 #9 done5 line(s)

    shot 9·sharpness 943.4

    1. THE PROBLEM WASN'T THE LLM0.99
    2. We built a distributed1.00
    3. system...1.00
    4. ...without1.00
    5. distributed system thinking0.99

  • 5:14 #10 done6 line(s)

    shot 10·sharpness 997.5

    1. THE1.00
    2. COMPLEXITY1.00
    3. Nu nts0.86
    4. CURVE1.00
    5. 5 agents = 25x comnlex0.93
    6. Coordination complexity1.00

  • 5:25 #11 done6 line(s)

    shot 11·sharpness 2113.0

    1. THE1.00
    2. COMPLEXITY1.00
    3. Nu nts0.82
    4. CURVE1.00
    5. 5 agents = 25x comnlex0.98
    6. Coordination complexity1.00

  • 5:39 #12 done1 line(s)

    shot 12·sharpness 534.5

    1. PATTERNS1.00

  • 6:16 #13 done7 line(s)

    shot 13·sharpness 1426.3

    1. THE FUNDAMENTAL QUESTION1.00
    2. How Should Your Agents Coordinate?1.00
    3. VS0.99
    4. CHOREOGRAPHY1.00
    5. ORCHESTRATION1.00
    6. Decentralized. Event-driven. Autonomous1.00
    7. Centralized. Coordinated. Controlled0.98

  • 6:43 #14 done13 line(s)

    shot 14·sharpness 1233.2

    1. CHOREOGRAPHY1.00
    2. EVENT-DRIVEN COORDINATION1.00
    3. Research1.00
    4. Analysis1.00
    5. Report1.00
    6. Agent1.00
    7. Agent1.00
    8. Agent1.00
    9. Publish 'research_completed'0.99
    10. Subscribe & Consume0.99
    11. Publish'analysis_ready'1.00
    12. Subscribe & Consume1.00
    13. Message Bus0.96

  • 7:05 #15 skipped

    shot 15·duplicate of #14

  • 7:39 #16 skipped

    shot 16·duplicate of #14

  • 8:09 #17 done23 line(s)

    shot 17·sharpness 1190.3

    1. CHOREOGRAPHY1.00
    2. Complex1.00
    3. Dependencies1.00
    4. WHEN TO USE AND AVOID0.99
    5. Need1.00
    6. Loosely1.00
    7. High Agent1.00
    8. Centralized1.00
    9. Coupled1.00
    10. Autonomy1.00
    11. Rollback1.00
    12. Workflows1.00
    13. needed1.00
    14. Weak1.00
    15. Observability1.00
    16. Frequently1.00
    17. Strong1.00
    18. Added Agents1.00
    19. Observability1.00
    20. Infrastructure1.00
    21. Debugging is1.00
    22. TE0.70
    23. Difficult1.00

  • 8:35 #18 done12 line(s)

    shot 18·sharpness 1124.2

    1. ORCHESTRATION1.00
    2. CENTRALIZED COORDINATION1.00
    3. Step 10.95
    4. Agent A1.00
    5. Agent B1.00
    6. Step 2 (parallel)1.00
    7. Orchestrator1.00
    8. Step 2 (parallel)1.00
    9. Agent C1.00
    10. 601lf0.52
    11. Step 30.94
    12. Agent D0.99

  • 9:21 #19 done11 line(s)

    shot 19·sharpness 1129.0

    1. ORCHESTRATION1.00
    2. CENTRALIZED COORDINATION1.00
    3. Step 10.94
    4. Agent A0.99
    5. Agent B1.00
    6. Step 2 (parallel)0.98
    7. Orchestrator1.00
    8. Step 2 (parallel)1.00
    9. Agent C1.00
    10. Step 30.94
    11. Agent D0.99

  • 9:37 #20 done23 line(s)

    shot 20·sharpness 1171.7

    1. ORCHESTRATION1.00
    2. High1.00
    3. Autonomy1.00
    4. WHEN TO USE AND AVOID1.00
    5. Required1.00
    6. Can't Tolerate0.97
    7. Complex1.00
    8. Rollback1.00
    9. Bottleneck1.00
    10. Dependencies1.00
    11. Capability1.00
    12. Required0.99
    13. Constantly1.00
    14. Changing1.00
    15. Need1.00
    16. Stable1.00
    17. Workflows1.00
    18. Centralized1.00
    19. Workflows1.00
    20. Control1.00
    21. Need1.00
    22. Distributed1.00
    23. Scaling1.00

  • 10:20 #21 done19 line(s)

    shot 21·sharpness 831.9

    1. DECISION1.00
    2. MATRIX1.00
    3. HH0.61
    4. CHOREOGRAPHY1.00
    5. HYBRID1.00
    6. Event-driven1.00
    7. Distributed1.00
    8. microservices1.00
    9. transactions1.00
    10. Autmy0.77
    11. →MO70.84
    12. SIMPLE1.00
    13. FULL ORCHESTRATION0.99
    14. ORCHESTRATION1.00
    15. Linear Workflows0.98
    16. Enterprise Workflows1.00
    17. SIMPLE1.00
    18. Workflow complexity1.00
    19. COMPLEX1.00

  • 11:04 #22 skipped

    shot 22·duplicate of #21

  • 11:32 #23 done4 line(s)

    shot 23·sharpness 528.9

    1. Q0.94
    2. Q0.96
    3. STATE1.00
    4. MANAGEMENT1.00

Transcript

252 cues· 4,013 words· 23,681 chars

  1. 0:00 Hi everyone, I'm Sandy.
  2. 0:01 I've spent 18 years building data systems, a major part of it focusing on building and scaling distributed data systems in the cloud.
  3. 0:09 I've done it for multi-tenant systems for software and SaaS companies, and then for scaling data and AI platforms in regulated industries like financial services and healthcare.
  4. 0:19 I've learned a great deal about production grade distributed systems while I have been working at AWS and now in Databricks.
  5. 0:26 For the last two years, I've been deploying multi-agent AI systems in production and I have watched brilliant engineers make the same mistakes over and over.
  6. 0:36 They think adding more agents is just like adding more features.
  7. 0:40 It's not.
  8. 0:41 It's building a distributed system.
  9. 0:43 And today I'm going to show you the patterns that actually work when you make that transition.
  10. 0:48 These are lessons that I have learned working in the trenches.
  11. 0:51 And today I'm here to share it with you.
  12. 0:53 Here's what we are covering today.
  13. 0:55 First, the problem.
  14. 0:56 I'll share you a very basic production war story about race conditions and why complexity explodes when you go from one agent to five agents.
  15. 1:08 I'll talk about the patterns, choreography and orchestration patterns for coordination of agents.
  16. 1:13 I'll talk about state management, talk about failure recovery and how we can design for failure in production systems.
  17. 1:21 And then I'll share how a production grade architecture would look like in as simple way possible.
  18. 1:28 And I'll also show you an example on how we build this on Databricks.
  19. 1:31 So let's dive into it.
  20. 1:32 You see, one agent works beautifully.
  21. 1:35 You have got your LLM, some prompts, maybe a retrieval augmented generation pipeline, maybe some tool calls.
  22. 1:43 It demos great.
  23. 1:45 Leadership loves it.
  24. 1:46 You feel happy and your team is happy.
  25. 1:49 And then product comes back with a request that changes everything.
  26. 1:54 They want five more agents.
  27. 1:55 And here's what happens.
  28. 1:57 You think, okay, I know how to build agents and I will add five more.
  29. 2:01 Except now you have coordination problems.
  30. 2:04 Agent A produces data that agent B needs.
  31. 2:08 Agent C is waiting on both agent A and agent B.
  32. 2:11 agent b just updated the shared state that agent b was reading and agent e just crashed and took down this entire workflow this is no longer an ai problem this is a distributed system problem and most of you didn't sign up to be distributed systems engineer
  33. 2:30 let me tell you about a production deployment where this went very wrong we built a credit decisioning system for a financial services company the first agent credit score calculation worked perfectly it worked great in demos two weeks in production zero issues then we added four more agents income verification risk assessment fraud detection and final approval
  34. 2:52 We deployed all five.
  35. 2:53 In three days time, we started seeing weird approvals.
  36. 2:57 20% of the decisions had incorrect risk ratings.
  37. 3:01 Customers who should have been flagged were getting approved.
  38. 3:04 The business team was panicking.
  39. 3:06 It took us two days to find out what was happening.
  40. 3:09 One credit score agent calculated a score of 750 and wrote to the database.
  41. 3:14 The risk assessment agent, on the other hand, read from the database 500 milliseconds later and got a score of 680 for the same customer.
  42. 3:24 Why did it happen?
  43. 3:25 Because we had a caching layer for customer records.
  44. 3:28 The write to PostgreSQL succeeded, but the cache was not invalidated.
  45. 3:33 The risk agent read from the cache and it got stale data.
  46. 3:40 used it used the wrong score and made the wrong decision this is a classic distributive systems problem we had caching layer between the agents and the database cache invalidation failed and the agent was reading stale values the race condition wasn't in the database it was in the architecture multiple agents shared cache no coordination on cache invalidation
  47. 4:06 this took us quite a while to find the pattern it created delays in delivery and led to wrong decisions and here's the lesson we learned the problem was of course not with the model the problem wasn't with the prompts the problem was we built a distributed system without distributed system thinking and that's what kills multi-agent projects not bad ai but bad architecture
  48. 4:32 now i will show you the architecture that works we will also look into a production grade architecture but first let's understand why this complexity explodes so quickly now when you move from a one agent system to a multi-agent let's say five agent systems it doesn't get just five times harder it gets 25 times more complex coordination complexity grows exponentially one agent has got zero coordination problems
  49. 4:59 Two agents have got at least one connection.
  50. 5:01 Five agents have got at least 10 potential connections and coordination.

Open at this second