vidtheque.

read-only demo

Videos APh1Vx0oLmQ

Deterministic Infra for Non-Deterministic AI Agents - Nishant Gupta, Meta Superintelligence Labs

index_state ready data_status ok

AI Engineer· published 2026-06-29· 0:07:13· en-US· indexed 2026-08-11 05:40

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:28, 1 of 1 keyframes kept
  2. Shot 1, 0:28 to 0:56, 0 of 1 keyframes kept
  3. Shot 2, 0:56 to 1:28, 1 of 1 keyframes kept
  4. Shot 3, 1:28 to 2:04, 1 of 1 keyframes kept
  5. Shot 4, 2:04 to 2:31, 1 of 1 keyframes kept
  6. Shot 5, 2:31 to 3:06, 1 of 1 keyframes kept
  7. Shot 6, 3:06 to 3:28, 1 of 1 keyframes kept
  8. Shot 7, 3:28 to 3:55, 1 of 1 keyframes kept
  9. Shot 8, 3:55 to 4:21, 1 of 1 keyframes kept
  10. Shot 9, 4:21 to 4:49, 1 of 1 keyframes kept
  11. Shot 10, 4:49 to 5:08, 1 of 1 keyframes kept
  12. Shot 11, 5:08 to 5:33, 1 of 1 keyframes kept
  13. Shot 12, 5:33 to 6:02, 1 of 1 keyframes kept
  14. Shot 13, 6:02 to 6:26, 1 of 1 keyframes kept
  15. Shot 14, 6:26 to 6:47, 1 of 1 keyframes kept
  16. Shot 15, 6:47 to 7:11, 1 of 1 keyframes kept
  17. Shot 16, 7:11 to 7:13, 1 of 1 keyframes kept

17 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
115
whisperx 115
chunks
13
from 115 cues
keyframes
16
kept of 17 captured
frames with text
16
456 lines read
chapters
0
from the source metadata
keyframe bytes
2.9 MB
word timings on 115 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-11 05:38 1m 02s
stt done — 2026-08-11 05:39 8s
chunk done — 2026-08-11 05:39 0s
text_embed done — 2026-08-11 05:39 0s
keyframe done — 2026-08-11 05:39 18s
ocr done — 2026-08-11 05:40 7s
frame_embed done — 2026-08-11 05:40 3s

Frames, and what the machine read

  • 0:24 #0 done28 line(s)

    shot 0·sharpness 4098.4

    1. BUILDING DETERMINISTIC1.00
    2. INFRASTRUCTURE FOR0.99
    3. Nishant Gupta0.99
    4. DETERMINISTIC BOUNDARY [DB-S00]0.99
    5. NON-DETERMINISTIC1.00
    6. ENTROPY REDUCTION: 99.9%1.00
    7. PREDICTABILITY INDEX: 1.01.00
    8. SYSTEM VARIANCE: <0.01%0.99
    9. AIAGENTS1.00
    10. S0°0.82
    11. CONTROL ENCLOSURE [CE-99.99%]0.99
    12. THE EMERGING CONTROL PLANE1.00
    13. FOR AUTONOMOUS AI SYSTEMS0.99
    14. SIGNAL1.00
    15. RECTIFICATION1.00
    16. SIGNAL1.00
    17. SYNCHRONIZATION1.00
    18. STATE1.00
    19. SYNCHRONIZATION1.00
    20. 180.53
    21. 92-0.70
    22. 90°0.88
    23. RIGID FRAMEWORK1.00
    24. CONTROL ENCLOSURE1.00
    25. Nishant Gupta1.00
    26. [RF-AIZ]0.96
    27. [CE-99.99%]1.00
    28. Tech Lead @ Meta1.00

  • 0:42 #1 skipped

    shot 1·duplicate of #0

  • 1:24 #2 done29 line(s)

    shot 2·sharpness 4717.2

    1. Infrastructure was built for0.98
    2. Nishant Gupta1.00
    3. predictable microservices.1.00
    4. THE GREAT MISMATCH1.00
    5. Traditional1.00
    6. Autonomous1.00
    7. FRICTION POINTS &0.98
    8. Microservices1.00
    9. INCOMPATIBILITY ZONES0.98
    10. Al Agents0.96
    11. Stateless1.00
    12. →0.97
    13. Stateful1.00
    14. [STATE:0]0.99
    15. →0.89
    16. [STATE:PERSIST]0.99
    17. Deterministic1.00
    18. Probabilistic1.00
    19. [PATH:FIXED]0.98
    20. X0.87
    21. [PATH: DYNAMIC]0.96
    22. Request-response1.00
    23. Multi-step workflows1.00
    24. [FLOW: SYNC]0.96
    25. [FLOW:ASYNC]0.99
    26. Millisecond execution1.00
    27. Long-running1.00
    28. [TIME:<100MS]0.97
    29. [TIME: >MIN/HR]0.97

  • 1:56 #3 done17 line(s)

    shot 3·sharpness 2809.1

    1. Demos optimize for capability.1.00
    2. Nishant Gupta1.00
    3. Production demands reliability.1.00
    4. [STRUCTURAL INTEGRITY: CRITICAL]0.99
    5. Capabilities0.99
    6. Prompt0.94
    7. LM0.97
    8. 50.210.94
    9. Reliability1.00
    10. KISO0.88
    11. Core1.00
    12. [L0Y38 0N]0.60
    13. Reliability Core0.99
    14. [LOAD BEARING]0.99
    15. [LOAD BEARING]0.99
    16. [STRUCTURAL INTEGRITY: CRITICAL]0.99
    17. [ISOMETRIC VIEW: SYSTEM CROSS-SECTION]0.99

  • 2:25 #4 done34 line(s)

    shot 4·sharpness 3687.1

    1. Real production failures originate1.00
    2. the infrastructure, not the model.1.00
    3. Diagnostic Failure Tree0.99
    4. 1091.00
    5. Logic1.00
    6. Recursive1.00
    7. Workflow1.00
    8. 0.130.93
    9. reasoning loops0.99
    10. deadlocks1.00
    11. DCCC0.73
    12. 0330.84
    13. 1501.00
    14. 1001.00
    15. Stochastic1.00
    16. Action1.00
    17. Tool1.00
    18. Retry1.00
    19. Cost1.00
    20. Model Output1.00
    21. 0.30.92
    22. hallucinations1.00
    23. F391.00
    24. storms1.00
    25. explosions1.00
    26. 5551.00
    27. 1590.95
    28. State1.00
    29. Context1.00
    30. Memory1.00
    31. 0.JS0.71
    32. drift1.00
    33. poisoning1.00
    34. 5340.86

  • 3:01 #5 done69 line(s)

    shot 5·sharpness 3043.2

    1. The anatomy of an1.00
    2. [CRIT1.00
    3. Nishant Gupta0.99
    4. Exponential GPU0.98
    5. +85S%0.91
    6. agent retry storm.1.00
    7. Compute Spike0.98
    8. SPIKE1.00
    9. +4338%1.00
    10. SATURATION1.00
    11. RESOURCE1.00
    12. Step 4:1.00
    13. +5559%1.00
    14. COST1.00
    15. Recursive1.00
    16. EXPLOSION1.00
    17. reasoning loop1.00
    18. 4780.99
    19. 1001.00
    20. 5551.00
    21. locks up.1.00
    22. hallucinates1.00
    23. invalid API1.00
    24. parameters.1.00
    25. Step 1: Agent1.00
    26. Step 2:0.95
    27. Tool returns1.00
    28. error code.1.00
    29. attempts to fix0.98
    30. Step 3: Agent0.98
    31. hallucinates new1.00
    32. error but1.00
    33. RECURSION1.00
    34. LOOP1.00
    35. * INVALID0.94
    36. +0.13#0.83
    37. PARAMS1.00
    38. GPU c L OPda)0.52
    39. Step 5:0.96
    40. The1.00
    41. Consequence1.00
    42. invalid parameter.1.00
    43. 01'00.85
    44. 1601.00
    45. 1001.00
    46. Step 1: Agent1.00
    47. Step 3: Agent0.99
    48. Step 4:0.99
    49. hallucinates1.00
    50. invalid API1.00
    51. Step 2:1.00
    52. Tool returns1.00
    53. attempts to fix0.97
    54. error but0.99
    55. reasoning loop0.98
    56. Recursive1.00
    57. Step 5:1.00
    58. The1.00
    59. parameters.1.00
    60. error code.1.00
    61. hallucinates new1.00
    62. locks up.1.00
    63. Time (ms)1.00
    64. Consequence1.00
    65. invalid parameter.1.00
    66. 5180.89
    67. DATA SPIKE EVENT1.00
    68. [CRITICAL FAILURE PATH]0.99
    69. [TIMELINE AXIS0.98

  • 3:25 #6 done19 line(s)

    shot 6·sharpness 3774.9

    1. The platform decides.1.00
    2. Nishant Gupta0.99
    3. The model merely proposes.1.00
    4. Validate1.00
    5. Validate1.00
    6. Aligned1.00
    7. Output1.00
    8. Stochastic1.00
    9. Safe1.00
    10. Core1.00
    11. Execution1.00
    12. System1.00
    13. Compliance1.00
    14. Deterministic1.00
    15. Filter1.00
    16. Wrapper1.00
    17. Route1.00
    18. [ANALYSIS: STRUCTURAL INTEGRITY]1.00
    19. [FLOW: DETERMINISTIC CONTROL0.99

  • 3:39 #7 done41 line(s)

    shot 7·sharpness 4849.2

    1. The Agent Control Plane1.00
    2. ARCHITEC1.00
    3. Nishant Gupta1.00
    4. is the new infrastructure layer.1.00
    5. Agents require1.00
    6. an operating0.98
    7. system.1.00
    8. [Contzol Plane Blueprint]0.99
    9. Agent1.00
    10. Applications1.00
    11. Just as Kubernetes1.00
    12. Research1.00
    13. Ceding0.95
    14. Interaction1.00
    15. Passeosers1.00
    16. became the1.00
    17. control plane for1.00
    18. Memory Coordinator1.00
    19. Orchestration Engine1.00
    20. orchestrating1.00
    21. INDEX0.97
    22. containers, a1.00
    23. dedicated Agent1.00
    24. THE AGENT0.98
    25. Control Plane is0.99
    26. CONTROL PLANE0.97
    27. Safety Policy Node0.98
    28. Compute Scheduler1.00
    29. emerging to1.00
    30. !A!!!0.64
    31. govern the runtime1.00
    32. execution of1.00
    33. 区0.73
    34. VALIDATE & FILTER1.00
    35. RESOURCE OPTIKIZKTION0.97
    36. autonomous Al.1.00
    37. [ANALYSIIS: HIGH]0.99
    38. Foundational LLMs & Data APls0.99
    39. [COST 0ARAP: 10%]0.95
    40. 31.00
    41. [FLOW: DETERMINISTIC GOVERNANCE]0.99

  • 4:18 #8 done76 line(s)

    shot 8·sharpness 3398.2

    1. Logs are dead. Autonomous workflows1.00
    2. require multidimensional observability.1.00
    3. Nishant Gupta1.00
    4. Agent Trace Timeline0.98
    5. [Telemetry Sidebar]1.00
    6. T+0.1s0.92
    7. T+0.2s0.99
    8. T+8.1s1.00
    9. T+0.5s0.99
    10. T+1.0s1.00
    11. T+1.2s1.00
    12. [FLON: REASONING]0.99
    13. [FLON: REASONING]0.99
    14. LLM Decisions0.99
    15. Track 1:1.00
    16. Call Tool A1.00
    17. Decision:1.00
    18. Call Tool A0.99
    19. Decision:1.00
    20. Query Memory1.00
    21. Decision:1.00
    22. Query Memory1.00
    23. Decision:1.00
    24. Call Tool A1.00
    25. Decision:1.00
    26. [DATA: ACTIVE]0.99
    27. Track 2:1.00
    28. Plan:0.90
    29. Plan:1.00
    30. Plan:0.96
    31. Plans1.00
    32. Orchestration1.00
    33. Execute0.99
    34. Step 10.96
    35. Execute1.00
    36. Step 10.98
    37. Validate1.00
    38. Output1.00
    39. [FLOR: DRXTING]0.95
    40. Track 3:1.00
    41. Tool Calls &1.00
    42. Execution1.00
    43. [DATA: ACTIVE]0.99
    44. Tool A Executing1.00
    45. API Response1.00
    46. Received1.00
    47. [FLOW: REASONING]1.00
    48. [BATA: ACTIVE]0.96
    49. Track 4:1.00
    50. Read:1.00
    51. Write:1.00
    52. Memory Access1.00
    53. Context Block X0.99
    54. New State Y1.00
    55. [FLOW: REASONING]0.98
    56. [DATA: ACTIVE]0.97
    57. [04TA: ACTIVE]0.98
    58. Track 5:0.99
    59. State Transitions1.00
    60. Processing1.00
    61. State:1.00
    62. Waiting for Tool1.00
    63. State:1.00
    64. Updated1.00
    65. State:1.00
    66. Updated1.00
    67. State:1.00
    68. T+8.1s0.95
    69. T+0.1s0.95
    70. event1.00
    71. T+0.5s0.96
    72. T+1.0s0.98
    73. T+1.0s1.00
    74. 41.00
    75. [ANALYSIS: FLOW TRACING]1.00
    76. [DATA: CONTINUOUS STATE]0.99

  • 4:43 #9 done12 line(s)

    shot 9·sharpness 2991.8

    1. Shared memory coordinates1.00
    2. Nishant Gupta0.99
    3. chaotic multi-agent consistency1.00
    4. Shared Memory Architecture1.00
    5. Agent A1.00
    6. Agent B0.97
    7. State Store1.00
    8. [-25-3.00, 00, 5291]0.97
    9. Stale memory0.97
    10. Context drift1.00
    11. Agent C0.97
    12. Conflicting facts1.00

  • 4:58 #10 done10 line(s)

    shot 10·sharpness 4084.8

    1. Autonomous systems demand0.99
    2. Nishant Gupta1.00
    3. layered containment boundaries1.00
    4. Ring 5: Audit Layer0.99
    5. Ring 2: Tool Permissions0.99
    6. Ring 3: Policy Engine1.00
    7. BLOCKED1.00
    8. Model Output1.00
    9. Ring 4: Human Approval1.00
    10. Ring 5: Audit Layer0.99

  • 5:30 #11 done20 line(s)

    shot 11·sharpness 3158.2

    1. Human oversight is an escalation path,1.00
    2. not an operational bottleneck.1.00
    3. Workflow Routing Diagram1.00
    4. Decision Gate1.00
    5. Risk / Confidence Check0.97
    6. Confidence Score: >95X1.00
    7. Anomaly Detection: Low1.00
    8. Automated Agent Tasks1.00
    9. Automated Agent Tasks1.00
    10. 45°1.00
    11. 45°1.00
    12. Confidence Score: >95%0.98
    13. Anonaly Detection: Low0.99
    14. Edge Case1.00
    15. Approved State /0.97
    16. Routing1.00
    17. Resolution1.00
    18. Human Approval Node1.00
    19. Manual Review Required0.99
    20. [ANALYSIS: ESCALATION PATHWAYS]0.98

  • 5:45 #12 done18 line(s)

    shot 12·sharpness 2758.2

    1. Inference at scale fundamentally1.00
    2. becomes a cluster scheduling problem.1.00
    3. Elastic Scaling Graph1.00
    4. 3001.00
    5. Variable reasoning1.00
    6. depth1.00
    7. Long-running1.00
    8. workflows1.00
    9. 2001.00
    10. COME UMUUITS0.72
    11. Resource Contention1.00
    12. & GPU Limits1.00
    13. 1001.00
    14. Bursty Demand1.00
    15. 1001.00
    16. 2001.00
    17. 3001.00
    18. TIME (sec)0.96

  • 6:14 #13 done27 line(s)

    shot 13·sharpness 3343.7

    1. The Reliability Rosetta Stone.1.00
    2. 830.85
    3. Distributed Systems Pattern1.00
    4. Agent Equivalent1.00
    5. Circuit Breakers1.00
    6. Tool Isolation0.97
    7. [PATTERN: ISOLATION]0.98
    8. [PATTERN: ISOLATION]0.98
    9. Rate Limiting1.00
    10. Agent Limits0.96
    11. [LINIT_ENFORCEMENT]1.00
    12. [LINIT_ENFORCEMENT]1.00
    13. Retries1.00
    14. Controlled Recovery1.00
    15. [RECOVERY_LOGIC]1.00
    16. [RECOVERY_LOGIC]1.00
    17. Quotas1.00
    18. Cost Governance1.00
    19. [RESOURCE_CAPS]1.00
    20. [RESOURCE_CAPS]1.00
    21. Observability1.00
    22. Agent Tracing1.00
    23. [TRACEABILITY_SYSTEM]1.00
    24. [TRACEABILITY_SYSTEM]1.00
    25. 1250.81
    26. 1251.00
    27. [ANALYSIS: RELIABILITY MAPPING]0.99

  • 6:40 #14 done18 line(s)

    shot 14·sharpness 2509.5

    1. 中0.51
    2. Competitive advantage has shifted fron0.99
    3. prompt engineering to systems engineering.1.00
    4. THE PARADIGM SHIFT0.99
    5. PROMPTS1.00
    6. Compound Growth1.00
    7. Trajectory1.00
    8. MODELS1.00
    9. Commoditization1.00
    10. Piateau1.00
    11. =>=>>0.63
    12. Commoditization1.00
    13. Plateau1.00
    14. INFRASTRUCTURE1.00
    15. 2020.75
    16. 20231.00
    17. 20241.00
    18. 2025+1.00

  • 7:01 #15 done36 line(s)

    shot 15·sharpness 3873.2

    1. Al agents are distributed systems.1.00
    2. FINAL DIREC0.99
    3. Nishant Gupta1.00
    4. Treat them accordingly.1.00
    5. The future1.00
    6. of Al won't1.00
    7. ..80.54
    8. be won by1.00
    9. CORE TAKEAWAYS0.99
    10. better1.00
    11. . Models are inherently stochastic;0.99
    12. prompts.1.00
    13. Control Plane1.00
    14. infrastructure must be strictly.1.00
    15. Blueprint1.00
    16. Poltoy Engines0.94
    17. Agest Execution0.93
    18. Reliability is no longer a model1.00
    19. It will be0.99
    20. .9.860.76
    21. problem—it is an infrastructure.0.99
    22. ececution0.94
    23. Agant1.00
    24. won by1.00
    25. Multidimensional observability is1.00
    26. mandatory, not optional.0.99
    27. better1.00
    28. ongite0.87
    29. State0.99
    30. • Agent control planes are the0.97
    31. systems.0.99
    32. Stute0.89
    33. required runtime layer for1.00
    34. production Al.1.00
    35. 2020.67
    36. 125°0.91

  • 7:13 #16 done2 line(s)

    shot 16·sharpness 344.6

    1. Nishant Gupta1.00
    2. THANK YOU0.98

Transcript

115 cues· 1,025 words· 6,976 chars

  1. 0:03 Hey, everyone.
  2. 0:04 My name is Dishant Gupta.
  3. 0:06 I'm a software engineering tech leader at Meta, working on building the training and infrastructure.
  4. 0:12 And today, we're going to be talking about building deterministic infrastructure for non-deterministic AI agents.
  5. 0:19 So most of the conversations around AI over the last few years has been focused on models.
  6. 0:23 Bigger models, more parameters, better reasoning.
  7. 0:26 But as organizations move from chatbots to autonomous agents, a different problem emerges.
  8. 0:31 The challenge is no longer intelligence.
  9. 0:33 The challenge is reliability.
  10. 0:36 At Meta and across the industry, we are seeing agents move beyond answering questions and beginning to plan tool calls, coordinate workflows, and make decisions that affect production systems.
  11. 0:46 These systems are fundamentally probabilistic.
  12. 0:50 Infrastructure is not allowed to be.
  13. 0:52 Today, I want to discuss this topic in more detail.
  14. 0:58 The modern cloud infrastructure evolved around a set of assumptions.
  15. 1:01 Most of the requests are short-lived.
  16. 1:03 Services are deterministic, more or less.
  17. 1:06 Execution paths are known.
  18. 1:08 Failures are bounded.
  19. 1:10 However, autonomous AI agents violate nearly every one of those assumptions.
  20. 1:14 They are stateful.
  21. 1:14 They are long-running.
  22. 1:15 They make decisions dynamically.
  23. 1:17 They may execute different workflows for the same inputs.
  24. 1:21 This is what I call the great mismatch.
  25. 1:23 We're trying to run autonomous systems on infrastructure that was designed for deterministic workflows.
  26. 1:30 This is probably the most important mind shift.
  27. 1:33 Most AI demos showcase capability.
  28. 1:36 But can it solve a problem?
  29. 1:37 Can it use a tool?
  30. 1:38 Can it complete a workflow?
  31. 1:40 Production systems have a different objective.
  32. 1:43 Can it do it reliably?
  33. 1:45 Can it do it 10,000 times, 100,000 times, million times?
  34. 1:49 Can it recover from failures?
  35. 1:50 Can it operate safely?
  36. 1:51 Can it do it at an acceptable cost with an acceptable latency, with an acceptable outcome?
  37. 1:57 The majority of the engineering effort moves below the model layer into orchestration, monitoring, safety, evaluation, and recovery systems.
  38. 2:06 When people hear AI failures, they immediately think hallucinations.
  39. 2:10 In reality, hallucinations are often the least interesting for their mood.
  40. 2:14 What we see instead are infrastructure failures, recursive reasoning loops, over-floated logs, retry amplification, context corruption, memory poisoning, cost explosions.
  41. 2:25 The model makes mistakes, but however the infrastructure turns that mistake into an outage.
  42. 2:29 That's the real challenge.
  43. 2:33 So as this slide shows a pattern that distributed system engineers will probably recognize immediately.
  44. 2:38 An agent calls a tool incorrectly.
  45. 2:40 The tool returns an error.
  46. 2:42 Instead of recovering, the agent generates a slightly different but still invalid request.
  47. 2:48 The cycle repeats.
  48. 2:49 Each retry consumes more compute, reasoning depth increases, GPU consumption rises, eventually you get exponential resource growth.
  49. 2:57 What started as a minor API error became a compute incident.
  50. 3:01 This is why uncontrolled retries are one of the biggest risks in agentic systems.

Open at this second