Learned Execution Graphs for Anomaly Detection & Drift in APIs — Ritvik Pandya, JP Morgan Chase

What was stored

cues

164

whisperx 164

chunks

33

from 164 cues

keyframes

24

kept of 45 captured

frames with text

24

643 lines read

chapters

11

from the source metadata

keyframe bytes

5.3 MB

word timings on 164 cues

Provenance

Frames, and what the machine read

• 

  0:02 #0 done2 line(s)

  shot 0·sharpness 453.4

  1. AlEngineer0.96
  2. World's Fair0.97
• 

  0:03 #1 done2 line(s)

  shot 1·sharpness 665.7

  1. AlEngineer0.95
  2. World's Fair0.99
• 

  0:10 #2 done24 line(s)

  shot 2·sharpness 2743.2

  1. LAB & PLATINUM SPONSORS0.98
  2. Amazon AGI Lab0.98
  3. ANTHROP\C1.00
  4. Google DeepMind1.00
  5. MINIMAX0.94
  6. OpenAI0.92
  7. Akamai1.00
  8. arize1.00
  9. aws1.00
  10. Braintrust bright data0.99
  11. B1.00
  12. Browserbase1.00
  13. docker1.00
  14. :neo4j0.93
  15. ORACLE1.00
  16. PayPal1.00
  17. qodo1.00
  18. reducto1.00
  19. Sonar1.00
  20. Makers of0.99
  21. togetherai1.00
  22. Unblocked1.00
  23. WorkOS1.00
  24. SonarQube1.00
• 

  0:49 #3 done19 line(s)

  shot 3·sharpness 1979.7

  1. AlEngineer0.97
  2. World'sFair0.98
  3. AI ENGINEER WORLD'S FAIR 20261.00
  4. GRAPHS TRACK1.00
  5. Learned Execution Graphs1.00
  6. PRESENTED BY1.00
  7. Microsoft1.00
  8. for Real-Time Anomaly Detection &1.00
  9. Drift Classification in APls0.99
  10. Turn every request's trace into an attributed execution DAG. Learn the distribution of normal graphs. Score0.99
  11. anomalies and classify drift — in near real time, at production throughput.1.00
  12. Ritvik Pandya1.00
  13. Engineering Lead · J.P. Morgan Chase - CIB Payments0.98
  14. Views are my own. Examples & figures generalized.1.00
  15. LEARNED EXECUTION GRAPHS0.97
  16. AIE WORLD'S FAIR 26 GRAPHS TRACK0.97
  17. 01/121.00
  18. Engineering the future of Al0.99
  19. World'sFair1.00
• 

  1:16 #4 done29 line(s)

  shot 4·sharpness 2899.9

  1. AlEngineer0.99
  2. World'sFair1.00
  3. 011.00
  4. FRAMING1.00
  5. A different kind of graph1.00
  6. PRESENTED BY1.00
  7. WHAT THE TRACK MEANS BY"GRAPH"0.99
  8. THIS TALK1.00
  9. Persistent / property graph0.99
  10. Execution graph (a trace)1.00
  11. Microsoft1.00
  12. Knowledge graphs, GraphRAG, entity links0.98
  13. spans1.00
  14. One execution DAG per request, built from its1.00
  15. Lives in a database (Neo4j, ..)0.97
  16. Lives in the request path for milliseconds0.99
  17. Queried for what is related to what0.99
  18. Describes how this call actually ran1.00
  19. Edges are facts; the graph is the data1.00
  20. Edges are causality; the graph is behavior1.00
  21. Same word, opposite lifecycle. One is a store you query. The other is a signal you have ~40 ms to read before it's gone.0.99
  22. Why the graph, not just metrics? For "is it slow?" you don't need it — but where, cause vs. symptom, and what kind of change are structural,0.99
  23. and a number can't answer them.1.00
  24. LEARNED EXECUTION GRAPHS1.00
  25. AIE WORLD'S FAIR '26 GRAPHS TRACK0.97
  26. 02 / 120.86
  27. TRACK 5·JULY 2,20260.96
  28. Graphs1.00
  29. World's Fair0.96
• 

  1:50 #5 done35 line(s)

  shot 5·sharpness 1817.6

  1. AlEngineer0.99
  2. World'sFair1.00
  3. 021.00
  4. DEFINITION1.00
  5. From span tree to execution DAG1.00
  6. OpenTelemetry gives you a span tree — one parent per span. We enrich it into an execution DAG: add span links, async0.99
  7. producerconsumer edges, shared-resource and join causality. ("Notify" re-converges via a constructed join edge.)0.99
  8. fan-out1.00
  9. join edge1.00
  10. Fraud Score1.00
  11. ml-svc1.00
  12. Edge GW1.00
  13. AuthN/z0.94
  14. Orchestrator1.00
  15. Ledger Write0.99
  16. Notify1.00
  17. ingress1.00
  18. token-svc1.00
  19. payments-core1.00
  20. cockroachdb1.00
  21. response1.00
  22. FX Rate1.00
  23. ext-api1.00
  24. G = (V, E)0.96
  25. V = spans (operations)0.99
  26. E = parent-child + links + async + join1.00
  27. node feats: svc,1.00
  28. op, status, duration1.00
  29. edge feats: sync/async, lag0.99
  30. LEARNED EXECUTION GRAPHS1.00
  31. AIE WORLD'S FAIR26GRAPHS TRACK0.97
  32. 03-/-120.97
  33. TRACK 5· JULY 2,20260.97
  34. Graphs1.00
  35. World'sFair1.00
• 

  2:36 #6 done30 line(s)

  shot 6·sharpness 2062.7

  1. AlEngineer0.99
  2. World's Fair0.97
  3. 031.00
  4. WHY A DAG1.00
  5. Acyclicity is load-bearing, not cosmetic0.99
  6. Topological order0.99
  7. Causal direction0.99
  8. Ordered propagation1.00
  9. A DAG has a valid linearization → you0.99
  10. Edges point parentchild. That lets the1.00
  11. Process spans in topological order -0.98
  12. can compute the critical path and1.00
  13. detector separate a root cause1.00
  14. each node aggregates its full upstream1.00
  15. attribute end-to-end latency to the0.99
  16. (upstream) from its symptoms1.00
  17. context in one causal pass, instead of0.99
  18. exact span that owns it.0.99
  19. (everything downstream of it).1.00
  20. iterating rounds and over-smoothing the1.00
  21. whole graph.1.00
  22. THE HONEST CAVEAT - RETRIES & LOOPS0.98
  23. Retries, polling, and sagas can introduce cycles. Restore the DAG before modeling: unroll repeats into distinct nodes (call#1, call#2) or1.00
  24. type the edge as ret ry/as ync. A retry storm then shows up as a structural feature — exactly the signal you want.0.98
  25. LEARNED EXECUTION GRAPHS1.00
  26. AIE WORLD'S FAIR'26 GRAPHS TRACK0.99
  27. 04 / 120.89
  28. TRACK 5·JULY 2,20260.97
  29. Graphs1.00
  30. World'sFair1.00
• 

  2:59 #7 skipped

  shot 7·duplicate of #6

• 

  3:30 #8 done43 line(s)

  shot 8·sharpness 2283.4

  1. AlEngineer0.98
  2. World'sFair1.00
  3. 041.00
  4. THE MODEL1.00
  5. "Learned" = model the distribution, not rules0.99
  6. Hand-written thresholds break on every deploy. Instead, learn per-node baselines and the graph's normal structure from1.00
  7. telemetry — no labels, nothing to train. Three statistical layers, cheapest first:0.99
  8. TIER1.00
  9. METHOD1.00
  10. LEARNS1.00
  11. CATCHES1.00
  12. GRANULAR1.00
  13. COST1.00
  14. ROLE1.00
  15. Tier 00.98
  16. graph-hash + timing baseline0.97
  17. topology + latency normal1.00
  18. rare / off-timing shape0.97
  19. √ by signature -us0.95
  20. gate1.00
  21. Tier 10.93
  22. deviation ratios + structural compare0.99
  23. per-node baselines + topology0.98
  24. change1.00
  25. slow node + structural1.00
  26. √ per node0.97
  27. <1ms1.00
  28. attribute1.00
  29. Tier 20.99
  30. per-client EMA + KL divergence0.99
  31. client graph profiles1.00
  32. behavioral drift + cause1.00
  33. √ per client0.94
  34. ~ms0.99
  35. classify1.00
  36. All three run in microseconds to milliseconds — no GPU, no inference, no black box. (A learned graph autoencoder is a natural1.00
  37. extension; for payments, statistics wins on latency and interpretability.)1.00
  38. LEARNED EXECUTION GRAPHS1.00
  39. AIE WORLD'S FAIR'26 GRAPHS TRACK0.98
  40. 05 / 120.89
  41. TRACK 5·JULY 2,20260.99
  42. Graphs1.00
  43. World's Fair0.98
• 

  3:59 #9 skipped

  shot 9·duplicate of #8

• 

  4:42 #10 skipped

  shot 10·duplicate of #8

• 

  4:48 #11 skipped

  shot 11·duplicate of #8

• 

  5:17 #12 skipped

  shot 12·duplicate of #8

• 

  5:26 #13 done39 line(s)

  shot 13·sharpness 2207.6

  1. AlEngineer0.97
  2. World's Fair0.98
  3. 051.00
  4. THE APPROACH1.00
  5. From graph to localized cause1.00
  6. No model to train — learn per-node baselines from normal traffic. A request's deviation from them is the score; the worst node1.00
  7. is the cause.1.00
  8. PRESENTED BY1.00
  9. Represent1.00
  10. Baseline1.00
  11. Deviate1.00
  12. Localize0.95
  13. Threshold1.00
  14. trace → DAG0.96
  15. per-node stats1.00
  16. obs / baseline1.00
  17. argmax ratio1.00
  18. FP/day budget0.97
  19. Microsoft1.00
  20. THE BASELINES - learned, not trained0.99
  21. TWO CHECKS ON ONE GRAPH0.99
  22. Per-node statistical baselines — latency (μ, σ, p99), dependency set,0.99
  23. execution frequency. Mined from normal traffic. No labels, no neural1.00
  24. deviation ratio → bottleneck node attribute1.00
  25. net.1.00
  26. baseline(n) = {latency u/o, deps, freq)0.96
  27. structure check →missing/ reordered /new structural0.97
  28. deviation(n) = observed / baseline - score0.97
  29. cause = argmax(deviation)- localize0.97
  30. or a mandatory one that's gone.0.99
  31. One finds a node that's slow; the other finds a node that shouldn't be there -0.98
  32. Threshold = a false-positives-per-day budget, not a latency guess. Baselines recalibrate as normal re-learns — gated by the drift classifier and0.99
  33. the deploy log, so an expected change doesn't page.1.00
  34. LEARNED EXECUTION GRAPHS1.00
  35. AIE WORLD'S FAIR '26 · GRAPHS TRACK0.96
  36. 06 / 120.93
  37. TRACK 5· JULY 2, 20260.93
  38. Graphs1.00
  39. World's Fair0.99
• 

  6:18 #14 done42 line(s)

  shot 14·sharpness 2123.8

  1. AlEngineer0.98
  2. World's Fair0.97
  3. 061.00
  4. ANOMALY DETECTION0.97
  5. Score the graph, then localize the span0.98
  6. The same payment DAG — now the external FX dependency degrades. Each node's observed latency over its baseline: the offending0.99
  7. span stands an order of magnitude above normal while the rest sit at ~1×.0.99
  8. PRESENTED BY0.98
  9. Fraud Score1.00
  10. Microsoft1.00
  11. ml-svc0.99
  12. Edge GW1.00
  13. AuthN/z0.94
  14. Orchestrator1.00
  15. Ledger Write1.00
  16. Notify1.00
  17. ingress1.00
  18. token-svc1.00
  19. payments-core1.00
  20. cockroachdb1.00
  21. response1.00
  22. FX Rate1.00
  23. ext-api1.00
  24. PER-NODE DEVIATION1.00
  25. (observed / baseline)1.00
  26. edge-gw 1.0×0.99
  27. auth 1.1×0.99
  28. orchestrator 1.0×1.00
  29. fraud 1.2×1.00
  30. ledger 1.1×0.99
  31. notify 1.0×1.00
  32. fx-rate 38×0.97
  33. fx-rate z-score > threshold0.97
  34. ANOMALY, localized to fx-rate1.00
  35. end-to-end p99 barely moved1.00
  36. LEARNED EXECUTION GRAPHS1.00
  37. AIE WORLD'S FAIR'260.96
  38. GRAPHS TRACK1.00
  39. 07 / 120.93
  40. TRACK 5·JULY 2,20260.96
  41. Graphs1.00
  42. World's Fair0.99
• 

  7:03 #15 done31 line(s)

  shot 15·sharpness 1874.4

  1. AlEngineer0.98
  2. World's Fair0.96
  3. 071.00
  4. EVIDENCE1.00
  5. One real experiment beats ten illustrations0.98
  6. ILLUSTRATIVE result shape - reproduce on the public benchmark, then replace with your measured run. No production0.99
  7. data needed.1.00
  8. SETUP1.00
  9. RESULTS1.00
  10. (report these)1.00
  11. Benchmark1.00
  12. OpenTelemetry + DeathStarBench0.99
  13. Detection 86% of injected incidents0.99
  14. Train on1.00
  15. ~1.9M normal traces- 7-day window0.98
  16. Deviation scoring0.4 ms median0.99
  17. Inject1.00
  18. FX latency· missing span retry storm0.97
  19. Localization top-1 83% of detected0.96
  20. False alerts 4 / service / day0.98
  21. post-deploy topology· traffic shift0.98
  22. CPU sat.1.00
  23. Structural + KL check < 2 ms (no GPU)0.96
  24. Illustrative: "On ~1.9M synthetic traces across 6 injected failure classes, per-node deviation flagged 86% of incidents at 0.4 ms median latency and localized the0.99
  25. responsible node in 83% of detected cases — no GPU, no inference step."0.98
  26. LEARNED EXECUTION GRAPHS1.00
  27. AIE WORLD'S FAIR'26 GRAPHS TRACK0.98
  28. 08 / 120.88
  29. TRACK 5·JULY 2,20260.99
  30. Graphs1.00
  31. World's Fair0.96
• 

  7:40 #16 done17 line(s)

  shot 16·sharpness 1011.0

  1. AlEngineer0.97
  2. World'sFair1.00
  3. 08 · THE PIVOT0.93
  4. ANOMALY - a graph off the normal cloud1.00
  5. DRIFT - the normal cloud itself shifts1.00
  6. reference1.00
  7. current1.00
  8. instantaneous1.00
  9. a single request is wrong0.99
  10. over time "normal" has changed0.99
  11. An anomaly detector retrained on drift learns the drift as normal. You must detect the shift itself — and say what kind it is.0.99
  12. LEARNED EXECUTION GRAPHS1.00
  13. AIE WORLD'S FAIR '26 GRAPHS TRACK0.97
  14. 09/121.00
  15. TRACK 5· JULY 2,20260.96
  16. Graphs1.00
  17. World'sFair1.00
• 

  8:05 #17 done18 line(s)

  shot 17·sharpness 1009.1

  1. AlEngineer0.97
  2. World'sFair1.00
  3. 08· THE PIVOT0.96
  4. ANOMALY - a graph off the normal cloud0.99
  5. DRIFT - the normal cloud itself shifts1.00
  6. reference1.00
  7. current1.00
  8. instantaneous1.00
  9. a single request is wrong0.98
  10. over time "normal" has changed0.99
  11. An anomaly detector retrained on drift learns the drift as normal. You must detect the shift itself — and say what kind it is.0.99
  12. LEARNED EXECUTION GRAPHS1.00
  13. AIE WORLD'S FAIR 26 GRAPHS TRACK0.97
  14. 09/121.00
  15. TRACK 5· JULY 2,20260.96
  16. AlEngine0.93
  17. Graphs1.00
  18. World'sFair1.00
• 

  8:12 #18 skipped

  shot 18·duplicate of #17

• 

  8:40 #19 done37 line(s)

  shot 19·sharpness 1895.5

  1. AlEngineer0.98
  2. World'sFair1.00
  3. 091.00
  4. DRIFT - ACTION0.93
  5. Name the drift — the class picks the fix0.98
  6. DRIFT CLASS1.00
  7. what changed1.00
  8. VERDICT1.00
  9. ACTION1.00
  10. Structural1.00
  11. Expected /0.96
  12. Deploy-correlated + health-gated → controlled0.98
  13. topology Δ-0.91
  14. new nodes / edges1.00
  15. Investigate1.00
  16. rebaseline. Nothing shipped → page.0.99
  17. timing Δ - shape stable0.95
  18. Performance1.00
  19. Mitigate1.00
  20. Scale, shed load, or break the slow circuit.1.00
  21. Covariate1.00
  22. Absorb1.00
  23. Traffic shift, not a fault — update the baseline, don't0.98
  24. input mix Δ - system fine0.98
  25. page.1.00
  26. Concept1.00
  27. same inputs - new graphs0.97
  28. Regression1.00
  29. Flag for rollback; trigger model retrain.0.98
  30. Detection without classification is just a louder alarm. Responding to a KIND — not a magnitude — is what makes automation safe.1.00
  31. LEARNED EXECUTION GRAPHS1.00
  32. AIE WORLD'S FAIR '26 GRAPHS TRACK0.97
  33. 10 / 120.95
  34. TRACK 5· JULY 2, 20260.95
  35. AlEngine0.97
  36. Graphs1.00
  37. World's Fair0.99
• 

  9:18 #20 skipped

  shot 20·duplicate of #19

• 

  9:38 #21 skipped

  shot 21·duplicate of #19

• 

  10:22 #22 done38 line(s)

  shot 22·sharpness 2023.1

  1. AlEngineer0.99
  2. World'sFair1.00
  3. 091.00
  4. DRIFT - ACTION0.95
  5. Name the drift — the class picks the fix0.98
  6. DRIFT CLASS1.00
  7. what changed1.00
  8. VERDICT1.00
  9. ACTION1.00
  10. PRESENTED BY1.00
  11. Structural1.00
  12. Expected /0.97
  13. Deploy-correlated + health-gated → controlled0.98
  14. Microsoft1.00
  15. topology0.99
  16. new nodes / edges1.00
  17. Investigate1.00
  18. rebaseline. Nothing shipped → page.0.98
  19. Performance1.00
  20. timing Δ - shape stable0.98
  21. Mitigate1.00
  22. Scale, shed load, or break the slow circuit.1.00
  23. Covariate1.00
  24. input mix Δ - system fine0.98
  25. Absorb1.00
  26. page.1.00
  27. Traffic shift, not a fault — update the baseline, don't0.98
  28. Concept1.00
  29. same inputs - new graphs0.98
  30. Regression1.00
  31. Flag for rollback; trigger model retrain.0.99
  32. Detection without classification is just a louder alarm. Responding to a KIND — not a magnitude — is what makes automation safe.1.00
  33. LEARNED EXECUTION GRAPHS1.00
  34. AIE WORLD'S FAIR 26 GRAPHS TRACK0.97
  35. 10 / 120.95
  36. TRACK 5·JULY 2,20260.96
  37. Graphs1.00
  38. World's Fair0.99
• 

  10:37 #23 skipped

  shot 23·duplicate of #22

Transcript

164 cues· 2,388 words· 12,566 chars

Chapters

1. 0:00 Execution graphs for anomaly and drift detection
2. 1:07 What a short lived execution graph is
3. 3:28 Tiered checks and per client baselines
4. 5:23 The method: baseline, deviation, localize, act
5. 6:16 Localizing a slow node, and how the system is trained
6. 7:33 Anomaly versus drift
7. 8:55 The three kinds of drift: structural, volume, covariate
8. 12:46 The pipeline: from telemetry to gradual rollout
9. 13:54 Hot path versus recon, and worked examples
10. 15:21 Tuning it: delayed events, sampling, cold starts
11. 17:09 Results and lessons