vidtheque.

read-only demo

Videos MpZzWMdmQCE

Your coding agent doesn't always follow your rules — Talha Sheikh, Checkout.com

index_state ready data_status ok

AI Engineer· published 2026-07-08· 0:10:07· en-US· indexed 2026-08-11 04:35

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:05, 1 of 1 keyframes kept
  2. Shot 1, 0:05 to 0:09, 1 of 1 keyframes kept
  3. Shot 2, 0:09 to 0:14, 1 of 1 keyframes kept
  4. Shot 3, 0:14 to 0:54, 1 of 1 keyframes kept
  5. Shot 4, 0:54 to 0:57, 1 of 1 keyframes kept
  6. Shot 5, 0:57 to 1:18, 1 of 1 keyframes kept
  7. Shot 6, 1:18 to 1:57, 1 of 1 keyframes kept
  8. Shot 7, 1:57 to 2:15, 1 of 1 keyframes kept
  9. Shot 8, 2:15 to 2:42, 1 of 1 keyframes kept
  10. Shot 9, 2:42 to 3:08, 1 of 1 keyframes kept
  11. Shot 10, 3:08 to 3:35, 1 of 1 keyframes kept
  12. Shot 11, 3:35 to 3:44, 1 of 1 keyframes kept
  13. Shot 12, 3:44 to 4:01, 1 of 1 keyframes kept
  14. Shot 13, 4:01 to 4:23, 1 of 1 keyframes kept
  15. Shot 14, 4:23 to 4:41, 1 of 1 keyframes kept
  16. Shot 15, 4:41 to 5:13, 1 of 1 keyframes kept
  17. Shot 16, 5:13 to 5:17, 1 of 1 keyframes kept
  18. Shot 17, 5:17 to 6:02, 1 of 1 keyframes kept
  19. Shot 18, 6:02 to 6:30, 1 of 1 keyframes kept
  20. Shot 19, 6:30 to 6:57, 1 of 1 keyframes kept
  21. Shot 20, 6:57 to 7:12, 1 of 1 keyframes kept
  22. Shot 21, 7:12 to 7:27, 1 of 1 keyframes kept
  23. Shot 22, 7:27 to 7:45, 1 of 1 keyframes kept
  24. Shot 23, 7:45 to 7:59, 1 of 1 keyframes kept
  25. Shot 24, 7:59 to 8:13, 1 of 1 keyframes kept
  26. Shot 25, 8:13 to 8:40, 1 of 1 keyframes kept
  27. Shot 26, 8:40 to 8:51, 1 of 1 keyframes kept
  28. Shot 27, 8:51 to 8:53, 1 of 1 keyframes kept
  29. Shot 28, 8:53 to 9:23, 1 of 1 keyframes kept
  30. Shot 29, 9:23 to 9:52, 1 of 1 keyframes kept
  31. Shot 30, 9:52 to 10:07, 1 of 1 keyframes kept

31 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
134
whisperx 134
chunks
17
from 134 cues
keyframes
31
kept of 31 captured
frames with text
31
336 lines read
chapters
0
from the source metadata
keyframe bytes
2.5 MB
word timings on 134 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-11 04:33 1m 06s
stt done — 2026-08-11 04:34 11s
chunk done — 2026-08-11 04:34 0s
text_embed done — 2026-08-11 04:34 0s
keyframe done — 2026-08-11 04:34 52s
ocr done — 2026-08-11 04:35 10s
frame_embed done — 2026-08-11 04:35 5s

Frames, and what the machine read

  • 0:02 #0 done2 line(s)

    shot 0·sharpness 667.3

    1. Al Engineer0.94
    2. EUROPE1.00

  • 0:06 #1 done2 line(s)

    shot 1·sharpness 825.0

    1. PRESENTINGSPONSOR1.00
    2. Google DeepMind1.00

  • 0:12 #2 done3 line(s)

    shot 2·sharpness 938.6

    1. PLATINUM SPONSORS0.98
    2. # Braintrust0.95
    3. WorkOS OpenAI0.95

  • 0:38 #3 done1 line(s)

    shot 3·sharpness 366.8

    1. Vector1.00

  • 0:56 #4 done9 line(s)

    shot 4·sharpness 1141.8

    1. ktop/dev/vector-harmess/docs/presentation.html0.97
    2. AIE1.00
    3. Vector1.00
    4. ★1.00
    5. ★1.00
    6. ★1.00
    7. Intensity is vanity; Direction is sanity1.00
    8. Talha Sheikh1.00
    9. Google DeepMind1.00

  • 1:10 #5 done6 line(s)

    shot 5·sharpness 1260.7

    1. ***0.80
    2. AIE1.00
    3. ★1.00
    4. ★1.00
    5. ★1.00
    6. Engineering the future of Al0.99

  • 1:30 #6 done11 line(s)

    shot 6·sharpness 1554.4

    1. The Lie Detector - Talha Sh0.95
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. @0.59
    4. AIE1.00
    5. You.The Enforcement.1.00
    6. ★1.00
    7. ★1.00
    8. The agent says 'done.' You check anyway.0.98
    9. Not because you want to — because nothing else does.1.00
    10. # Braintrust0.94
    11. WorkOS OpenAI0.95

  • 2:13 #7 done8 line(s)

    shot 7·sharpness 1125.5

    1. op/dev/vector-hamess/docs/presentation.html1.00
    2. Deterministic?1.00
    3. AIE1.00
    4. ITriedIt0.98
    5. ★0.99
    6. ★1.00
    7. AlEngineer0.96
    8. EUROPE1.00

  • 2:26 #8 done30 line(s)

    shot 8·sharpness 1072.1

    1. The Lie Detector - Talha Sh0.97
    2. ① File0.90
    3. /Users/talhashelkh/Desktop/dev/vector-haress/docs/presentation.html0.99
    4. ☆0.98
    5. @0.59
    6. THE IDEA1.00
    7. I Tried It0.97
    8. #.vector/config.yaml0.98
    9. checks:1.00
    10. test-pass:1.00
    11. run: "npm test"0.99
    12. AIE1.00
    13. retries: 30.97
    14. ★1.00
    15. type-check:1.00
    16. ★1.00
    17. ★1.00
    18. ★1.00
    19. run: "npx tsc --noEmit"0.99
    20. coverage:1.00
    21. run: "npx vitest --coverage --threshold 80"0.98
    22. retries: 20.95
    23. lint:0.98
    24. run: "npm run lint"1.00
    25. vectors:1.00
    26. v1:0.99
    27. checks: [test-pass, type-check, coverage, lint]1.00
    28. Every check is a shell command. Exit 0 = pass.0.98
    29. AlEngineer0.97
    30. EUROPE1.00

  • 3:03 #9 done10 line(s)

    shot 9·sharpness 1142.4

    1. The Lie Detector -Talha Sh0.96
    2. h/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. AIE1.00
    4. Trust1.00
    5. ★0.92
    6. ★1.00
    7. ★1.00
    8. Not because the Al got smarter → because something verified it.0.99
    9. Then I made the mistake of telling people.0.99
    10. Engineering the future of Al0.99

  • 3:27 #10 done9 line(s)

    shot 10·sharpness 1293.4

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. 00.57
    4. Trust@0.92
    5. AIE1.00
    6. The PushbackX0.96
    7. 'We're making models better. You won't need this.'1.00
    8. That shook me.0.97
    9. Engineering the future of Al1.00

  • 3:42 #11 done14 line(s)

    shot 11·sharpness 1703.1

    1. esktop/dev/vector-hamess/docs/presentation.html0.98
    2. @0.63
    3. Crisis mo0.95
    4. AIE1.00
    5. Project1.00
    6. ★0.97
    7. ★1.00
    8. ★0.99
    9. ★1.00
    10. Glasswing1.00
    11. Securing critical software1.00
    12. for the AI era0.99
    13. Continue reading0.99
    14. Engineering the future of Al0.98

  • 3:51 #12 done8 line(s)

    shot 12·sharpness 1480.2

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.99
    3. Project1.00
    4. Glasswing1.00
    5. AIE1.00
    6. Capability ≠ Reliability0.98
    7. Engineering the future of Al0.98
    8. ineer1.00

  • 4:14 #13 done8 line(s)

    shot 13·sharpness 1507.2

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. Capability≠Reliability0.99
    4. AIE1.00
    5. Instructions ≠ Verification1.00
    6. AlEngineer0.97
    7. EUROPE1.00
    8. er1.00

  • 4:36 #14 done10 line(s)

    shot 14·sharpness 1460.6

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. Instructions ≠ Verification0.99
    4. AIE1.00
    5. ★0.95
    6. Guardrails > Model Size0.97
    7. ★1.00
    8. ★1.00
    9. AlEngineer0.96
    10. EUROPE1.00

  • 5:09 #15 done6 line(s)

    shot 15·sharpness 1517.1

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. ThenIHeardIt0.99
    4. AIE1.00
    5. Everywhere1.00
    6. Engineering the future of Al0.99

  • 5:16 #16 done7 line(s)

    shot 16·sharpness 1004.4

    1. ktop/dev/vector-haress/docs/presentation.html0.98
    2. ThenIHeard It0.96
    3. Everywhere1.00
    4. AIE1.00
    5. ★1.00
    6. ★1.00
    7. Engineering the future of Al0.99

  • 5:35 #17 done6 line(s)

    shot 17·sharpness 1393.7

    1. The Lie Detector -Talha Sh0.96
    2. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.98
    3. AIE1.00
    4. The Real Question0.99
    5. Q0.51
    6. Engineering the future of Al0.99

  • 6:24 #18 done39 line(s)

    shot 18·sharpness 1190.4

    1. The Lie Detector - Talha Sh0.96
    2. ① File0.87
    3. /Users/talhashelkh/Desktop/dev/vector-haress/docs/presentation.html0.99
    4. ☆0.98
    5. The Pattern0.99
    6. THE PATTERN1.00
    7. Verification at Every Layer1.00
    8. AIE1.00
    9. Layer1.00
    10. Check Point1.00
    11. Checks1.00
    12. ★1.00
    13. Conversation1.00
    14. on-chat1.00
    15. tests, types, coverage0.99
    16. ★1.00
    17. ★1.00
    18. ★1.00
    19. Step0.97
    20. Conmit0.92
    21. on-commit1.00
    22. on-step1.00
    23. build, e2e, security0.99
    24. lint, tests, docs0.99
    25. Async1.00
    26. on-async1.00
    27. smoke, health, perf1.00
    28. Adversarial1.00
    29. on-adversarial1.00
    30. full suite, coverage 90%+1.00
    31. Language:0.98
    32. JS · Python · Go · Rust · anything with a shell0.92
    33. Agent:1.00
    34. Claude · GPT · Cursor · Copilot · any0.93
    35. Checks:1.00
    36. exit 0 = pass . your commands, your rules0.98
    37. AlEngineer0.96
    38. EUROPE1.00
    39. er1.00

  • 6:54 #19 done7 line(s)

    shot 19·sharpness 1512.5

    1. The Lie Detector - Taiha Sh0.94
    2. File /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.97
    3. The Contract0.98
    4. TheIndustryIs0.99
    5. AIE1.00
    6. Converging1.00
    7. Engineering the future of Al0.98

  • 7:08 #20 done26 line(s)

    shot 20·sharpness 1380.4

    1. The Lie Detector -Taiha Sh0.92
    2. ① File0.86
    3. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.99
    4. ☆0.98
    5. TheIndustryls0.96
    6. ANTHROPIC1.00
    7. Converg1.00
    8. The advisor strategy0.99
    9. AIE1.00
    10. Executor1.00
    11. Advisor1.00
    12. ★1.00
    13. Main loop0.99
    14. Sonnet1.00
    15. Tool call—0.97
    16. Opus1.00
    17. ★1.00
    18. ★1.00
    19. ★1.00
    20. Runseverytun0.99
    21. Read / write0.95
    22. Shared context1.00
    23. Reviews context0.98
    24. Advisor reads the same context as Executor0.97
    25. Advisor Strategy → optimize which model thinks. Doesn't verify the output.0.98
    26. Engineering the future of Al1.00

  • 7:25 #21 done14 line(s)

    shot 21·sharpness 1014.3

    1. /Users/talhashelkh/Desktop/dev/vector-harness/docs/presentation.html0.99
    2. ☆0.72
    3. OPENAI1.00
    4. *★★0.60
    5. AIE1.00
    6. leveraging Codex in an0.99
    7. Harness engineering:0.99
    8. ★1.00
    9. agent-first world0.97
    10. ★1.00
    11. ★1.00
    12. ★1.00
    13. Harness Engineering → 'Humans steer. Agents execute.' The harness is the product.1.00
    14. Engineering the future of Al1.00

  • 7:31 #22 done12 line(s)

    shot 22·sharpness 1271.6

    1. The Lie Detector0.99
    2. h/Desktop/dev/vector-harness/docs/presentation.html0.99
    3. ☆0.95
    4. QODO0.91
    5. qodo0.79
    6. AIE1.00
    7. ★1.00
    8. ★1.00
    9. Code Review by Qodo1.00
    10. Review-first, not copilot-first. Verification as the entire business.0.99
    11. Engineering the future of Al0.97
    12. er1.00

  • 7:51 #23 done17 line(s)

    shot 23·sharpness 1072.7

    1. The Lie Detector - Talha Sh0.94
    2. ①File0.75
    3. /Users/talhashelkh/Desktop/dev/vector-haress/docs/presentation.html0.99
    4. ☆0.86
    5. WhatI Actually Learned0.96
    6. AIE1.00
    7. Enforce, don't instruct.0.98
    8. ★1.00
    9. A pipeline gate fires every time. A prompt gets forgotten by token 50,000.0.98
    10. ★1.00
    11. ★1.00
    12. ★1.00
    13. Guide, don't prescribe.0.98
    14. Tell the model where the landmines are. Let it do what it's aiready good at.0.99
    15. Measure, don't assume.0.99
    16. Trust is a pass rate, a hash, a delta score. Not a feeling.0.96
    17. Engineering the future of Al0.99

Transcript

134 cues· 1,754 words· 9,259 chars

  1. 0:15 Hello, hello.
  2. 0:17 Have you ever given a task to Cloud Code?
  3. 0:19 And you give it a feature, and you're like, OK, cool.
  4. 0:23 Can you build this for me?
  5. 0:24 And Cloud Code starts putting it out into subtasks.
  6. 0:27 And you see it, OK, this is pretty cool.
  7. 0:29 You see it running multiple subagents.
  8. 0:32 All right, this is really cool.
  9. 0:33 And you can see it ripping through all of your tasks, subagents being completed.
  10. 0:36 And it gives you a final output, task completed.
  11. 0:39 Amazing, great.
  12. 0:41 But when you actually try to run it, it should be like, oh, well, it's not.
  13. 0:45 Something has failed.
  14. 0:45 Hey, Claude, can you fix this little bit thing?
  15. 0:47 Oh, well, let's try it again.
  16. 0:48 OK, it's fixed.
  17. 0:49 Everything should be working.
  18. 0:50 Oh, no.
  19. 0:51 Actually, just this tiny little thing is just missing.
  20. 0:55 And that's what my talk is about.
  21. 0:58 All I want to do is play Cyberpunk on my Xbox while I have Cloud Code do some work for me.
  22. 1:04 And what I realized was the problem is that I kept on telling Cloud, like, hey, fix this, fix that, fix this, fix that, even though if I give it a spec, if I give it some instructions, if I give it little to no instructions, every time there is something that I need to tell it.
  23. 1:19 So what that means is like, I am the enforcement.
  24. 1:23 I am the enforcement there.
  25. 1:24 I have to tell Claude on what exactly you need to do and how exactly this needs to be enforced.
  26. 1:30 So the agent says it's done, but you have to check it anyway because there is nothing else that can check it for you.
  27. 1:38 So what I wanted was something to be very deterministic.
  28. 1:41 So when an agent says it's completed, have this enforcement layer deterministically check something like whether it's actually been done or not.
  29. 1:47 Actually, the way I wanted it to be done, because it says it is done, but is it the way that I want it?
  30. 1:53 So I needed some deterministic way to do that.
  31. 1:57 So I tried it, I built my own vector, I call it my own product called Vector V1, and it deterministically checks Cloud's output.
  32. 2:05 And the way it did that is through using Cloud Hooks, so that way whenever Cloud finishes its session, it automatically, the hook calls my vector product or program, and it checks it for me.
  33. 2:16 Cool, and this is how it essentially looks.
  34. 2:17 So I basically give it a config file, define all of my tests over here, like what I needed to be checked, and if it fails, it can actually keep on telling Cloud, like, hey, look, this is failing, try again, try again, try again.
  35. 2:28 Sorry if it's a little bit, little.
  36. 2:31 So you can see one of the test outputs over here.
  37. 2:34 So it's like, okay, first the test pass has failed, then it retries again, then all of the things pass.
  38. 2:39 Okay, cool.
  39. 2:40 So what that means is it's not about whether Cloud can actually do the task, it's about trust.
  40. 2:45 Can I trust Claude to actually do everything for me?
  41. 2:48 And by the way, when I say Claude, I'm just talking in general about LLM agents in general, is when I give a task to a coding agent, does it actually complete it?
  42. 2:58 Yeah.
  43. 2:59 So, and that's something that, and what I started doing was started telling this about to people about and going to different events.
  44. 3:05 And it was, it was really cool.
  45. 3:06 Like, Hey, look, what about this verification feature that I built?
  46. 3:08 It was so good.
  47. 3:09 It was so amazing.
  48. 3:10 And then I met one of the anthropic engineers and, um, he, and they just told me that we're not going to need this anymore.
  49. 3:18 Like we'll have like another agent or another model that it will be so smart that you won't need enforcement.
  50. 3:25 Okay.

Open at this second