vidtheque.

read-only demo

Videos shRR1e2HXMk

Codex, Behind the Harness — Dominik Kundel, OpenAI

index_state ready data_status ok

AI Engineer· published 2026-08-10· 0:20:54· en-US· indexed 2026-08-10 23:58

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:03, 1 of 1 keyframes kept
  2. Shot 1, 0:03 to 0:05, 1 of 1 keyframes kept
  3. Shot 2, 0:05 to 0:12, 1 of 1 keyframes kept
  4. Shot 3, 0:12 to 0:22, 1 of 1 keyframes kept
  5. Shot 4, 0:22 to 0:53, 1 of 1 keyframes kept
  6. Shot 5, 0:53 to 0:58, 1 of 1 keyframes kept
  7. Shot 6, 0:58 to 1:14, 1 of 1 keyframes kept
  8. Shot 7, 1:14 to 1:33, 1 of 1 keyframes kept
  9. Shot 8, 1:33 to 1:53, 1 of 1 keyframes kept
  10. Shot 9, 1:53 to 2:01, 1 of 1 keyframes kept
  11. Shot 10, 2:01 to 2:27, 1 of 1 keyframes kept
  12. Shot 11, 2:27 to 2:54, 1 of 1 keyframes kept
  13. Shot 12, 2:54 to 3:05, 1 of 1 keyframes kept
  14. Shot 13, 3:05 to 3:13, 1 of 1 keyframes kept
  15. Shot 14, 3:13 to 3:40, 1 of 1 keyframes kept
  16. Shot 15, 3:40 to 4:07, 0 of 1 keyframes kept
  17. Shot 16, 4:07 to 4:18, 1 of 1 keyframes kept
  18. Shot 17, 4:18 to 4:25, 0 of 1 keyframes kept
  19. Shot 18, 4:25 to 5:05, 1 of 1 keyframes kept
  20. Shot 19, 5:05 to 5:18, 1 of 1 keyframes kept
  21. Shot 20, 5:18 to 5:22, 1 of 1 keyframes kept
  22. Shot 21, 5:22 to 5:24, 1 of 1 keyframes kept
  23. Shot 22, 5:24 to 5:25, 0 of 1 keyframes kept
  24. Shot 23, 5:25 to 5:28, 1 of 1 keyframes kept
  25. Shot 24, 5:28 to 5:30, 1 of 1 keyframes kept
  26. Shot 25, 5:30 to 5:41, 0 of 1 keyframes kept
  27. Shot 26, 5:41 to 5:43, 1 of 1 keyframes kept
  28. Shot 27, 5:43 to 5:46, 0 of 1 keyframes kept
  29. Shot 28, 5:46 to 5:51, 0 of 1 keyframes kept
  30. Shot 29, 5:51 to 6:21, 0 of 1 keyframes kept
  31. Shot 30, 6:21 to 6:24, 1 of 1 keyframes kept
  32. Shot 31, 6:24 to 6:35, 0 of 1 keyframes kept
  33. Shot 32, 6:35 to 7:00, 1 of 1 keyframes kept
  34. Shot 33, 7:00 to 7:07, 0 of 1 keyframes kept
  35. Shot 34, 7:07 to 7:18, 1 of 1 keyframes kept
  36. Shot 35, 7:18 to 7:28, 0 of 1 keyframes kept
  37. Shot 36, 7:28 to 7:58, 1 of 1 keyframes kept
  38. Shot 37, 7:58 to 8:09, 0 of 1 keyframes kept
  39. Shot 38, 8:09 to 8:15, 1 of 1 keyframes kept
  40. Shot 39, 8:15 to 8:42, 1 of 1 keyframes kept
  41. Shot 40, 8:42 to 9:03, 1 of 1 keyframes kept
  42. Shot 41, 9:03 to 9:08, 0 of 1 keyframes kept
  43. Shot 42, 9:08 to 9:37, 1 of 1 keyframes kept
  44. Shot 43, 9:37 to 10:06, 0 of 1 keyframes kept
  45. Shot 44, 10:06 to 10:14, 0 of 1 keyframes kept
  46. Shot 45, 10:14 to 10:46, 1 of 1 keyframes kept
  47. Shot 46, 10:46 to 11:18, 1 of 1 keyframes kept
  48. Shot 47, 11:18 to 11:39, 1 of 1 keyframes kept
  49. Shot 48, 11:39 to 11:57, 1 of 1 keyframes kept
  50. Shot 49, 11:57 to 12:04, 0 of 1 keyframes kept
  51. Shot 50, 12:04 to 12:36, 1 of 1 keyframes kept
  52. Shot 51, 12:36 to 13:13, 0 of 1 keyframes kept
  53. Shot 52, 13:13 to 13:33, 1 of 1 keyframes kept
  54. Shot 53, 13:33 to 13:53, 0 of 1 keyframes kept
  55. Shot 54, 13:53 to 13:55, 0 of 1 keyframes kept
  56. Shot 55, 13:55 to 14:21, 1 of 1 keyframes kept
  57. Shot 56, 14:21 to 14:47, 0 of 1 keyframes kept
  58. Shot 57, 14:47 to 15:17, 0 of 1 keyframes kept
  59. Shot 58, 15:17 to 15:30, 1 of 1 keyframes kept
  60. Shot 59, 15:30 to 15:34, 1 of 1 keyframes kept
  61. Shot 60, 15:34 to 15:42, 1 of 1 keyframes kept
  62. Shot 61, 15:42 to 15:51, 1 of 1 keyframes kept
  63. Shot 62, 15:51 to 15:56, 1 of 1 keyframes kept
  64. Shot 63, 15:56 to 16:02, 0 of 1 keyframes kept
  65. Shot 64, 16:02 to 16:30, 1 of 1 keyframes kept
  66. Shot 65, 16:30 to 16:33, 0 of 1 keyframes kept
  67. Shot 66, 16:33 to 16:37, 1 of 1 keyframes kept
  68. Shot 67, 16:37 to 16:39, 1 of 1 keyframes kept
  69. Shot 68, 16:39 to 16:40, 0 of 1 keyframes kept
  70. Shot 69, 16:40 to 16:44, 1 of 1 keyframes kept
  71. Shot 70, 16:44 to 16:46, 0 of 1 keyframes kept
  72. Shot 71, 16:46 to 17:02, 0 of 1 keyframes kept
  73. Shot 72, 17:02 to 17:10, 0 of 1 keyframes kept
  74. Shot 73, 17:10 to 17:12, 1 of 1 keyframes kept
  75. Shot 74, 17:12 to 17:15, 1 of 1 keyframes kept
  76. Shot 75, 17:15 to 17:18, 0 of 1 keyframes kept
  77. Shot 76, 17:18 to 17:21, 0 of 1 keyframes kept
  78. Shot 77, 17:21 to 17:23, 1 of 1 keyframes kept
  79. Shot 78, 17:23 to 17:32, 1 of 1 keyframes kept
  80. Shot 79, 17:32 to 17:40, 0 of 1 keyframes kept
  81. Shot 80, 17:40 to 17:45, 1 of 1 keyframes kept
  82. Shot 81, 17:45 to 17:47, 1 of 1 keyframes kept
  83. Shot 82, 17:47 to 17:48, 0 of 1 keyframes kept
  84. Shot 83, 17:48 to 17:50, 0 of 1 keyframes kept
  85. Shot 84, 17:50 to 17:52, 1 of 1 keyframes kept
  86. Shot 85, 17:52 to 18:04, 1 of 1 keyframes kept
  87. Shot 86, 18:04 to 18:13, 1 of 1 keyframes kept
  88. Shot 87, 18:13 to 18:21, 0 of 1 keyframes kept
  89. Shot 88, 18:21 to 18:34, 1 of 1 keyframes kept
  90. Shot 89, 18:34 to 18:35, 0 of 1 keyframes kept
  91. Shot 90, 18:35 to 18:43, 1 of 1 keyframes kept
  92. Shot 91, 18:43 to 18:48, 1 of 1 keyframes kept
  93. Shot 92, 18:48 to 18:49, 1 of 1 keyframes kept
  94. Shot 93, 18:49 to 18:52, 0 of 1 keyframes kept
  95. Shot 94, 18:52 to 19:15, 1 of 1 keyframes kept
  96. Shot 95, 19:15 to 19:17, 0 of 1 keyframes kept
  97. Shot 96, 19:17 to 19:19, 1 of 1 keyframes kept
  98. Shot 97, 19:19 to 19:35, 1 of 1 keyframes kept
  99. Shot 98, 19:35 to 19:42, 0 of 1 keyframes kept
  100. Shot 99, 19:42 to 20:31, 1 of 1 keyframes kept
  101. Shot 100, 20:31 to 20:35, 1 of 1 keyframes kept
  102. Shot 101, 20:35 to 20:37, 1 of 1 keyframes kept
  103. Shot 102, 20:37 to 20:54, 0 of 1 keyframes kept

103 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
187
whisperx 187
chunks
37
from 187 cues
keyframes
66
kept of 103 captured
frames with text
66
1,698 lines read
chapters
0
from the source metadata
keyframe bytes
13.5 MB
word timings on 187 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-10 23:52 1m 19s
stt done — 2026-08-10 23:53 22s
chunk done — 2026-08-10 23:54 0s
text_embed done — 2026-08-10 23:54 1s
keyframe done — 2026-08-10 23:54 3m 01s
ocr done — 2026-08-10 23:57 41s
frame_embed done — 2026-08-10 23:57 11s

Frames, and what the machine read

  • 0:02 #0 done2 line(s)

    shot 0·sharpness 459.9

    1. AlEngineer0.96
    2. World's Fair1.00

  • 0:03 #1 done2 line(s)

    shot 1·sharpness 662.1

    1. AlEngineer0.95
    2. World's Fair0.99

  • 0:10 #2 done24 line(s)

    shot 2·sharpness 2744.8

    1. LAB & PLATINUM SPONSORS0.99
    2. Amazon AGI Lab0.98
    3. ANTHROP\C1.00
    4. Google DeepMind1.00
    5. MINIMAX0.94
    6. OpenAI0.92
    7. Akamai1.00
    8. arize1.00
    9. aws1.00
    10. Braintrust bright data0.98
    11. B1.00
    12. Browserbase1.00
    13. docker1.00
    14. :neo4j0.92
    15. ORACLE1.00
    16. PayPal1.00
    17. qodo1.00
    18. reducto1.00
    19. Sonar1.00
    20. Makers of1.00
    21. togetherai1.00
    22. Unblocked1.00
    23. WorkOS1.00
    24. SonarQube1.00

  • 0:14 #3 done2 line(s)

    shot 3·sharpness 157.2

    1. AlEngineer0.99
    2. World's Fair1.00

  • 0:40 #4 done10 line(s)

    shot 4·sharpness 1466.4

    1. OpenAl0.93
    2. AlEngineer0.98
    3. World'sFair1.00
    4. Codex: Behind the harness0.99
    5. PRESENTED BY1.00
    6. Microsoft1.00
    7. Dominik Kundel1.00
    8. Developer Experience1.00
    9. World's Fair0.97
    10. Engineering the future of Al1.00

  • 0:54 #5 done14 line(s)

    shot 5·sharpness 1464.2

    1. Agent loop1.00
    2. AlEngineer0.97
    3. World'sFair1.00
    4. ASSEMBLE CONVERSATION HISTORY0.99
    5. PRESENTED BY1.00
    6. LATEST USER INPUT0.97
    7. Microsoft1.00
    8. MODELINFERENCE1.00
    9. AGENT RESPONSE0.99
    10. TOOL CALLS0.99
    11. OpenAl0.96
    12. 5 / 550.85
    13. TRACK 8• JULY 2, 20260.95
    14. Agentic Engineering1.00

  • 1:10 #6 done14 line(s)

    shot 6·sharpness 958.9

    1. AlEngineer0.99
    2. World'sFair1.00
    3. PRESENTED BY1.00
    4. Do anything1.00
    5. Microsoft1.00
    6. +0.58
    7. Approve for me0.94
    8. 5.5 Extra High0.97
    9. Choose project1.00
    10. OpenAl0.95
    11. 6 / 550.90
    12. World's Fair0.96
    13. TRACK 8· JULY 2, 20260.94
    14. Agentic Engineering1.00

  • 1:16 #7 done90 line(s)

    shot 7·sharpness 1679.8

    1. AlEngineer0.94
    2. World's Fair0.98
    3. PlatformSolutionsResourcesOpen Source EnterprisePricing0.98
    4. O. Search or jump to...0.93
    5. Sign in0.99
    6. Sign up0.98
    7. openai/codex Public0.90
    8. Notifications1.00
    9. ￥ Fark 13.7k0.91
    10. Star 92.8k0.95
    11. () Code0.83
    12. Issues 5k+0.95
    13. à Pull requests 4290.98
    14. Discussions1.00
    15. Actions0.95
    16. ① Security and quality0.98
    17. Insights1.00
    18. P main0.96
    19. 4096 Branches1069 Tags0.97
    20. Q Go to file0.97
    21. <> Code0.93
    22. About0.98
    23. Lightweight coding agent that runs in1.00
    24. PRESENTED BY1.00
    25. bookholt-oai and viyatb-oai [codex] Reject uniowered PowerShell AST0.96
    26. 27122b5 - 31 minutes ago0.92
    27. 7,714 Commits0.96
    28. your terminal0.96
    29. .codex0.99
    30. core: log AGENTS.md paths as URis (#28989)0.99
    31. 4 days ago1.00
    32. Readme1.00
    33. Microsoft1.00
    34. devcontainer1.00
    35. build: run buildifier from just fmt (#28125)0.98
    36. last week0.99
    37. Apache-2.0 license0.99
    38. Ra Contributing0.92
    39. -github0.93
    40. ci restore custom Windows runner with hermetic LL.VM 0.7..0.95
    41. 10 hours ago1.00
    42. Security policy0.98
    43. I .vscode0.86
    44. Recommend Bazel VSCode extension. (#25161)1.00
    45. 3 weeks ago1.00
    46. Custom properties0.97
    47. A Activity0.87
    48. bazel1.00
    49. core: load AGENTS.md from foreign environments (#28958)0.97
    50. 4 days ago1.00
    51. ☆ 92.8k stars0.95
    52. codex-ci0.97
    53. cll: add package path from install context (#26189)0.98
    54. 3 weeks ago0.92
    55. ② 504 watching0.93
    56. ￥ 13.7k forks0.95
    57. codex-rs1.00
    58. [codex] Reject unlowered PowerShell AST regions (#24092)0.98
    59. 31 minutes ago1.00
    60. Report repository1.00
    61. I docs0.87
    62. [codex] Remove child AGENTS.md prompt experiment (#2...0.98
    63. 4 days ago1.00
    64. patches1.00
    65. ck restore custom Windows runner with hermetic LLVM 0.7...0.97
    66. 10 hours ago1.00
    67. Releases 8650.94
    68. 0.142.0 Latest0.96
    69. scripts1.00
    70. Make formatter output quiet on success(#29467)0.98
    71. 2 hours ago0.98
    72. 5 hours ago0.98
    73. sdk0.88
    74. Make formatter output quiet on success (#29467)0.99
    75. 2 hours ago0.99
    76. + 864 releases0.96
    77. third_party0.97
    78. bazel: add PowerShell o Wine test harness (#28120)0.99
    79. last week1.00
    80. Packages1.00
    81. tools1.00
    82. build: run buildifier from juast fmt (#28125)0.97
    83. last week0.98
    84. No packages published0.93
    85. OpenAl0.98
    86. 7/ 650.82
    87. 00.51
    88. World's Fair0.97
    89. TRACK 8· JULY 2, 20260.95
    90. Agentic Engineering1.00

  • 1:43 #8 done10 line(s)

    shot 8·sharpness 1353.9

    1. AlEngineer0.99
    2. World's Fair0.96
    3. Disclaimer:1.00
    4. Current snapshot1.00
    5. in time0.93
    6. OpenAl0.96
    7. 8 / 55 >0.78
    8. World's Fair0.95
    9. TRACK 8• JULY 2, 20260.95
    10. Agentic Engineering1.00

  • 1:56 #9 done8 line(s)

    shot 9·sharpness 851.8

    1. Protocols1.00
    2. AlEngineer0.99
    3. World'sFair1.00
    4. OpenAl0.91
    5. 9 / 55 30.88
    6. World's Fair0.94
    7. TRACK 8• JULY 2, 20260.95
    8. Agentic Engineering1.00

  • 2:17 #10 done16 line(s)

    shot 10·sharpness 1733.5

    1. Protocols powering Codex1.00
    2. AlEngineer1.00
    3. World's Fair0.96
    4. 010.97
    5. 021.00
    6. App server1.00
    7. Responses APl0.98
    8. Handles the UI to0.98
    9. Handles the Agent to0.98
    10. Agent communication0.98
    11. Inference communication1.00
    12. OpenAl0.97
    13. 10 / 55 30.81
    14. Worid's Fair0.92
    15. TRACK 8• JULY 2, 20260.96
    16. Agentic Engineering1.00

  • 2:51 #11 done21 line(s)

    shot 11·sharpness 1935.4

    1. App server0.98
    2. AlEngineer0.97
    3. World's Fair0.98
    4. Client1.00
    5. Server1.00
    6. First-party products1.00
    7. Third-party integrations1.00
    8. Codex Desktop App1.00
    9. JetBrains IDEs1.00
    10. CODEX HARNESS0.98
    11. Codex TUI/CLI0.99
    12. JSON-RPC1.00
    13. JSON-RPC1.00
    14. VS Code (Codex)1.00
    15. VIA APP SERVER0.99
    16. Codex Web Runtime1.00
    17. Xcode1.00
    18. OpenAl0.97
    19. 11 / 55 30.93
    20. TRACK 8• JULY 2, 20260.95
    21. Agentic Engineering0.99

  • 2:56 #12 done12 line(s)

    shot 12·sharpness 1359.2

    1. AlEngineer0.97
    2. World'sFair1.00
    3. Claude Code v2.1.850.99
    4. Opus 4.6 · Claude Pro0.94
    5. ~/Developer/codex1.00
    6. /plugin install codex@openai-codex0.99
    7. medium · /effort0.94
    8. OpenAl0.91
    9. 12 / 55 )0.89
    10. World's Fair0.95
    11. TRACK 8• JULY 2, 20260.95
    12. Agentic Engineering1.00

  • 3:12 #13 done17 line(s)

    shot 13·sharpness 1404.8

    1. AlEngineer0.97
    2. World's Fair0.98
    3. CODEX 1010.98
    4. 520100%1.00
    5. 21.00
    6. 200%1.00
    7. 4000.99
    8. 3001.00
    9. 1001.00
    10. AMMOHEALTHARMS1.00
    11. ARMOR0.97
    12. 5201.00
    13. OpenAl0.97
    14. 13 / 550.90
    15. World's Fair0.94
    16. TRACK 8· JULY 2, 20260.95
    17. Agentic Engineering1.00

  • 3:34 #14 done23 line(s)

    shot 14·sharpness 2163.2

    1. Responses API0.96
    2. AlEngineer0.99
    3. World'sFair1.00
    4. Agentic by design1.00
    5. Redesign of Chat Completions1.00
    6. with agentic use cases in mind.1.00
    7. curl https://api.openai.com/v1/responses1.00
    8. -H "Content-Type: application/json"0.98
    9. -H "Authorization: Bearer $OPENAI_API_KEY"0.99
    10. Built-in capabilities1.00
    11. Includes built-in web search,0.99
    12. "model": "gpt-5.5",0.98
    13. image generation, and more.1.00
    14. "tools": [{"type": "web_search"}],0.99
    15. "input": "Build an AIEWF landing page"0.99
    16. Built on OpenResponses1.00
    17. Open protocol with a wide1.00
    18. range of hosting providers.1.00
    19. OpenAl0.96
    20. 14 / 55 )0.82
    21. World's Fair0.92
    22. TRACK 8· JULY 2, 20260.95
    23. Agentic Engineering1.00

  • 3:51 #15 skipped

    shot 15·duplicate of #14

  • 4:11 #16 done12 line(s)

    shot 16·sharpness 940.4

    1. AlEngineer0.97
    2. World'sFair1.00
    3. UI (e.g. Codex app)0.98
    4. App server1.00
    5. HARNESS1.00
    6. Responses1.00
    7. LLMINFERENCE1.00
    8. OpenAl0.95
    9. 15 / 550.93
    10. World's Fair0.97
    11. TRACK 8· JULY 2, 20260.94
    12. Agentic Engineering1.00

  • 4:24 #17 skipped

    shot 17·duplicate of #9

  • 4:45 #18 done20 line(s)

    shot 18·sharpness 1740.6

    1. Building the context1.00
    2. AlEngineer0.98
    3. World's Fair0.97
    4. 010.98
    5. 021.00
    6. 031.00
    7. Size1.00
    8. Flexibility1.00
    9. Cacheability1.00
    10. Avoid cost1.00
    11. Everyone's Codex1.00
    12. Better cost1.00
    13. and confusion1.00
    14. looks different1.00
    15. and speed control1.00
    16. OpenAl0.98
    17. 18 / 550.90
    18. World's Fair0.96
    19. TRACK 8• JULY 2, 20260.95
    20. Agentic Engineering1.00

  • 5:16 #19 done19 line(s)

    shot 19·sharpness 966.2

    1. Nano Codex1.00
    2. AlEngineer0.99
    3. Responses input1.00
    4. witing0.99
    5. World's Fair0.98
    6. Role:1.00
    7. System1.00
    8. Developer1.00
    9. User1.00
    10. Send the task to inspect the exact request0.99
    11. Find the subagent tool for inspecting tests, then tell me which tool you1.00
    12. would use. Do not execute it.1.00
    13. +0.99
    14. gpt-5.51.00
    15. 19 / 55 30.89
    16. 00.54
    17. World's Fair0.96
    18. TRACK 8· JULY 2, 20260.94
    19. Agentic Engineering1.00

  • 5:19 #20 done24 line(s)

    shot 20·sharpness 1065.2

    1. Nano Codex1.00
    2. AlEngineer0.99
    3. Responses input1.00
    4. World'sFair1.00
    5. Find the subagent tool for inspecting tests, then tell me0.99
    6. which tool you would use. Do not execute it.1.00
    7. Role:1.00
    8. 5 parts0.92
    9. System1.00
    10. Developer1.00
    11. User1.00
    12. Model instructions0.99
    13. Tool registry0.99
    14. Permissions + runtime0.97
    15. Available skills0.99
    16. 15%0.99
    17. Do anything1.00
    18. +0.99
    19. gpt-5.51.00
    20. Current task1.00
    21. <1%0.93
    22. World's Fair0.96
    23. TRACK 8· JULY 2, 20260.95
    24. Agentic Engineering0.99

  • 5:23 #21 done48 line(s)

    shot 21·sharpness 2214.5

    1. Nano Codex0.99
    2. AlEngineer0.96
    3. Responses input1.00
    4. Find the subagent tool for inspecting tests, then tell me0.98
    5. 6 parts0.90
    6. World'sFair1.00
    7. which tool you would use. Do not execute it.1.00
    8. Role:1.00
    9. System1.00
    10. Developer1.00
    11. User1.00
    12. Ran shell command0.95
    13. Model instructions1.00
    14. 66%0.97
    15. I'll use the test-triage skill for the test-inspection context, then search available0.99
    16. subagent tools.1.00
    17. Tool registry1.00
    18. PRESENTED BY0.99
    19. Permissions + runtime0.97
    20. Microsoft1.00
    21. Available skills1.00
    22. <skills_instructions> ## Skills A skill is a set of instructions provided through a SKILL.md' source. Below is the list of0.98
    23. 13%1.00
    24. skills that can be used. Each entry includes a name, description, and source locator. file' locators are on the host0.99
    25. filesystem, 'environment resource locators are owned by an execution environment, orchestrator resource' locators0.97
    26. are opaque non-filesystem resources, and 'custom resource locators use their provider's access mechanism. ###0.98
    27. Available skill - frontend-debugging: Debug small TypeScript and UI failures. (file: /workspace/.codex/skills/frontend-0.98
    28. debugging/SKILL.md) - test-triage: Interpret test output and narrow regresslions. (file: /workspace/.codex/skills/test-0.97
    29. triage/SKILL.md) - safe-shell: Use shell commands with narrow scope and explicit verification. (file:0.99
    30. /workspace/.codex/skills/safe-shell/SKILL.md) ### How to use skil - Discovery: The list above is the skills available in0.98
    31. this session (name + description + source locator). 'file' entries live on the host filesystem, 'environment resource0.99
    32. entries are owned by their execution environment, 'orchestrator resource' entries must be accessed through0.99
    33. skilslist' and 'skills.read', and custom resource' entries use their provider's access mechanism. - Trigger rules: If0.96
    34. the user names a skill (with '$SkillName' or plain text) OR the task clearly matches a skill's description shown above,0.98
    35. you must use that skill for that turn. Multiple mentions mean use them all. o not carry skills across turns unless re-0.99
    36. mentioned. - Missing/blocked: If a named skill isn't in the list or its source can't be read, say so briefly and continue0.99
    37. with the best fallback. - How to use a skill (progressive disclosure): 1) After deciding to use a skill, the main agent0.99
    38. must read its 'SKILL.md' completely before taking task actions. For a file' entry, open the listed path. For an0.97
    39. 'environment resource', use the filesystem of the owning environment. For an `orchestrator resource', call skills.list0.98
    40. Do anything1.00
    41. with ("authority":("kind*:"orchestrator*))', select the matching package, and pass its 'main_resource' to0.98
    42. +0.99
    43. gpt-5.51.00
    44. Thread history1.00
    45. 9%1.00
    46. Worild's Fair0.94
    47. TRACK 8· JULY 2,20260.94
    48. Agentic Engineering1.00

  • 5:25 #22 skipped

    shot 22·duplicate of #21

  • 5:25 #23 done38 line(s)

    shot 23·sharpness 1650.0

    1. Nano Codex1.00
    2. World'sFair1.00
    3. AlEngineer0.96
    4. which tool you would use. Do not execute it.0.99
    5. Find the subagent tool for inspecting tests, then tell me1.00
    6. Role:1.00
    7. Responses input1.00
    8. System1.00
    9. Developer1.00
    10. User1.00
    11. 00.60
    12. Ran shell command0.97
    13. I'll use the test-triage skill for the test-inspection context, then search available0.99
    14. Model instructions0.99
    15. subagent tools.1.00
    16. PRESENTED BY1.00
    17. I found the subagent tool I would use: spawn_agent. spawn_agent.0.99
    18. Microsoft0.96
    19. For inspecting tests without changing code, I'd choose it with agent_type:0.99
    20. "explorer" and a bounded message asking the subagent to inspect the0.99
    21. Tool registry0.99
    22. relevant test files and report findings.1.00
    23. I did not execute the subagent tool. Note: the test-triage skill file was listed0.99
    24. but not present at /workspace/.codex/skills/test-triage/SKILL.md, sol0.99
    25. Permissions + runtime0.99
    26. continued with the available tool discovery.1.00
    27. Available skills0.94
    28. 13%0.97
    29. Thread history1.00
    30. 9%1.00
    31. Do anything1.00
    32. +0.98
    33. gpt-5.51.00
    34. Current task1.00
    35. d%0.62
    36. World's Fair0.95
    37. TRACK 8· JULY 2, 20260.96
    38. Agentic Engineering1.00

Transcript

187 cues· 3,496 words· 18,933 chars

  1. 0:12 Hi, everyone.
  2. 0:13 We're going to start right on time because I'm going to speak basically at 2x.
  3. 0:17 I'm sorry I have a lot of content.
  4. 0:19 I'm trying to get you out here on time.
  5. 0:21 I want to start with a quick raise of hands.
  6. 0:23 So how many of you have built your own agents or are currently building your own agents?
  7. 0:28 Perfectly.
  8. 0:28 You're the right audience for this.
  9. 0:30 Over the next 20 minutes, I want to talk to you about a couple of different things that we're doing in the Codex harness that hopefully you can learn to apply to your own use cases or even just use the Codex harness with this in
  10. 0:43 in your own projects, or at bare minimum, learn what happens when you actually use codecs.
  11. 0:49 Since we're at iEngineer World's Fair, and we're actually on an agentic engineering track, I'm gonna stop bothering you with how does an agent work?
  12. 0:57 What is an agent?
  13. 0:59 And instead, I want to talk a bit more specifically about some key features that we have in the agent that I think are particularly interesting and are challenges you have to solve.
  14. 1:08 And so let's cover it from the lens of what actually happens when you send off a message.
  15. 1:15 Also, a quick reminder, if you're unaware, the Codex harness and everything I'm showing you is actually open source.
  16. 1:22 It's Apache 2 license, and the harness is written in Rust, so feel free to either learn from it, ask Codex deeper questions about what I'm covering, or fork it and make it your own.
  17. 1:34 Also disclaimer before we dive deep into it, this is a current state of affairs.
  18. 1:39 Things change so quickly.
  19. 1:41 You can always refer back to asking Codex what the current state is, but especially with new model releases, we often release new APIs and change sort of how the harness works.
  20. 1:50 So feel free to follow along as new models come out.
  21. 1:55 If we want to talk about how the Codex agent works, we first need to talk about what actually happens when you send off your message.
  22. 2:01 Namely, there's two protocols that are involved with the Codex agent.
  23. 2:07 The first one is what happens when you send it off in the UI and it goes to the harness.
  24. 2:11 We call that the app server.
  25. 2:12 I talked about that yesterday, so we're not going to spend too much time about it.
  26. 2:15 There will be a talk online that you can follow along.
  27. 2:18 The second part is the responses API, which handles the communication between the harness and the inference.
  28. 2:25 Both of these, though, are designed for an open ecosystem, meaning if you're building your own UI, you're building your own agent interface, you can actually build on top of the Codex harness using the app server protocol.
  29. 2:39 We use that same app server to power the Codex app, so it has really all of that functionality that you might expect from Codex, as well as we have a lot of third-party community projects that build on top of it, including Theos T3 Code or RemoteX, for example.
  30. 2:56 I even used that same app server to put Codex into Cloud Code.
  31. 2:59 So if you're a Cloud Code user and you want to leverage Codex, you can use that plugin.
  32. 3:05 And if you joined my talk yesterday, you saw me using that same protocol to actually put Codex into Doom, which was a fun adventure as well.
  33. 3:14 I mentioned the other part is the responses API.
  34. 3:17 So the response API was released last year as like a rethinking of the chat completions API in a more agentic world, meaning we redesigned slightly the structure, but more importantly, we added a lot of like building capabilities that are important for agents like web search, image gen, or other more complex capabilities that you will see as part of this talk.
  35. 3:38 We also want to make sure that this is like an open ecosystem.
  36. 3:42 So we worked with a lot of partners, including Ulama, LM Studio, NVIDIA, and others to codify an open responses schema and have a governance body for that so that other companies and platforms can actually build on that same responses API.
  37. 3:58 And you can use any responses API compatible harness model provider and actually plug it into the Codex harness.
  38. 4:08 So that's an overview of how these protocols work.
  39. 4:10 We're going from the UI to the harness with the app server protocol, and then from the harness to the LM inference using responses.
  40. 4:18 But what happens in the actual harness?
  41. 4:20 The first step, and arguably one of the most important ones, is context construction.
  42. 4:26 And during that, we care about three things quite a lot.
  43. 4:30 The first one is size.
  44. 4:31 We want to make sure that we don't blast through your token budgets and throw in a bunch of unnecessary content.
  45. 4:37 But also, the more context you have in your context, the higher it is that you have contradicting information, and it causes confusion for the model.
  46. 4:48 The other part is flexibility.
  47. 4:49 We want to make sure that regardless of many or how little skills you're using, you have a great experience, regardless of how many plugins and MCPs you install.
  48. 4:58 And of course, we want to make sure that things are performant and we know you're cost-sensitive, so cacheability is important as well.
  49. 5:06 To show you this and a couple of other things, I actually built this little nano codex here, which functions the same way.
  50. 5:13 It's built using the same code that is on the public repo, just turned into TypeScript.

Open at this second