vidtheque.

read-only demo

Videos SbcQYbrvAfI

Build a Prompt Learning Loop - SallyAnn DeLucia & Fuad Ali, Arize

index_state ready data_status ok

AI Engineer· published 2026-01-06· 0:52:08· en-US· indexed 2026-08-10 23:15

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:12, 1 of 1 keyframes kept
  2. Shot 1, 0:12 to 0:16, 1 of 1 keyframes kept
  3. Shot 2, 0:16 to 0:20, 1 of 1 keyframes kept
  4. Shot 3, 0:20 to 0:49, 1 of 1 keyframes kept
  5. Shot 4, 0:49 to 1:18, 1 of 1 keyframes kept
  6. Shot 5, 1:18 to 1:47, 0 of 1 keyframes kept
  7. Shot 6, 1:47 to 1:50, 1 of 1 keyframes kept
  8. Shot 7, 1:50 to 2:09, 1 of 1 keyframes kept
  9. Shot 8, 2:09 to 2:25, 1 of 1 keyframes kept
  10. Shot 9, 2:25 to 3:09, 1 of 1 keyframes kept
  11. Shot 10, 3:09 to 3:49, 1 of 1 keyframes kept
  12. Shot 11, 3:49 to 4:28, 1 of 1 keyframes kept
  13. Shot 12, 4:28 to 4:37, 1 of 1 keyframes kept
  14. Shot 13, 4:37 to 5:00, 0 of 1 keyframes kept
  15. Shot 14, 5:00 to 5:47, 1 of 1 keyframes kept
  16. Shot 15, 5:47 to 6:30, 1 of 1 keyframes kept
  17. Shot 16, 6:30 to 7:14, 1 of 1 keyframes kept
  18. Shot 17, 7:14 to 7:32, 1 of 1 keyframes kept
  19. Shot 18, 7:32 to 8:15, 1 of 1 keyframes kept
  20. Shot 19, 8:15 to 8:57, 1 of 1 keyframes kept
  21. Shot 20, 8:57 to 8:59, 1 of 1 keyframes kept
  22. Shot 21, 8:59 to 9:22, 1 of 1 keyframes kept
  23. Shot 22, 9:22 to 9:45, 1 of 1 keyframes kept
  24. Shot 23, 9:45 to 10:22, 1 of 1 keyframes kept
  25. Shot 24, 10:22 to 10:43, 1 of 1 keyframes kept
  26. Shot 25, 10:43 to 11:29, 1 of 1 keyframes kept
  27. Shot 26, 11:29 to 11:42, 0 of 1 keyframes kept
  28. Shot 27, 11:42 to 11:44, 0 of 1 keyframes kept
  29. Shot 28, 11:44 to 12:31, 1 of 1 keyframes kept
  30. Shot 29, 12:31 to 13:05, 1 of 1 keyframes kept
  31. Shot 30, 13:05 to 13:15, 0 of 1 keyframes kept
  32. Shot 31, 13:15 to 13:33, 1 of 1 keyframes kept
  33. Shot 32, 13:33 to 13:36, 1 of 1 keyframes kept
  34. Shot 33, 13:36 to 13:55, 1 of 1 keyframes kept
  35. Shot 34, 13:55 to 13:56, 1 of 1 keyframes kept
  36. Shot 35, 13:56 to 14:22, 1 of 1 keyframes kept
  37. Shot 36, 14:22 to 14:47, 0 of 1 keyframes kept
  38. Shot 37, 14:47 to 15:15, 1 of 1 keyframes kept
  39. Shot 38, 15:15 to 15:43, 0 of 1 keyframes kept
  40. Shot 39, 15:43 to 15:46, 1 of 1 keyframes kept
  41. Shot 40, 15:46 to 16:03, 0 of 1 keyframes kept
  42. Shot 41, 16:03 to 16:12, 0 of 1 keyframes kept
  43. Shot 42, 16:12 to 16:14, 1 of 1 keyframes kept
  44. Shot 43, 16:14 to 16:42, 1 of 1 keyframes kept
  45. Shot 44, 16:42 to 17:11, 0 of 1 keyframes kept
  46. Shot 45, 17:11 to 17:39, 1 of 1 keyframes kept
  47. Shot 46, 17:39 to 18:04, 0 of 1 keyframes kept
  48. Shot 47, 18:04 to 18:30, 0 of 1 keyframes kept
  49. Shot 48, 18:30 to 18:56, 0 of 1 keyframes kept
  50. Shot 49, 18:56 to 19:21, 0 of 1 keyframes kept
  51. Shot 50, 19:21 to 19:47, 0 of 1 keyframes kept
  52. Shot 51, 19:47 to 20:12, 0 of 1 keyframes kept
  53. Shot 52, 20:12 to 20:38, 0 of 1 keyframes kept
  54. Shot 53, 20:38 to 21:04, 0 of 1 keyframes kept
  55. Shot 54, 21:04 to 21:29, 0 of 1 keyframes kept
  56. Shot 55, 21:29 to 21:31, 1 of 1 keyframes kept
  57. Shot 56, 21:31 to 21:41, 1 of 1 keyframes kept
  58. Shot 57, 21:41 to 21:43, 1 of 1 keyframes kept
  59. Shot 58, 21:43 to 22:11, 1 of 1 keyframes kept
  60. Shot 59, 22:11 to 22:40, 0 of 1 keyframes kept
  61. Shot 60, 22:40 to 23:08, 0 of 1 keyframes kept
  62. Shot 61, 23:08 to 23:19, 1 of 1 keyframes kept
  63. Shot 62, 23:19 to 23:32, 1 of 1 keyframes kept
  64. Shot 63, 23:32 to 23:34, 1 of 1 keyframes kept
  65. Shot 64, 23:34 to 23:44, 1 of 1 keyframes kept
  66. Shot 65, 23:44 to 23:50, 1 of 1 keyframes kept
  67. Shot 66, 23:50 to 23:53, 0 of 1 keyframes kept
  68. Shot 67, 23:53 to 23:57, 0 of 1 keyframes kept
  69. Shot 68, 23:57 to 24:08, 0 of 1 keyframes kept
  70. Shot 69, 24:08 to 24:37, 1 of 1 keyframes kept
  71. Shot 70, 24:37 to 25:05, 0 of 1 keyframes kept
  72. Shot 71, 25:05 to 25:07, 1 of 1 keyframes kept
  73. Shot 72, 25:07 to 25:33, 1 of 1 keyframes kept
  74. Shot 73, 25:33 to 25:58, 1 of 1 keyframes kept
  75. Shot 74, 25:58 to 26:24, 1 of 1 keyframes kept
  76. Shot 75, 26:24 to 26:50, 1 of 1 keyframes kept
  77. Shot 76, 26:50 to 27:17, 1 of 1 keyframes kept
  78. Shot 77, 27:17 to 27:45, 0 of 1 keyframes kept
  79. Shot 78, 27:45 to 28:13, 0 of 1 keyframes kept
  80. Shot 79, 28:13 to 28:20, 1 of 1 keyframes kept
  81. Shot 80, 28:20 to 29:09, 1 of 1 keyframes kept
  82. Shot 81, 29:09 to 29:11, 1 of 1 keyframes kept
  83. Shot 82, 29:11 to 29:45, 1 of 1 keyframes kept
  84. Shot 83, 29:45 to 30:19, 0 of 1 keyframes kept
  85. Shot 84, 30:19 to 30:20, 1 of 1 keyframes kept
  86. Shot 85, 30:20 to 30:41, 1 of 1 keyframes kept
  87. Shot 86, 30:41 to 30:46, 1 of 1 keyframes kept
  88. Shot 87, 30:46 to 31:02, 1 of 1 keyframes kept
  89. Shot 88, 31:02 to 31:21, 1 of 1 keyframes kept
  90. Shot 89, 31:21 to 31:38, 1 of 1 keyframes kept
  91. Shot 90, 31:38 to 31:50, 1 of 1 keyframes kept
  92. Shot 91, 31:50 to 31:58, 1 of 1 keyframes kept
  93. Shot 92, 31:58 to 32:03, 1 of 1 keyframes kept
  94. Shot 93, 32:03 to 32:39, 1 of 1 keyframes kept
  95. Shot 94, 32:39 to 32:41, 1 of 1 keyframes kept
  96. Shot 95, 32:41 to 32:44, 1 of 1 keyframes kept
  97. Shot 96, 32:44 to 33:12, 1 of 1 keyframes kept
  98. Shot 97, 33:12 to 33:39, 1 of 1 keyframes kept
  99. Shot 98, 33:39 to 34:06, 1 of 1 keyframes kept
  100. Shot 99, 34:06 to 34:34, 1 of 1 keyframes kept
  101. Shot 100, 34:34 to 35:01, 1 of 1 keyframes kept
  102. Shot 101, 35:01 to 35:28, 1 of 1 keyframes kept
  103. Shot 102, 35:28 to 35:55, 1 of 1 keyframes kept
  104. Shot 103, 35:55 to 36:23, 1 of 1 keyframes kept
  105. Shot 104, 36:23 to 36:50, 1 of 1 keyframes kept
  106. Shot 105, 36:50 to 37:05, 1 of 1 keyframes kept
  107. Shot 106, 37:05 to 37:11, 1 of 1 keyframes kept
  108. Shot 107, 37:11 to 37:37, 1 of 1 keyframes kept
  109. Shot 108, 37:37 to 38:03, 0 of 1 keyframes kept
  110. Shot 109, 38:03 to 38:04, 1 of 1 keyframes kept
  111. Shot 110, 38:04 to 38:20, 1 of 1 keyframes kept
  112. Shot 111, 38:20 to 38:56, 1 of 1 keyframes kept
  113. Shot 112, 38:56 to 39:32, 1 of 1 keyframes kept
  114. Shot 113, 39:32 to 39:50, 1 of 1 keyframes kept
  115. Shot 114, 39:50 to 39:59, 1 of 1 keyframes kept
  116. Shot 115, 39:59 to 40:04, 1 of 1 keyframes kept
  117. Shot 116, 40:04 to 40:41, 1 of 1 keyframes kept
  118. Shot 117, 40:41 to 41:13, 1 of 1 keyframes kept
  119. Shot 118, 41:13 to 41:45, 0 of 1 keyframes kept
  120. Shot 119, 41:45 to 42:06, 1 of 1 keyframes kept
  121. Shot 120, 42:06 to 42:31, 1 of 1 keyframes kept
  122. Shot 121, 42:31 to 42:56, 0 of 1 keyframes kept
  123. Shot 122, 42:56 to 43:40, 1 of 1 keyframes kept
  124. Shot 123, 43:40 to 44:00, 1 of 1 keyframes kept
  125. Shot 124, 44:00 to 44:21, 1 of 1 keyframes kept
  126. Shot 125, 44:21 to 44:31, 1 of 1 keyframes kept
  127. Shot 126, 44:31 to 44:58, 1 of 1 keyframes kept
  128. Shot 127, 44:58 to 45:25, 1 of 1 keyframes kept
  129. Shot 128, 45:25 to 45:53, 1 of 1 keyframes kept
  130. Shot 129, 45:53 to 46:20, 1 of 1 keyframes kept
  131. Shot 130, 46:20 to 46:47, 1 of 1 keyframes kept
  132. Shot 131, 46:47 to 46:56, 1 of 1 keyframes kept
  133. Shot 132, 46:56 to 47:01, 1 of 1 keyframes kept
  134. Shot 133, 47:01 to 47:02, 0 of 1 keyframes kept
  135. Shot 134, 47:02 to 47:09, 1 of 1 keyframes kept
  136. Shot 135, 47:09 to 47:12, 1 of 1 keyframes kept
  137. Shot 136, 47:12 to 47:15, 1 of 1 keyframes kept
  138. Shot 137, 47:15 to 47:21, 1 of 1 keyframes kept
  139. Shot 138, 47:21 to 47:34, 1 of 1 keyframes kept
  140. Shot 139, 47:34 to 47:50, 1 of 1 keyframes kept
  141. Shot 140, 47:50 to 47:53, 1 of 1 keyframes kept
  142. Shot 141, 47:53 to 47:59, 1 of 1 keyframes kept
  143. Shot 142, 47:59 to 48:14, 1 of 1 keyframes kept
  144. Shot 143, 48:14 to 48:21, 1 of 1 keyframes kept
  145. Shot 144, 48:21 to 48:27, 1 of 1 keyframes kept
  146. Shot 145, 48:27 to 48:54, 1 of 1 keyframes kept
  147. Shot 146, 48:54 to 48:55, 1 of 1 keyframes kept
  148. Shot 147, 48:55 to 49:13, 1 of 1 keyframes kept
  149. Shot 148, 49:13 to 49:27, 1 of 1 keyframes kept
  150. Shot 149, 49:27 to 49:30, 1 of 1 keyframes kept
  151. Shot 150, 49:30 to 49:34, 1 of 1 keyframes kept
  152. Shot 151, 49:34 to 49:36, 1 of 1 keyframes kept
  153. Shot 152, 49:36 to 49:51, 1 of 1 keyframes kept
  154. Shot 153, 49:51 to 50:04, 1 of 1 keyframes kept
  155. Shot 154, 50:04 to 50:06, 1 of 1 keyframes kept
  156. Shot 155, 50:06 to 50:27, 1 of 1 keyframes kept
  157. Shot 156, 50:27 to 50:29, 1 of 1 keyframes kept
  158. Shot 157, 50:29 to 51:08, 1 of 1 keyframes kept
  159. Shot 158, 51:08 to 51:11, 1 of 1 keyframes kept
  160. Shot 159, 51:11 to 51:12, 0 of 1 keyframes kept
  161. Shot 160, 51:12 to 51:15, 1 of 1 keyframes kept
  162. Shot 161, 51:15 to 51:25, 0 of 1 keyframes kept
  163. Shot 162, 51:25 to 51:51, 1 of 1 keyframes kept
  164. Shot 163, 51:51 to 52:07, 1 of 1 keyframes kept

164 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
596
whisperx 596
chunks
92
from 596 cues
keyframes
130
kept of 164 captured
frames with text
130
6,091 lines read
chapters
0
from the source metadata
keyframe bytes
18.6 MB
word timings on 596 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-10 23:09 1m 05s
stt done — 2026-08-10 23:10 59s
chunk done — 2026-08-10 23:11 0s
text_embed done — 2026-08-10 23:11 1s
keyframe done — 2026-08-10 23:11 1m 38s
ocr done — 2026-08-10 23:12 2m 21s
frame_embed done — 2026-08-10 23:15 22s

Frames, and what the machine read

  • 0:08 #0 done8 line(s)

    shot 0·sharpness 1325.1

    1. ANTHROPIC0.99
    2. Cline0.99
    3. Google DeepMind0.97
    4. HumanLayer1.00
    5. arize1.00
    6. replit1.00
    7. Google DeepMind1.00
    8. CURSOR1.00

  • 0:15 #1 done2 line(s)

    shot 1·sharpness 686.2

    1. PRESENTING SPONSOR0.99
    2. Google DeepMind0.99

  • 0:17 #2 done2 line(s)

    shot 2·sharpness 498.2

    1. PLATINUMSPONSOR1.00
    2. ANTHROP\C1.00

  • 0:43 #3 done7 line(s)

    shot 3·sharpness 694.3

    1. arize1.00
    2. Applied Prompt Learning:0.98
    3. Building a Eval-Driven1.00
    4. Optimization Loop1.00
    5. SallyAnn DeLucia & Fuad Ali0.98
    6. 2025-11-22 12:33:590.99
    7. JFK27-B1.3001.00

  • 1:14 #4 done8 line(s)

    shot 4·sharpness 697.8

    1. arize1.00
    2. Applied Prompt Learning0.99
    3. Building a Eval-Driven1.00
    4. Optimization Loop1.00
    5. SallyAnn DeLucia & Fuad Ali0.99
    6. a0.70
    7. 2025-11-22 12:34:300.97
    8. JFK27-B1.3001.00

  • 1:24 #5 skipped

    shot 5·duplicate of #4

  • 1:49 #6 done17 line(s)

    shot 6·sharpness 558.6

    1. Agenda1.00
    2. /010.98
    3. /020.98
    4. /030.98
    5. Why Agents Fail0.99
    6. What Is Prompt1.00
    7. Case Study: Coding0.98
    8. Today1.00
    9. Learning?1.00
    10. Agents1.00
    11. 1040.89
    12. /050.98
    13. Prompt Learning vs0.98
    14. Workshop1.00
    15. GEPA1.00
    16. 2025-11-2212:35:051.00
    17. JFK27-B1.3001.00

  • 2:00 #7 done16 line(s)

    shot 7·sharpness 1096.3

    1. Agenda1.00
    2. /010.99
    3. /020.97
    4. /030.98
    5. Why Agents Fail1.00
    6. What Is Prompt1.00
    7. Case Study: Coding1.00
    8. Today1.00
    9. Learning?1.00
    10. Agents1.00
    11. /040.98
    12. /050.96
    13. Prompt Learning vs0.99
    14. Workshop1.00
    15. GEPA1.00
    16. 2025-11-22 12:35:150.99

  • 2:17 #8 done16 line(s)

    shot 8·sharpness 565.0

    1. Agenda1.00
    2. /010.97
    3. /030.92
    4. Why Agents Fail0.99
    5. What Is Prompt1.00
    6. Case Study: Coding0.98
    7. Today1.00
    8. Learning?1.00
    9. Agents1.00
    10. /040.96
    11. /050.98
    12. Prompt Learning vs1.00
    13. Workshop1.00
    14. GEPA1.00
    15. 2025-11-22 12:35:330.97
    16. JFK27-B1.3001.00

  • 2:47 #9 done12 line(s)

    shot 9·sharpness 1969.7

    1. Where Agents are Breaking in 20251.00
    2. No System Instructions Learned1.00
    3. No Planning or1.00
    4. Missing Tools0.98
    5. Very Static Planning1.00
    6. From Environment0.98
    7. Tool Guidance1.00
    8. Missing Context / State0.99
    9. Management1.00
    10. (Pre Pruned Data)1.00
    11. 2025-11-2212:36:031.00
    12. arizeWe Make Al Work0.99

  • 3:41 #10 done18 line(s)

    shot 10·sharpness 1473.2

    1. Core Issues Distilled1.00
    2. Adaptability & Self1.00
    3. Determinism vs1.00
    4. Context1.00
    5. Learning1.00
    6. Non Determinism1.00
    7. Engineering1.00
    8. Balance1.00
    9. No System Instructions1.00
    10. No Planning or1.00
    11. Missing Tools1.00
    12. Very Static Planning1.00
    13. Tool Guidance0.99
    14. Learned From Environment1.00
    15. Missing Context0.99
    16. (Pre Pruned Data)1.00
    17. 2025-11-22 12:36:560.97
    18. arizeWe Make Al Work0.98

  • 4:12 #11 done21 line(s)

    shot 11·sharpness 1198.0

    1. 1 Other Issue l'd Like to Mention0.98
    2. Technical Users1.00
    3. Domain Experts0.97
    4. Al Engineer0.96
    5. Data1.00
    6. 1010101011.00
    7. Scientist1.00
    8. Subject Matter1.00
    9. Al Product1.00
    10. Experts1.00
    11. Manager1.00
    12. Developer1.00
    13. Responsibilities1.00
    14. Responsibilities1.00
    15. Code/Automation1.00
    16. Domain Prompt engineering1.00
    17. Pipelines/Frameworks1.00
    18. Track and run evals0.99
    19. Application Performance / Costs0.99
    20. Ensure product success1.00
    21. 2025-11-22 12:37:280.98

  • 4:36 #12 done21 line(s)

    shot 12·sharpness 659.5

    1. 1 Other Issue l'd Like to Mention0.98
    2. Technical Users1.00
    3. Domain Experts0.99
    4. Al Engineer1.00
    5. Data1.00
    6. Scientist1.00
    7. Subject Matter1.00
    8. Al Product0.99
    9. Experts1.00
    10. Manager1.00
    11. Developer0.98
    12. Responsibilities1.00
    13. Responsibilities1.00
    14. Code/Automation0.99
    15. Domain Prompt engineering0.99
    16. Pipelines/Frameworks1.00
    17. Track and run evals0.98
    18. Application Performance / Costs0.98
    19. Ensure product success1.00
    20. 2025-11-2212:37:521.00
    21. JFK27-B1.3001.00

  • 4:51 #13 skipped

    shot 13·duplicate of #12

  • 5:32 #14 done15 line(s)

    shot 14·sharpness 1547.8

    1. Reinforcement Learning0.99
    2. RL Model1.00
    3. Action1.00
    4. Reward Function1.00
    5. (Student's Brain)1.00
    6. (Takes Exam)1.00
    7. (Exam Scorer)1.00
    8. Update Weights1.00
    9. Scalar Reward1.00
    10. (Student's Brain)1.00
    11. (Exam Score)0.98
    12. Algorithm: Gradient Descent, PPO,1.00
    13. Q-learning1.00
    14. 2025-11-22 12:38:480.99
    15. arizeWe Make Models Work0.99

  • 6:16 #15 done16 line(s)

    shot 15·sharpness 1662.0

    1. Meta Prompting - Almost...0.99
    2. Meta Prompting - Ask an LLM to improve your prompt0.99
    3. Agent1.00
    4. Output1.00
    5. Scorer1.00
    6. (Student)1.00
    7. (Takes Exam)1.00
    8. (Exam Scorer)1.00
    9. Update Prompt1.00
    10. Scalar Reward0.99
    11. (Lessons, HWs)0.99
    12. (Exam Score)1.00
    13. Algorithm: Meta-Prompting1.00
    14. (Teaching)1.00
    15. 2025-11-22 12:39:320.98
    16. arizeWe Make Models Work0.98

  • 6:39 #16 done20 line(s)

    shot 16·sharpness 1275.3

    1. Prompt Learning1.00
    2. Agent1.00
    3. Output1.00
    4. LLM Evals1.00
    5. (Student)1.00
    6. (Takes Exam)1.00
    7. (Teacher)1.00
    8. English Feedback1.00
    9. Update Prompt1.00
    10. • which answers0.95
    11. (Lessons, HWs)1.00
    12. were wrong1.00
    13. WHY answers1.00
    14. were wrong1.00
    15. Algorithm: Meta-Prompting1.00
    16. WHERE student1.00
    17. (Teaching)1.00
    18. needs to study1.00
    19. 2025-11-22 12:39:550.99
    20. arize We Make Models Work0.95

  • 7:30 #17 done12 line(s)

    shot 17·sharpness 1018.3

    1. Traditional Prompt Optimization1.00
    2. Formulated Like an ML Problem1.00
    3. Data1.00
    4. Prediction1.00
    5. Prompt1.00
    6. Labels1.00
    7. X0.87
    8. 二0.70
    9. Optimize This1.00
    10. Maximize This0.98
    11. 2025-11-22 12:40:460.98
    12. arizeWe Make Models Work0.98

  • 7:53 #18 done19 line(s)

    shot 18·sharpness 1059.6

    1. System Prompt Learning1.00
    2. Data1.00
    3. Human Instrunctions,1.00
    4. Eval Explanations1.00
    5. Prompt1.00
    6. Prediction1.00
    7. Labels1.00
    8. Eval Explanations,1.00
    9. why Failed0.94
    10. Why it Failed0.97
    11. AI Why it Failed0.96
    12. +0.58
    13. 十0.51
    14. 十0.69
    15. +0.58
    16. Add Instructions or changes to System Prompt here,0.99
    17. to help it improve1.00
    18. 2025-11-22 12:41:091.00
    19. arizeWe Make Models Work0.97

  • 8:44 #19 done15 line(s)

    shot 19·sharpness 584.5

    1. System Prompt Learning1.00
    2. Data0.99
    3. Human Instrunetions,0.96
    4. Why it Faled0.90
    5. Eval Explanations0.99
    6. AI Why it Failed0.91
    7. Prompt0.98
    8. Prediction1.00
    9. Labels0.99
    10. Eval Explanations,1.00
    11. Why Failed0.97
    12. Add Instructions or changes to Systen Prompt here,0.98
    13. to help it improve0.97
    14. 2025-11-2212:42:001.00
    15. JFK27-B1.3001.00

  • 8:59 #20 done25 line(s)

    shot 20·sharpness 678.3

    1. Optimizing Coding Agents, just through their Prompts1.00
    2. Claude1.00
    3. cline0.95
    4. Claude Cade swstem prompt0.90
    5. Cine system prompt0.97
    6. You are a Cloude agent bulit on0.85
    7. You are Cline. a highly skilled0.95
    8. antheopie's Claude Apent S0K0.74
    9. softwore engineer with extensive0.98
    10. knowledge in mong pregranming0.89
    11. helps wsers with software0.89
    12. You are an interactive CLI tool that0.91
    13. patterns. and best peactices.0.92
    14. languages, fromenorks, design0.78
    15. engineering tasks. Use the0.91
    16. ovasloble to you to assist the user0.89
    17. instructions below and the toola0.97
    18. CLAUDE1.00
    19. Rules (--append-systen-prompt)0.89
    20. Rules (./clinerules)0.99
    21. <Empty>0.93
    22. <Empty>0.87
    23. CODE1.00
    24. 2025-11-2212:42:151.00
    25. JFK27-B1.3000.96

  • 9:19 #21 done29 line(s)

    shot 21·sharpness 1891.2

    1. Optimizing Coding Agents, just through their Prompts1.00
    2. 米Claude1.00
    3. Cline0.99
    4. The collaborative1.00
    5. coding agent1.00
    6. for complex work0.98
    7. Cline system prompt0.99
    8. Claude Code system prompt0.99
    9. You are a Claude agent, built on1.00
    10. You are Cline, a highly skilled1.00
    11. Anthropic's Claude Agent SDK.1.00
    12. software engineer with extensive0.99
    13. knowledge in many programming1.00
    14. You are an interactive CLI tool that1.00
    15. languages, frameworks, design0.99
    16. helps users with software0.99
    17. patterns, and best practices...1.00
    18. engineering tasks. Use the1.00
    19. instructions below and the tools1.00
    20. available to you to assist the user.0.99
    21. *Welcome to Claude Code0.99
    22. Rules (./clinerules)1.00
    23. Rules (--append-system-prompt)1.00
    24. <Empty>1.00
    25. <Empty>1.00
    26. Press Enter to continue1.00
    27. 2025-11-22 12:42:350.99
    28. arize1.00
    29. We Make Al Work0.97

  • 9:42 #22 done30 line(s)

    shot 22·sharpness 1869.9

    1. Coding Agents on SWE-Bench Lite, No Prompt Changes0.99
    2. CLAUDE1.00
    3. Cline1.00
    4. CODE0.99
    5. Sonnet 4-50.99
    6. GPT 4.10.99
    7. Sonnet 4-50.99
    8. Haiku4.51.00
    9. Cost: $3/1M tokens0.99
    10. Cost: $2/1M tokens0.97
    11. Cost: $3/1M tokens1.00
    12. Cost: $1/1M tokens1.00
    13. Latency:1.00
    14. Latency:1.00
    15. Latency:1.00
    16. Latency:1.00
    17. 30.00%1.00
    18. 18.67%1.00
    19. 40.00%1.00
    20. 18.67%1.00
    21. Github Issues0.98
    22. Github Issues0.96
    23. GithubIssues1.00
    24. GithubIssues1.00
    25. resolved1.00
    26. resolved1.00
    27. resolved1.00
    28. resolved1.00
    29. 2025-11-2212:42:581.00
    30. arizeWe Make Al Work0.93

  • 10:11 #23 done31 line(s)

    shot 23·sharpness 1985.3

    1. Optimizing Coding Agent System Prompt1.00
    2. Claude Code system prompt1.00
    3. OLD1.00
    4. Claude Code system prompt1.00
    5. NEW1.00
    6. You are a Claude agent, built on Anthropic's...0.99
    7. You are a Claude agent, built on Anthropic's...1.00
    8. Rules Section0.99
    9. Rules Section1.00
    10. <Empty>1.00
    11. 1.0.99
    12. When dealing with errors or exceptions,1.00
    13. consider the immediate cause and0.99
    14. underlying issues that may contribute1.00
    15. to the problem.1.00
    16. 2.0.98
    17. Ensure changes align with the overall1.00
    18. system design; avoid ad-hoc fixes that1.00
    19. introduce technical debt.1.00
    20. 3.0.97
    21. Any change should be accompanied by0.99
    22. appropriate tests, covering edge cases1.00
    23. and ensuring correctness and0.99
    24. robustness.1.00
    25. 4.1.00
    26. Always consider anomalies, None values,0.98
    27. and unexpected inputs when modifying1.00
    28. data flows.1.00
    29. 2025-11-22 12:43:260.97
    30. 5.0.99
    31. Ensure changes don't introduce0.99

Transcript

596 cues· 9,599 words· 51,149 chars

  1. 0:21 Hey, everyone.
  2. 0:21 Gonna get started here.
  3. 0:23 Thanks so much for joining us today.
  4. 0:25 I'm Sally Anne.
  5. 0:26 I'm the director of PROMPT at Arise.
  6. 0:28 I'm gonna be walking you through some of PROMPT learning.
  7. 0:30 We're actually gonna be building an algorithm optimization loop for the part of the workshop.
  8. 0:35 I have a particular background in data science.
  9. 0:38 Before I make my way over to product, I do like to still be touching code today.
  10. 0:43 I think one of my bigger projects that I work on is building our own agent into our platform.
  11. 0:48 So I'm very familiar with all of the pain points and how important it is to optimize your prompt.
  12. 0:53 So I'm going to just set the scene, make sure everybody here has context on what we're going to be doing, and then we'll jump into the code.
  13. 1:01 I love me, so I'll let you do a little bit of an intro.
  14. 1:03 Yeah, thank you so much.
  15. 1:04 Great to meet all of you.
  16. 1:05 Excited to be walking through prompt learning with you all.
  17. 1:09 I don't know if you got a chance to see our partners talk yesterday, but hopefully that gave you some good background on how powerful that prompting and prompt learning can be.
  18. 1:18 So my name is Will.
  19. 1:19 I'm a product manager here at Arise as well.
  20. 1:21 And like Sally said, we like to stay in code.
  21. 1:23 We'll be doing a few slides, then we'll walk through the code and we'll be floating around helping you guys debug and things like that.
  22. 1:29 My background is also
  23. 1:30 technical.
  24. 1:31 So I was back in distributed systems engineering for a long time.
  25. 1:34 So no stranger to how important observability infrastructure really is.
  26. 1:38 And I think it's an appropriate setting in AWS for that.
  27. 1:41 So yeah, excited to dive deep into front loading with you all.
  28. 1:44 Thank you.
  29. 1:46 So, all right, so we're going to get started.
  30. 1:48 Just give you a little bit of an agenda of the things I'm going to be covering.
  31. 1:50 So we're going to talk about why agents fail today, what is evening prompt learning.
  32. 1:54 I want to go through a case study, kind of show you all why this actually works.
  33. 1:58 And we'll talk about prompt learning versus GEPA.
  34. 1:59 I think every week I have a few people come up to me over the conference about, like, what about GEPA?
  35. 2:04 We have some benchmarking against that, and then we'll hop into our workshop.
  36. 2:08 But with this, I want to ask a question.
  37. 2:09 How many people here are building agents today?
  38. 2:12 Okay, that's what I expected.
  39. 2:14 And how many people actually feel like the agents they're building are reliable?
  40. 2:19 Yeah, that's what I also thought.
  41. 2:20 So let's talk a little bit about why agents fail today.
  42. 2:23 So why do they fail?
  43. 2:24 Well, there's a few things that we're seeing with a lot of our folks, and we're seeing even internally as we build with Alex for why agents are breaking.
  44. 2:31 So I think that a lot of times it's not because the models are weak.
  45. 2:35 It's a lot of times the environment and the instructions are weak.
  46. 2:39 So having no instructions from their learned environment
  47. 2:44 No planning or very static planning.
  48. 2:46 I feel like a lot of agents right now don't have planning.
  49. 2:48 We do have some good examples of planning, like we have Cloud Code, Cursor.
  50. 2:52 Those are really great examples, but I'm not seeing it make its way into every agent that I come across.

Open at this second