Videos JomVvNDjGb8

Your Coding Agent Should Do AI System Engineering — Ben Burtenshaw, Hugging Face

index_state ready data_status ok

AI Engineer· published 2026-05-21· 0:18:25· en-US· indexed 2026-08-10 19:55

Open on YouTube

93 shot(s).

keyframes kept every frame deduplicated

What was stored

cues: 213; whisperx 213
chunks: 33; from 213 cues
keyframes: 85; kept of 93 captured
frames with text: 81; 2,276 lines read
chapters: 11; from the source metadata
keyframe bytes: 10.2 MB; word timings on 213 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage	state	model	started	took
`fetch`	done	—	2026-08-10 18:27	1m 46s
`stt`	done	—	2026-08-10 18:29	21s
`chunk`	done	—	2026-08-10 18:29	0s
`text_embed`	done	—	2026-08-10 19:55	0s
`keyframe`	done	—	2026-08-10 18:29	2m 16s
`ocr`	done	—	2026-08-10 18:31	54s
`frame_embed`	done	—	2026-08-10 19:55	15s

Frames, and what the machine read

0:03 #0 done2 line(s)

shot 0·sharpness 658.4
1. AlEngineer0.98
2. EUROPE1.00
0:08 #1 done2 line(s)

shot 1·sharpness 827.9
1. PRESENTINGSPONSOR1.00
2. Google DeepMind1.00
0:12 #2 done3 line(s)

shot 2·sharpness 907.6
1. PLATINUM SPONSORS0.98
2. # Braintrust0.96
3. WorkOS OpenAI0.95
0:17 #3 done28 line(s)

shot 3·sharpness 927.7
1. ai sys0.97
2. AlEngineer0.99
3. OpenAl0.99
4. AlEngineer1.00
5. EUROPE0.98
6. EUROPE0.99
7. engin1.00
8. AlEngineer0.99
9. WorkOS1.00
10. AlEngineer0.99
11. IIElev0.89
12. EUROPE1.00
13. EUROPE0.99
14. △Trigger.dev0.94
15. Cnnala DeepMind0.79
16. AlEngineer0.99
17. EUROPE1.00
18. AlEngineer0.99
19. arize1.00
20. AlEngineer0.98
21. EUROPE0.99
22. EUROPE1.00
23. PRESENTED BY1.00
24. Google DeepMind1.00
25. Unblocked1.00
26. AIE1.00
27. AlEngineer0.98
28. EUROPE1.00
0:20 #4 done25 line(s)

shot 4·sharpness 802.2
1. Araintru your coding0.96
2. AIEngineer0.95
3. agent should do0.96
4. AIEngineer0.95
5. EURC0.91
6. EUROPE1.00
7. eer1.00
8. ai systems1.00
9. OpenA1.00
10. Google DeepMir0.96
11. eer1.00
12. engineering1.00
13. croso0.89
14. RY1.00
15. ATEngineer0.86
16. crosoft1.00
17. AIEngineer0.95
18. EUROPE1.00
19. EUROPE1.00
20. Google DeepMind1.00
21. AlEngineer0.96
22. EUROPE1.00
23. Modal1.00
24. ineer1.00
25. BeNCORD0.95
0:22 #5 done11 line(s)

shot 5·sharpness 1848.8
1. your coding0.96
2. Braintrus1.00
3. agent should do1.00
4. eer1.00
5. ai systems0.97
6. engineering1.00
7. croso1.00
8. neer1.00
9. Google DeepMind1.00
10. AlEngineer0.94
11. EUROPE1.00
0:30 #6 done13 line(s)

shot 6·sharpness 2349.1
1. Takeaways1.00
2. Praintru0.90
3. Fun part: We can use code agents to do Al1.00
4. Sys Engineering / ML Engineering0.99
5. eer1.00
6. Boring part: We need standard repos and1.00
7. primitives to know what's going on0.99
8. cros1.00
9. 10.76
10. ineer1.00
11. Google DeepMind1.00
12. AlEngineer0.98
13. EUROPE1.00
0:41 #7 done19 line(s)

shot 7·sharpness 2404.5
1. eepMind1.00
2. AlEngineer0.97
3. Takeaways1.00
4. EUROPE0.93
5. ineer0.96
6. ATrige0.79
7. Zed0.94
8. Fun part: We can use code agents to do Al0.99
9. Sys Engineering / ML Engineering0.99
10. gineer0.91
11. o4j0.95
12. AIE1.00
13. Boring part: We need standard repos and0.99
14. primitives to know what's going on1.00
15. AlEngineer0.99
16. # Braintrust0.96
17. WorkOS1.00
18. OpenAI0.93
19. EUROPE1.00
0:49 #8 done47 line(s)

shot 8·sharpness 2340.1
1. eepMind0.99
2. AlEngineer1.00
3. EUROPE0.99
4. Coding Agents0.97
5. jineer0.91
6. ΔTrigger.dev0.89
7. Zed0.99
8. AlEngine0.97
9. EUROPE0.93
10. I was inspired by this so I wanted to see if Claude Code can get into my0.99
11. Andrej Karpathy@karpathy - Dec 28, 20250.93
12. Q0.90
13. "I also get some programmers are eager to tune it out. The hype drones on,0.98
14. DHH@dhh Jan 70.92
15. Lutron home automation system.1.00
16. the fantastical claims are still far off, and there's uncertainty where this0.99
17. leaves the profession. But that's not reason to miss out on this incredible0.99
18. gineer0.99
19. come0.99
20. - it found my Lutron controllers on the local wifi network0.98
21. - checked for open ports, connected, got some metadata and identified the1.00
22. moment in human history!"0.98
23. o4j0.93
24. AlEngineer1.00
25. shades, HVAC temperature control, motion sensors etc.)0.99
26. - searched the internet, found the pdf for my system1.00
27. - instructed me on what button to press to pair and get the certificates1.00
28. - it connected to the system and found all the home devices (lights,0.98
29. devices and their firmware0.98
30. Promoting Al agents1.00
31. me. Partly because the models got better, but mor...0.99
32. At the end of last year, Al agents really came alive for0.99
33. world.hey.com1.00
34. - it turned on and off my kitchen lights to check that things are working0.99
35. (lol!)0.96
36. 601.00
37. t7 1220.86
38. 1K0.69
39. I0.98
40. l 150K0.82
41. 口0.66
42. have been accepted.1.00
43. AlEngineer0.97
44. Braintrust1.00
45. WorkOS1.00
46. OpenAl0.92
47. EUROPE1.00
1:07 #9 done28 line(s)

shot 9·sharpness 3026.6
1. Coding Agents1.00
2. Andrej Karpathy@karpathy·Dec 28, 20250.97
3. DHH@dhh Jan70.96
4. I was inspired by this so I wanted to see if Claude Code can get into my0.98
5. "I also get some programmers are eager to tune it out. The hype drones on,0.99
6. Lutron home automation system.1.00
7. the fantastical claims are still far off, and there's uncertainty where this0.99
8. leaves the profession. But that's not reason to miss out on this incredible0.99
9. - it found my Lutron controllers on the local wifi network0.99
10. moment in human history!"0.99
11. - checked for open ports, connected, got some metadata and identified the0.99
12. - searched the internet, found the pdf for my system0.98
13. - instructed me on what button to press to pair and get the certificates0.99
14. devices and their firmware0.99
15. world.hey.com1.00
16. Promoting Al agents1.00
17. - it connected to the system and found all the home devices (lights,0.99
18. At the end of last year, Al agents really came alive for1.00
19. shades, HVAC temperature control, motion sensors etc.)1.00
20. me. Partly because the models got better, but mor...0.98
21. - it turned on and off my kitchen lights to check that things are working0.99
22. (lol!)0.92
23. 601.00
24. t1220.99
25. 1K0.98
26. I0.94
27. 150K1.00
28. have been accepted.1.00
1:19 #10 skipped

shot 10·duplicate of #9
1:23 #11 done7 line(s)

shot 11·sharpness 2088.2
1. the bosses1.00
2. 1. write custom kernels1.00
3. interactively with an agent1.00
4. 2. yolo an agent to finetune1.00
5. an LLM0.96
6. 3. setup a multi-agent0.98
7. autoreasearch lab1.00
1:36 #12 done18 line(s)

shot 12·sharpness 2565.1
1. Mind1.00
2. AlEngineer0.92
3. BUROPE0.99
4. the bosses1.00
5. A Triggerd0.87
6. 1. write custom kernels0.97
7. AIEr0.88
8. interactively with an agent1.00
9. 2. yolo an agent to finetune1.00
10. COI0.87
11. an LLM1.00
12. 3. setup a multi-agent0.98
13. AlEngin1.00
14. EUROPE0.98
15. autoreasearch lab1.00
16. Google DeepMind1.00
17. AlEngineer0.98
18. EUROPE1.00
1:54 #13 done11 line(s)

shot 13·sharpness 2522.6
1. the bosses1.00
2. 1. write custom kernels0.99
3. interactively with an agent1.00
4. 2. yolo an agent to finetune1.00
5. an LLM1.00
6. 3. setup a multi-agent0.97
7. autoreasearch lab1.00
8. # Braintrust0.97
9. WorkOS OpenAI0.92
10. AlEngineer0.98
11. EUROPE1.00
1:59 #14 done13 line(s)

shot 14·sharpness 1597.4
1. Microsoft1.00
2. BOSS 10.99
3. OpenA1.00
4. AlEngine0.92
5. Son0.99
6. stripe0.92
7. Snork1.00
8. human + coding agent1.00
9. Ägent Skills for Kernels0.99
10. AlEngineer1.00
11. # Braintrust0.96
12. WorkOS OpenAI0.96
13. EUROPE1.00
2:00 #15 done13 line(s)

shot 15·sharpness 1495.2
1. Microsoft0.97
2. BOSS 10.98
3. OpenAl0.94
4. NEng0.95
5. NEng0.89
6. Sonar0.99
7. stripe1.00
8. Snorkel0.99
9. human + coding agent1.00
10. Ägent Skills for Kernels0.99
11. Engineering the future of Al1.00
12. AlEngineer0.99
13. EUROPE1.00
2:20 #16 done8 line(s)

shot 16·sharpness 1339.3
1. onAl0.92
2. CLAUDE CODE0.99
3. DOES NOT WRITE CUDA KERNELS0.99
4. ust0.99
5. imgflip.com0.94
6. Google DeepMind0.97
7. AlEngineer0.99
8. EUROPE1.00
2:32 #17 done15 line(s)

shot 17·sharpness 1478.1
1. Mind0.85
2. AlEngineer0.99
3. BUROPE0.95
4. ATrigger.de0.92
5. CLAUDE CODE0.99
6. AlEngine1.00
7. BUROPE0.91
8. con1.00
9. AlEngineer1.00
10. EUROPE0.96
11. DOES NOT WRITE CUDA KERNELS0.99
12. imgflip.com0.94
13. Google DeepMind0.99
14. AlEngineer0.99
15. EUROPE1.00
2:58 #18 done26 line(s)

shot 18·sharpness 1696.2
1. ai1.00
2. AlEngineer0.98
3. EUROPE0.96
4. WTF is a kernel?1.00
5. AIE1.00
6. Your Python code1.00
7. GPU hardware1.00
8. Runs on CPU1.00
9. Thousands of threads1.00
10. Mic1.00
11. A function compiled to run0.99
12. torch.matmul(A, B)1.00
13. Kernel dispatch1.00
14. on a GPU typically1.00
15. executed from Python.0.98
16. F.softmax(x)1.00
17. AlEngin0.99
18. matmul_kernel0.98
19. EUROPE0.99
20. Compiled for GPU1.00
21. F.gelu(x)1.00
22. AlEngineer0.99
23. Braintrust1.00
24. WorkOS1.00
25. OpenAl0.93
26. EUROPE1.00
3:36 #19 done14 line(s)

shot 19·sharpness 1473.4
1. WTF is a kernel?1.00
2. Your Python code1.00
3. GPU hardware1.00
4. Runs on CPU1.00
5. Thousands of threads1.00
6. A function compiled to run0.98
7. torch.matmul(A, B)1.00
8. Kernel dispatch1.00
9. on a GPU typically1.00
10. executed from Python.1.00
11. F.softmax(x)1.00
12. matmul_kernel1.00
13. Compiled for GPU1.00
14. F.gelu(x)1.00
3:55 #20 done18 line(s)

shot 20·sharpness 2216.1
1. Efficiency in Deep Learning1.00
2. Gor0.88
3. DeepMin0.95
4. Compute1.00
5. Memory1.00
6. Overhead1.00
7. Time spent in computing1.00
8. Time spent in communicating1.00
9. Everything else1.00
10. actual FLOPs1.00
11. tensors1.00
12. :OS0.61
13. Hugging Face1.00
14. AlEngineer0.99
15. # Braintrust0.98
16. WorkOS1.00
17. OpenAl0.96
18. EUROPE1.00
4:22 #21 done19 line(s)

shot 21·sharpness 2275.2
1. Zed0.97
2. Efficiency in Deep Learning1.00
3. WorkOS0.91
4. AlEngine0.90
5. arize1.00
6. neo4j0.83
7. Compute1.00
8. Memory1.00
9. Overhead1.00
10. Modal1.00
11. Time spent in computing0.99
12. Time spent in communicating1.00
13. Everything else1.00
14. actual FLOPs1.00
15. tensors1.00
16. Hugging Face0.96
17. Google DeepMind1.00
18. AlEngineer0.98
19. EUROPE1.00
4:35 #22 done15 line(s)

shot 22·sharpness 2061.8
1. Efficiency in Deep Learning1.00
2. rkOS0.87
3. Compute1.00
4. Memory1.00
5. Overhead1.00
6. Time spent in computing1.00
7. Time spent in communicating1.00
8. Everything else1.00
9. actual FLOPs1.00
10. tensors1.00
11. Jst0.96
12. Hugging Face1.00
13. Google DeepMind0.99
14. AlEngineer0.99
15. EUROPE1.00
4:49 #23 done15 line(s)

shot 23·sharpness 1718.5
1. flash-attention1.00
2. Flash Attention custom kernel1.00
3. Increase arithmetic density0.99
4. 21.9k1.00
5. 2.3k1.00
6. 1451.00
7. Spend less time in communicating tensors0.99
8. Keep the GPUs warm0.97
9. Br.0.82
10. AIE1.00
11. AlEngineer0.99
12. Braintrust0.97
13. WorkOS1.00
14. OpenAI0.93
15. EUROPE1.00

Transcript

213 cues· 3,452 words· 18,282 chars

chunk 0 · 0:15–0:55 · 130 words · 704 chars
0:15 Hi, everyone.
0:16 As you heard, I'm Ben from Hugging Face.
0:18 And the talk that I'm going to present to you today is called Your Coding Agent Should Do AI Systems Engineering.
0:24 So there are two main takeaways that I want you to get from this talk.
0:29 One, and probably the fun part, is that we can use coding agents to tackle the hardest engineering problems in AI, so systems engineering and machine learning engineering.
0:37 And maybe the boring part is that in order to do this, we're gonna need standard repos, and we're gonna need those on the hub.
0:44 And in many cases, we already have them.
chunk 1 · 0:47–1:30 · 129 words · 700 chars
0:47 So I think in this case, I'm preaching to the choir here, but in case you haven't noticed, coding agents have been accepted.
0:55 Many of us have been using them for a few years, but in the last few months, they seem to have crossed a sort of acceptance gradient where a broader group of people are using them.
1:05 So with this in mind, how do we keep our careers, our engineering kind of contemporary?
1:11 And how do we keep challenging ourselves in new areas?
1:14 And my proposal is that we need to go kind of closer to the silicon and tackle harder problems.
chunk 2 · 1:20–2:01 · 101 words · 559 chars
1:20 And that's where AI systems engineering comes in.
1:23 I've broken this talk down into three progressively more complex steps and more autonomous steps as well.
1:30 And I've defined those like three bosses from games.
1:34 The first one is a hybrid approach where you interactively use an agent to write a CUDA kernel.
1:43 The second is a zero shot task where an agent takes a prompt and trains an LLM.
1:50 on Hugging Face.
chunk 3 · 1:51–2:23 · 82 words · 469 chars
1:51 The third is a multi-agent auto research setup, like a kind of automated AI lab.
1:57 So let's get started on the first boss, right?
2:00 This is writing CUDA kernels.
2:02 So for a while writing custom kernels was seen as this unattainable goal for the humble agent.
2:09 They required complex DSLs, they required integration with relevant hardware to be benchmarked and to be tested and it was seen as something that couldn't be achieved by agents.
chunk 4 · 2:21–3:02 · 132 words · 708 chars
2:21 However, that in most cases was wrong.
2:23 If you look at kernel hackathons like those on GPU mode, the recent AMD hackathon, if you look at papers like kernel bench, you'll see that agents are able to write valid and optimized CUDA kernels.
2:37 And that's really cool and something that totally inspires me.
2:40 I'm a part of GPU mode, I contribute to that, and something that I think everyone should be doing.
2:45 However, what do we do with them?
2:47 How do we distribute them and how do we get them into our inference engine so that we actually are using these optimized kernels that we're generating?
chunk 5 · 2:54–3:33 · 120 words · 640 chars
2:54 And that's part of the question of this part of the talk.
2:58 Let's take a step back now and just say what a kernel is, right?
3:02 So when you run an AI model on a GPU, the actual work is executed through a kernel.
3:09 This will be defined in a relevant language for that hardware, and it will use relevant features to that hardware that may not be available on other hardware.
3:18 We can write custom kernels that will take advantage of that hardware for a specific math operation, kind of squeeze everything we can out of it so that the model will infer faster.
chunk 6 · 3:29–4:09 · 108 words · 629 chars
3:29 In general, this requires a lot of expertise about writing CUDA kernels, about the hardware.
3:34 And it's also a bit of an insulation hell as you deal with a pretty large install matrix from hardware to software to generations and versions of, say, CUDA and these kind of issues.
3:44 So in short, it's hard.
3:48 Efficiency in deep learning, so efficiency in kernels, is split into three main sections.
3:53 One, compute.
3:54 Two, memory.
3:54 And three, overhead.
3:56 Compute is the flops.
3:58 These are the matrix multiplications and the real math of the process.
chunk 7 · 4:02–4:45 · 114 words · 664 chars
4:02 Memory is the time spent moving data or tensors around memory, typically from slow to fast memory.
4:09 And overhead is basically everything else, the Python environment, PyTorch dispatch of those kernels, these kinds of things.
4:15 In general, most people might assume that the compute is the bottleneck here because it's doing most of the math, right?
4:24 That's not correct.
4:25 In most cases, memory is usually the bottleneck.
4:29 And that's because a modern GPU, let's take a H100 for example, can do a petaflop a second of computation, but its memory bandwidth is three terabytes.
chunk 8 · 4:38–5:21 · 129 words · 696 chars
4:38 So in short, the GPU is often waiting idle for these tensors to come back for them to be computed.

Chapters

0:00 Introduction to AI Systems Engineering
1:59 Boss 1: Writing and Distributing CUDA Kernels
3:48 Efficiency in Deep Learning
6:08 Using Skills for Agentic Workflows
8:37 Benchmarking and Evaluating Skills with Upskill
9:26 Boss 2: End-to-End Fine-tuning of LLMs
10:16 Boss 3: Multi-Agent Auto Research Labs
12:09 Architecture of the Multi-Agent Research System
13:40 Implementing the Research Agent in OpenCode
15:28 Monitoring Experiments with Trackio
16:45 Final Takeaways and Conclusion