Videos Sir59K8ZDPU

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

index_state ready data_status ok

AI Engineer· published 2026-07-23· 0:21:18· en-US· indexed 2026-08-10 19:34

Open on YouTube

51 shot(s).

keyframes kept every frame deduplicated

What was stored

cues: 241; whisperx 241
chunks: 37; from 241 cues
keyframes: 22; kept of 51 captured
frames with text: 22; 480 lines read
chapters: 11; from the source metadata
keyframe bytes: 6.4 MB; word timings on 241 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage	state	model	started	took
`fetch`	done	—	2026-08-09 20:59	0s
`stt`	done	—	2026-08-08 21:09	31s
`chunk`	done	—	2026-08-08 21:09	0s
`text_embed`	done	—	2026-08-10 19:33	0s
`keyframe`	done	—	2026-08-08 21:09	3m 35s
`ocr`	done	—	2026-08-08 21:13	14s
`frame_embed`	done	—	2026-08-10 19:33	4s

Frames, and what the machine read

0:02 #0 done2 line(s)

shot 0·sharpness 452.6
1. AlEngineer0.96
2. World's Fair0.98
0:03 #1 done2 line(s)

shot 1·sharpness 660.2
1. AlEngineer0.95
2. World's Fair0.99
0:10 #2 done24 line(s)

shot 2·sharpness 2733.8
1. LAB & PLATINUM SPONSORS0.99
2. Amazon AGI Lab0.98
3. ANTHROP\C1.00
4. Google DeepMind1.00
5. MINIMAX0.95
6. OpenAI0.92
7. Akamai1.00
8. arize1.00
9. aws1.00
10. Braintrust bright data0.97
11. B1.00
12. Browserbase1.00
13. docker1.00
14. :neo4j0.93
15. ORACLE1.00
16. PayPal1.00
17. qodo1.00
18. reducto1.00
19. Sonar1.00
20. Makers of1.00
21. togetherai1.00
22. Unblocked1.00
23. WorkOS1.00
24. SonarQube1.00
0:23 #3 done13 line(s)

shot 3·sharpness 1727.5
1. AlEngineer0.97
2. World'sFair1.00
3. AI ENGINEER WORLD'S FAIR · GRAPHS & ONTOLOGIES0.97
4. Why Agentic Systems0.98
5. PRESENTED BY1.00
6. Need Ontologies1.00
7. Microsoft1.00
8. Probabilistic reasoning inside. Logical guardrails outside.1.00
9. Frank P. Coyle, PhD1.00
10. University of California, Berkeley1.00
11. July 2,20260.99
12. Engineering the future of Al1.00
13. World'sFair1.00
0:43 #4 skipped

shot 4·duplicate of #3
1:12 #5 done24 line(s)

shot 5·sharpness 1748.2
1. AlEngineer0.98
2. World's Fair0.98
3. MY TEACHING PHILOSOPHY0.99
4. CC0.64
5. Nothing is a mistake.1.00
6. PRESENTED BY1.00
7. There's no win and no fail.0.99
8. Microsoft1.00
9. There's only make.0.99
10. Sister Corita Kent · Rule 6, Some Rules for Students and Teachers (1967)0.99
11. artist, educator, and nun — popularized by composer John Cage0.99
12. Experiment freely0.99
13. Build to understand1.00
14. 80.99
15. Iterate without fear0.99
16. Run the code, break it, leam from what0.98
17. We learn Al by making with it — not just0.99
18. No final grade on a first attempt; only the next1.00
19. happens.1.00
20. reading about it.1.00
21. version.1.00
22. TRACK 5· JULY 2,20260.96
23. Graphs1.00
24. World's Fair0.99
2:11 #6 skipped

shot 6·duplicate of #5
2:40 #7 done30 line(s)

shot 7·sharpness 2166.2
1. Two lineages0.98
2. AlEngineer1.00
3. World's Fair0.97
4. A probabilistic substrate and a symbolic one.0.99
5. AGENT1.00
6. ONTOLOGY1.00
7. the neural lineage1.00
8. the symbolic lineage1.00
9. origin: the philosophy of being — what exists, and how it's0.99
10. McCarthy (1956) — coins "artificial intelligence"0.97
11. categorized1.00
12. Selfridge (1959) — Pandemonium: competing "demons"0.98
13. Aristotle (~350 BC)1.00
14. Minsky (1986) — Society of Mind0.98
15. Categories — the kinds of being0.98
16. Quine (1948)1.00
17. Intelligence as many autonomous entities, not one monolith.0.99
18. "to be is to be the value of a bound variable"0.99
19. RUSSELL & NORVIG - the rational agent0.98
20. Gruber (1993)0.97
21. a formal spec of a shared conceptualization1.00
22. →0.80
23. PERCEIVE1.00
24. DECIDE1.00
25. ACT1.00
26. On a probabilistic neural net — proposes what is likely.0.99
27. Symbols + inference — declares what is permitted.0.99
28. TRACK 5· JULY 2, 20260.96
29. Graphs1.00
30. World's Fair0.99
3:14 #8 skipped

shot 8·duplicate of #7
3:41 #9 skipped

shot 9·duplicate of #7
4:06 #10 skipped

shot 10·duplicate of #7
4:36 #11 done20 line(s)

shot 11·sharpness 2591.4
1. AlEngineer0.98
2. The convergence: neurosymbolic AI1.00
3. World'sFair1.00
4. Neither lineage is enough alone — the fracture from Al's founding, finally healed.0.99
5. AGENT1.00
6. ONTOLOGY1.00
7. Probabilistic engine. Powerful but uncertain —1.00
8. Formalized guardrails. Trustworthy but rigid —1.00
9. reasons over a domain it only partly understands.1.00
10. symbols and logic over a bounded domain.1.00
11. It proposes.1.00
12. It permits.1.00
13. NEUROSYMBOLIC — the probabilistic agent operating inside formalized1.00
14. guardrails1.00
15. The agent proposes. The ontology permits. That is the synthesis.1.00
16. Probabilistic reasoning inside.1.00
17. Logical constraints outside.0.98
18. TRACK 5· JULY 2, 20260.95
19. Graphs1.00
20. World'sFair1.00
5:19 #12 done22 line(s)

shot 12·sharpness 2723.1
1. AlEngineer0.98
2. The convergence: neurosymbolic AI1.00
3. World'sFair1.00
4. Neither lineage is enough alone — the fracture from Al's founding, finally healed.0.99
5. AGENT1.00
6. ONTOLOGY1.00
7. PRESENTED BY1.00
8. Probabilistic engine. Powerful but uncertain —0.99
9. Formalized guardrails. Trustworthy but rigid —0.99
10. reasons over a domain it only partly understands.1.00
11. symbols and logic over a bounded domain.1.00
12. Microsoft1.00
13. It proposes.1.00
14. It permits.0.99
15. NEUROSYMBOLIC — the probabilistic agent operating inside formalized1.00
16. guardrails1.00
17. The agent proposes. The ontology permits. That is the synthesis.0.99
18. Probabilistic reasoning inside.0.98
19. Logical constraints outside.1.00
20. TRACK 5• JULY 2, 20260.95
21. Graphs1.00
22. World's Fair0.98
5:34 #13 done23 line(s)

shot 13·sharpness 1817.0
1. AlEngineer0.98
2. Ontology → Data as Graphs0.98
3. World's Fair0.99
4. Entities become nodes, relationships become edges, properties live inside.1.00
5. PRESENTED BY0.98
6. Microsoft1.00
7. PLACED1.00
8. Customer1.00
9. Order1.00
10. name email1.00
11. REFUNDED1.00
12. CONTAINS1.00
13. 自0.72
14. Refund1.00
15. Product1.00
16. amount, reason1.00
17. sku, price1.00
18. nodes = entities (typed)0.98
19. edges=relationships1.00
20. text inside = properties0.99
21. TRACK 5· JULY 2, 20260.95
22. Graphs1.00
23. World's Fair1.00
6:10 #14 skipped

shot 14·duplicate of #13
6:33 #15 done19 line(s)

shot 15·sharpness 1827.1
1. How to build an ontology for your organization1.00
2. AlEngineer0.98
3. World'sFair1.00
4. Two approaches — not in conflict. Meet in the middle, and reuse what already exists.0.99
5. TOP DOWN· experts define entities, properties, relations &0.99
6. "Expert Systems 2.0"1.00
7. constraints1.00
8. PLACED1.00
9. Customer1.00
10. Order1.00
11. REFUNDED1.00
12. CONTAINS1.00
13. 自0.94
14. Refund1.00
15. Product1.00
16. BOTTOM UP · extract from docs & reviews — NER / LLM mining0.98
17. TRACK 5· JULY 2, 20260.94
18. Graphs1.00
19. World'sFair1.00
7:09 #16 skipped

shot 16·duplicate of #15
7:23 #17 skipped

shot 17·duplicate of #15
8:05 #18 skipped

shot 18·duplicate of #15
8:28 #19 skipped

shot 19·duplicate of #15
8:54 #20 skipped

shot 20·duplicate of #15
9:32 #21 done30 line(s)

shot 21·sharpness 2404.7
1. AlEngineer0.99
2. RDFS infers. OWL constrains.0.99
3. World's Fair0.96
4. The graph isn't a drawing — it derives facts and enforces rules.0.99
5. RDFS1.00
6. OWL1.00
7. Domain & range → inference0.97
8. Transitive + functional0.99
9. teaches rdfs:domain Teacher1.00
10. TRANSITIVE — chains close0.98
11. teaches rdfs:range Student0.98
12. ancestor a TransitiveProperty1.00
13. Sue ancestor Mary0.98
14. Bob teaches Scooter1.00
15. Mary ancestor Anne1.00
16. → Sue ancestor Anne0.98
17. ↓ the reasoner derives, unasked:0.99
18. FUNCTIONAL — at most one0.98
19. Bob1.00
20. a Teacher1.00
21. hasFather a FunctionalProperty1.00
22. Scooter a Student1.00
23. Stewie hasFather Peter1.00
24. Stewie hasFather Peter_G1.00
25. = Peter owl:sameAs Peter_G0.97
26. These derivations and constraints don't just sit in the graph — they become the guardrails an agentic loop must0.99
27. obey. Next: the loop, and where it goes wrong.0.99
28. TRACK 5· JULY 2,20260.97
29. Graphs1.00
30. World's Fair0.98
9:49 #22 skipped

shot 22·duplicate of #21
10:25 #23 skipped

shot 23·duplicate of #21

Transcript

241 cues· 2,897 words· 15,610 chars

chunk 0 · 0:13–0:54 · 101 words · 521 chars
0:13 Okay, we're going to launch here.
0:15 So my name is Frank Coyle.
0:18 I'm an educator.
0:20 I'm teaching at Berkeley now.
0:21 I've been doing this computer science stuff for, oh, 30, 35 years.
0:29 Right now, it's kind of a critical time for poor computer science students.
0:34 It used to be the only game in town, degree was a guaranteed job, and now, thanks to AI, it's not.
0:40 But then again, 5,000 people are here, so AI and agents seem to be the way to go.
chunk 1 · 0:48–1:32 · 98 words · 534 chars
0:48 So the question is, how do we leverage this new universe
0:52 that we are moving quickly into.
0:54 And so I want to talk about how agents and ontologies, big word, fit together.
1:00 But before I do that, I wanted to give you my educational philosophy.
1:09 And this comes from someone called Sister Corita Kent, and it was made popular by John Cage, who was an avant-garde musician.
chunk 2 · 1:21–2:06 · 112 words · 621 chars
1:21 You've got to think about this a little bit.
1:23 Nothing is a mistake.
1:25 There is no win.
1:26 There's no fail.
1:28 There's only make.
1:30 And more and more today, that's what's important.
1:33 Get down and make stuff, and that's how you're going to learn, not by necessarily reading.
1:38 I'm also a big fan of writing.
1:41 My early career was in neuroscience.
1:43 I'm kind of coming back into it now that agentic AI is bringing kind of cognitive science back.
1:51 But engage your senses.
chunk 3 · 1:53–2:35 · 97 words · 547 chars
1:53 Get a notebook.
1:54 Get a pen, a pencil.
1:57 Draw pictures.
1:58 Write stuff down.
2:00 Just don't type, because when you're typing, your brain is thinking about the letters on the keyboard.
2:06 When you're writing in a book, your whole brain, all your sensory systems are engaged, and you're gonna learn faster that way.
2:16 Okay, on to our talk.
2:21 Agents and ontology.
2:22 So there are two lineages here, and I wanna talk about both, give you a little philosophical background.
chunk 4 · 2:29–3:14 · 111 words · 639 chars
2:29 Agents, when did we start talking about agents?
2:32 Well, it goes back to the early initial days of AI.
2:36 People like John McCarthy, Selfridge, Marvin Minsky, Society of Mind, people started thinking about the fact that this new computing technology
2:48 was going to lead us into some kind of artificial intelligence, which is a term that came in 1956 when all these characters got together and tried to figure out where the future was going.
chunk 5 · 3:01–3:41 · 107 words · 596 chars
3:01 And the concept of an agent finally evolved, things that
3:05 perceive and decide and then act, and that's what we're seeing now.
3:09 Now, what about ontologies?
3:10 Well, it turns out ontologies are not that new, okay?
3:14 It was actually Aristotle who first came up with the concept of we need a philosophy of being, like, ooh, kind of heavy,
3:24 but came up with categories of being.
3:26 And this kind of relates to what people are doing now with graph databases and knowledge representation.
chunk 6 · 3:33–4:15 · 87 words · 517 chars
3:33 And there are a couple of other people who kind of formalized it.
3:37 Von Quine was a philosopher and then this guy Gruber, 1993.
3:42 And I think this captures what knowledge and
3:48 graph technology really represents.
3:51 It is a formal specification of a shared conceptualization.
3:57 And that's what we want to give to our agents.

Chapters

0:00 Intro and an educator's philosophy
2:21 Two lineages: agents and ontologies
4:04 Neurosymbolic AI: guardrails around a probabilistic model
5:23 What an ontology actually is
6:14 Building one, and the expert systems era
7:55 Reusing existing taxonomies
9:12 RDFS and OWL: inference and constraints
12:12 Agents, loops, and how they break
14:22 A Claude tool use loop with an ontology validator
17:47 Pydantic at the door, ontology at the ledger
18:52 The errors an ontology catches that English cannot