Videos Owb8g3yDyzo

Why Off-the-Shelf AI Doesn't Understand Money — Udi Menkes, Intuit

index_state ready data_status ok

AI Engineer· published 2026-07-29· 0:19:50· en-US· indexed 2026-08-10 19:42

Open on YouTube

47 shot(s).

keyframes kept every frame deduplicated

What was stored

cues: 202; whisperx 202
chunks: 35; from 202 cues
keyframes: 25; kept of 47 captured
frames with text: 25; 416 lines read
chapters: 11; from the source metadata
keyframe bytes: 5.0 MB; word timings on 202 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage	state	model	started	took
`fetch`	done	—	2026-08-09 23:19	0s
`stt`	done	—	2026-08-09 13:59	20s
`chunk`	done	—	2026-08-09 13:59	0s
`text_embed`	done	—	2026-08-10 19:42	0s
`keyframe`	done	—	2026-08-09 13:59	3m 39s
`ocr`	done	—	2026-08-09 14:03	16s
`frame_embed`	done	—	2026-08-10 19:42	4s

Frames, and what the machine read

0:02 #0 done2 line(s)

shot 0·sharpness 459.9
1. AlEngineer0.96
2. World's Fair1.00
0:03 #1 done2 line(s)

shot 1·sharpness 662.1
1. AlEngineer0.95
2. World's Fair0.99
0:10 #2 done24 line(s)

shot 2·sharpness 2744.8
1. LAB & PLATINUM SPONSORS0.99
2. Amazon AGI Lab0.98
3. ANTHROP\C1.00
4. Google DeepMind1.00
5. MINIMAX0.94
6. OpenAI0.92
7. Akamai1.00
8. arize1.00
9. aws1.00
10. Braintrust bright data0.98
11. B1.00
12. Browserbase1.00
13. docker1.00
14. :neo4j0.92
15. ORACLE1.00
16. PayPal1.00
17. qodo1.00
18. reducto1.00
19. Sonar1.00
20. Makers of1.00
21. togetherai1.00
22. Unblocked1.00
23. WorkOS1.00
24. SonarQube1.00
0:14 #3 done2 line(s)

shot 3·sharpness 89.3
1. AlEngineer0.98
2. World's Fair1.00
0:36 #4 done14 line(s)

shot 4·sharpness 1545.6
1. AlEngineer0.96
2. World'sFair1.00
3. AI ENGINEER WORLD'S FAIR· 20260.97
4. PRESENTED BY1.00
5. Why Off-the-Shelf Al0.94
6. Microsoft1.00
7. Doesn't Understand Money1.00
8. Even the best LLMs can get your business confidently, dangerously0.99
9. wrong - and what actually fixes it.0.97
10. Udi Menkes1.00
11. Principal Al Product Manager, Intuit0.98
12. in/udimenkes- X @menkesu0.96
13. Engineering the future of Al0.99
14. World'sFair1.00
0:52 #5 skipped

shot 5·duplicate of #4
1:17 #6 done15 line(s)

shot 6·sharpness 1449.6
1. AlEngineer0.96
2. World'sFair1.00
3. AI ENGINEER WORLD'S FAIR· 20260.97
4. PRESENTED BY1.00
5. Why Off-the-Shelf Al0.96
6. Microsoft1.00
7. Doesn't Understand Money1.00
8. Even the best LLMs can get your business confidently, dangerously1.00
9. wrong - and what actually fixes it.0.96
10. Udi Menkes1.00
11. Principal Al Product Manager, Intuit1.00
12. in/udimenkes· X @menkesu0.97
13. TRACK 3· JULY 2, 20260.94
14. Alin Finance0.98
15. World'sFair1.00
1:50 #7 done13 line(s)

shot 7·sharpness 1314.4
1. AlEngineer0.97
2. World'sFair1.00
3. AI ENGINEER WORLD'S FAIR· 20260.98
4. Why Off-the-Shelf Al0.97
5. Doesn't Understand Money1.00
6. Even the best LLMs can get your business confidently, dangerously1.00
7. wrong - and what actually fixes it.0.97
8. Udi Menkes0.99
9. Principal Al Product Manager, Intuit0.98
10. in/udimenkes· X @menkesu0.98
11. TRACK 3· JULY 2, 20260.95
12. Alin Finance0.98
13. World'sFair1.00
2:15 #8 skipped

shot 8·duplicate of #6
2:56 #9 done20 line(s)

shot 9·sharpness 1413.1
1. AlEngineer0.97
2. World'sFair1.00
3. ONE BUSINESS·ONE QUESTION· TWO VERY DIFFERENT ANSWERS0.98
4. 41.00
5. A small landlord. One rental property - already cash-flow negative every month, carrying loan payments,0.99
6. profit trending into the red. The question: "How do l improve my profit?"0.99
7. A FRONTIER MODEL0.99
8. GROUNDED IN REAL OUTCOMES1.00
9. "Acquire a second rental property to add an0.99
10. "Raise the rent on the existing tenant 5-10%0.99
11. income stream and offset the decline."1.00
12. before the next renewal."1.00
13. Flagged unsafe - take on more debt for a business already0.99
14. Respects the cash constraint: more profit, no new debt, right-0.98
15. losing money.1.00
16. sized to the business.1.00
17. 021.00
18. TRACK 3· JULY 2, 20260.94
19. Alin Finance0.97
20. World'sFair1.00
3:16 #10 skipped

shot 10·duplicate of #9
3:52 #11 skipped

shot 11·duplicate of #9
4:12 #12 done20 line(s)

shot 12·sharpness 1310.6
1. AlEngineer0.98
2. World'sFair1.00
3. NOT A ONE-OFF - THE SAME PATTERN, A DIFFERENT BUSINESS0.99
4. A small supplier. One customer is ~70% of its revenue. One vendor is nearly all of its costs. Same question:0.98
5. 41.00
6. "How do l improve my profit?"0.99
7. A FRONTIER MODEL0.98
8. GROUNDED IN REAL OUTCOMES0.98
9. "Raise prices 15-20% on that dominant1.00
10. "Negotiate volume-based pricing with the0.99
11. customer."1.00
12. main vendor for a 5-10% cost cut."0.99
13. Flagged unsafe - a hike on the customer who is ~70% of0.99
14. The cost-side lever – no risk to the revenue base. Same goal,0.99
15. revenue could end the business.1.00
16. opposite direction.1.00
17. Two businesses. Two confident answers. Both would have done real damage.0.99
18. TRACK 3· JULY 2, 20260.95
19. Alin Finance0.99
20. World'sFair1.00
4:41 #13 skipped

shot 13·duplicate of #9
5:10 #14 skipped

shot 14·duplicate of #12
5:53 #15 done15 line(s)

shot 15·sharpness 1152.2
1. AlEngineer0.96
2. World'sFair1.00
3. IT SOUNDS RIGHT BECAUSE IT READ ABOUT MONEY- NOT YOUR BUSINESS0.99
4. 40.93
5. The Fluent Bluff0.97
6. PRESENTED BY1.00
7. Microsoft1.00
8. Confident. Fluent. Generic.The model learned money from words on the0.99
9. internet - not from what happened to real businesses.0.97
10. Almost every out-of-the-box LLM answer about your money is one0.99
11. In our research, an off-the-shelf model told ~40% of businesses to "acquire new customers" and ~14% to "increase revenue by product" - the0.99
12. same two generic moves, whatever the business.1.00
13. TRACK 3· JULY 2,20260.94
14. Alin Finance0.96
15. World'sFair1.00
6:09 #16 done15 line(s)

shot 16·sharpness 1150.4
1. AlEngineer0.95
2. World'sFair1.00
3. IT SOUNDS RIGHT BECAUSE IT READ ABOUT MONEY- NOT YOUR BUSINESS0.99
4. 40.98
5. The FluentBluff0.99
6. PRESENTED BY1.00
7. Microsoft1.00
8. Confident. Fluent. Generic. The model learned money from words on the0.99
9. internet - not from what happened to real businesses.0.99
10. Almost every out-of-the-box LLM answer about your money is one0.99
11. In our research, an off-the-shelf model told ~40% of businesses to "acquire new customers" and ~14% to "increase revenue by product" - the0.99
12. same two generic moves, whatever the business.1.00
13. TRACK 3·JULY 2,20260.96
14. AlinFinance0.99
15. World'sFair0.97
6:57 #17 done22 line(s)

shot 17·sharpness 1540.2
1. AlEngineer0.99
2. World's Fair0.98
3. PRINCETON·CEO-BENCH·20261.00
4. Long-horizon1.00
5. $100.00M0.99
6. 40.97
7. business decisions:0.98
8. $10.00M0.98
9. Al's open frontier.1.00
10. Top models ran a startup for500 days,1.00
11. $1M to start. Most couldn't stay above0.99
12. where they began.1.00
13. And a simple rule-based bot – no LLM –0.98
14. beat most of them.0.99
15. 1001.00
16. 2000.86
17. Day1.00
18. 3001.00
19. 5001.00
20. TRACK 3· JULY 2, 20260.95
21. Al in Finance1.00
22. World's Fair0.97
7:06 #18 skipped

shot 18·duplicate of #17
7:38 #19 skipped

shot 19·duplicate of #17
8:13 #20 done13 line(s)

shot 20·sharpness 1018.3
1. AlEngineer0.98
2. World'sFair1.00
3. 41.00
4. A FRONTIER MODEL0.99
5. OURS0.99
6. has read0.96
7. has watched1.00
8. about money.0.99
9. what happens.1.00
10. That's the difference between sounding right and being right.0.99
11. TRACK 3· JULY 2, 20260.94
12. Alin Finance0.99
13. World'sFair0.99
8:31 #21 skipped

shot 21·duplicate of #20
8:47 #22 skipped

shot 22·duplicate of #20
9:17 #23 skipped

shot 23·duplicate of #20

Transcript

202 cues· 2,797 words· 15,204 chars

chunk 0 · 0:12–0:50 · 114 words · 592 chars
0:12 So I have a three-year-old daughter, and she's absolutely adorable.
0:18 And the parents here in the room know how insightful that age can be.
0:23 And she has a complete theory about money by now.
0:27 And I'll give an example.
0:28 So a couple weeks ago, I was driving the car.
0:31 I was coming into park, and there was something in my dead end.
0:35 And I scratched the car.
0:36 I went out.
0:37 I'm like, oh, man, I can't believe I scratched the car.
0:40 And then I hear my daughter from the back.
0:42 And she was like, Daddy, what's happened?
chunk 1 · 0:44–1:28 · 90 words · 483 chars
0:44 And I'm explaining it to her.
0:46 And then she says, what's the problem?
0:49 Just buy another one.
0:52 So anyway, I want to ask you today, with a raise of hand, who uses LLMs, has used LLMs for getting financial advice, a recommendation on something?
1:07 in the financial world?
1:08 Great.
1:09 Almost everyone.
1:10 Wait.
1:11 Keep your hand up if you trusted the answer and you actually followed the advice.
chunk 2 · 1:19–2:03 · 115 words · 585 chars
1:19 OK.
1:20 a lot of hands are going down.
1:23 And that's the core problem.
1:25 So I had the same thing a couple months ago.
1:28 I had a big decision I was looking to take.
1:31 Should I invest in real estate niche or in the stock market niche on a specific area?
1:38 And I do AI for finance for a living.
1:41 So I went the full-blown way, context, brain, the books, the knowledge, all my finances combined, all the latest models.
chunk 3 · 1:51–2:27 · 98 words · 505 chars
1:51 And it gave me a recommendation.
1:52 You should do A with great reasoning.
1:56 And then I changed just a little bit one of the assumptions, and it completely flipped.
2:00 You should do B, never do A.
2:04 And then I tweaked one more small thing, and it went all the way back to A.
2:10 And at that moment, I understood that the advice sounds good.
2:14 It sounds sound, but I can't really trust it.
2:18 And by the end of the talk today, you will understand why off-the-shelf LLMs don't understand money and what you need to do about it.
chunk 4 · 2:32–3:17 · 108 words · 616 chars
2:32 So I'm going to show you a couple of real examples from a study we're doing at Intuit
2:37 on thousands and thousands of businesses around 100,000 situations and timeframes.
2:44 This is an example of a small business, a new landlord that is building a rental property business.
2:50 His first property, and he's down, he's in negative cash flow, there's an open loan, and the profit is basically trending into the red.
3:00 And a question comes up, how do I improve my profit?
chunk 5 · 3:05–3:48 · 107 words · 576 chars
3:05 And a frontier model gives the following response.
3:09 go and acquire a second rental property because that will bring more income and compensate for the deficit.
3:17 And that model had all of the business's data.
3:21 Now, that's very risky for someone in the negative in the red to be doing.
3:27 On the other hand, a model that is grounded in real outcomes, and what I mean by that is a model that has seen similar situations of such businesses, what they did and what was the outcome, actually recommended to raise prices on the existing tenant by 5% to 10% and to do it before the renewal.
chunk 6 · 3:49–4:30 · 111 words · 568 chars
3:49 Now, some of you are thinking that it's just a matter of context.
3:54 Just give it more context.
3:56 And the thing is that this advice is coming based off on real situations of similar businesses and would actually move them into profitability in this case.
4:10 And this is not a one-off.

Chapters

0:00 A three year old's theory of money
2:21 Why off the shelf models fail at money
2:48 The rental property example
4:34 The fluent bluff
6:42 The Princeton million dollar study
9:45 Context is not experience
11:03 Correlation versus causation in pricing
13:37 Building state, action, outcome data
15:17 Testing it head to head
16:37 The era of outcome driven finance
17:55 Three things to remember