vidtheque.

read-only demo

Videos Bck7ABCZRZI

Stop Renting Your Cognitive Infrastructure - Thiyagarajan Maruthavanan, Kalmantic Labs

index_state ready data_status ok

AI Engineer· published 2026-07-18· 0:07:51· en-US· indexed 2026-08-11 00:44

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:27, 1 of 1 keyframes kept
  2. Shot 1, 0:27 to 1:00, 1 of 1 keyframes kept
  3. Shot 2, 1:00 to 1:31, 1 of 1 keyframes kept
  4. Shot 3, 1:31 to 2:02, 1 of 1 keyframes kept
  5. Shot 4, 2:02 to 2:34, 1 of 1 keyframes kept
  6. Shot 5, 2:34 to 3:11, 1 of 1 keyframes kept
  7. Shot 6, 3:11 to 3:46, 1 of 1 keyframes kept
  8. Shot 7, 3:46 to 3:57, 1 of 1 keyframes kept
  9. Shot 8, 3:57 to 4:22, 1 of 1 keyframes kept
  10. Shot 9, 4:22 to 4:47, 0 of 1 keyframes kept
  11. Shot 10, 4:47 to 5:20, 1 of 1 keyframes kept
  12. Shot 11, 5:20 to 5:42, 1 of 1 keyframes kept
  13. Shot 12, 5:42 to 6:14, 1 of 1 keyframes kept
  14. Shot 13, 6:14 to 6:40, 1 of 1 keyframes kept
  15. Shot 14, 6:40 to 7:06, 1 of 1 keyframes kept
  16. Shot 15, 7:06 to 7:32, 0 of 1 keyframes kept
  17. Shot 16, 7:32 to 7:51, 1 of 1 keyframes kept
  18. Shot 17, 7:51 to 7:51, 1 of 1 keyframes kept

18 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
87
whisperx 87
chunks
15
from 87 cues
keyframes
16
kept of 18 captured
frames with text
15
104 lines read
chapters
0
from the source metadata
keyframe bytes
1.0 MB
word timings on 87 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-11 00:43 1m 06s
stt done — 2026-08-11 00:44 9s
chunk done — 2026-08-11 00:44 0s
text_embed done — 2026-08-11 00:44 0s
keyframe done — 2026-08-11 00:44 19s
ocr done — 2026-08-11 00:44 5s
frame_embed done — 2026-08-11 00:44 2s

Frames, and what the machine read

  • 0:18 #0 done4 line(s)

    shot 0·sharpness 658.2

    1. THE ROOM YOU WEREN'T SHOWN0.99
    2. $200M1.00
    3. One enterprise. On the cost of inference. Then they built their own.1.00
    4. IMAGE - endless warehouse / hyperscale hall / cargo port.0.99

  • 0:53 #1 done6 line(s)

    shot 1·sharpness 1163.0

    1. THE DRAIN0.99
    2. The moment you rent a0.98
    3. token, it's expensive.0.99
    4. $25 per million looks like nothing. You load credits, load credits, load credits,0.99
    5. and never feel the spend until the budget is gone.0.99
    6. IMAGE - unopened mail / phone face-down / blinds shut.0.99

  • 1:15 #2 done6 line(s)

    shot 2·sharpness 1080.9

    1. THREE WEEKS AGO0.99
    2. I built UltaSuno on0.98
    3. rented intelligence.1.00
    4. Hundreds of users. Thousands of dollars in credits. Then my key got stolen, and0.98
    5. I watched a stranger drain the rest.0.99
    6. IMAGE - money in a torn bag / burn rate / counter ticking up.0.99

  • 1:35 #3 done6 line(s)

    shot 3·sharpness 1074.6

    1. THREE WEEKS AGO0.99
    2. I built UltaSuno on0.98
    3. rented intelligence.1.00
    4. Hundreds of users. Thousands of dollars in credits. Then my key got stolen, and0.98
    5. I watched a stranger drain the rest.1.00
    6. IMAGE - money in a torn bag / burn rate / counter ticking up.0.98

  • 2:18 #4 done6 line(s)

    shot 4·sharpness 1079.2

    1. THREE WEEKS AGO0.97
    2. I built UltaSuno on0.98
    3. rented intelligence.1.00
    4. Hundreds of users. Thousands of dollars in credits. Then my key got stolen, and0.99
    5. I watched a stranger drain the rest.0.99
    6. IMAGE - money in a torn bag / burn rate / counter ticking up.0.98

  • 2:38 #5 done6 line(s)

    shot 5·sharpness 1264.7

    1. THE SWITCH1.00
    2. Token factories just1.00
    3. switched the landlord.1.00
    4. Renting became leasing. Instead of paying Anthropic you pay Fireworks. Lost0.99
    5. keys, dumped tokens, retries, none of it goes away.1.00
    6. IMAGE - showroom / row of suits / slick sales floor.0.98

  • 3:36 #6 done7 line(s)

    shot 6·sharpness 1466.7

    1. ITRIED IT MYSELF0.99
    2. Local Inference0.97
    3. Memory was the1.00
    4. bottleneck1.00
    5. Raised the price citing a memory shortage. On a memory i0.98
    6. factory.1.00
    7. IMAGE - YOUR DGX Spark photo / brick-wall dead end.0.98

  • 3:54 #7 done6 line(s)

    shot 7·sharpness 1287.3

    1. THE POSITION0.96
    2. For an enterprise, renting1.00
    3. and leasing don't cut it.1.00
    4. The bill is only the first problem. The moment you rent, a second set of problems1.00
    5. shows up, and for an enterprise they're the ones that matter.0.99
    6. IMAGE - server room behind glass / access denied.0.98

  • 4:07 #8 done9 line(s)

    shot 8·sharpness 1449.2

    1. THREE REGULATED SHOPS1.00
    2. A fund. A hospital.0.99
    3. A tax practice.0.99
    4. Afund1.00
    5. Control. The reasoning can't sit behind someone else's rate limit.0.99
    6. Ahospital1.00
    7. Audit. You have to know exactly who you depend on, and prove it.0.99
    8. A tax practice0.94
    9. Reproducibility. If intelligence gave the answer, you must retrace it.0.99

  • 4:39 #9 skipped

    shot 9·duplicate of #8

  • 4:58 #10 done9 line(s)

    shot 10·sharpness 818.7

    1. FIND YOUR POSITION1.00
    2. Where do you sit?1.00
    3. Pre-PMF1.00
    4. Keep renting. You don't know what you'll need yet.0.99
    5. Post-PMF1.00
    6. Start building. You have to support what you've found.1.00
    7. Enterprise1.00
    8. No choice. You build your own inference infra.0.98
    9. IMAGE - three doors / fork in the road / sorting line.0.98

  • 5:36 #11 done9 line(s)

    shot 11·sharpness 1143.2

    1. THE TRAP, NAMED0.99
    2. Treat it like Airbnb.0.99
    3. You have to buy your own house.1.00
    4. Rent1.00
    5. Cheap monthly. Never yours.0.99
    6. Lease1.00
    7. Locked in for years. Still not yours.1.00
    8. Own0.95
    9. Costs upfront. The only one that's yours.1.00

  • 5:49 #12 done6 line(s)

    shot 12·sharpness 963.2

    1. NOT A HYPOTHESIS0.97
    2. So I built my own.0.97
    3. I call it JustInfer.0.99
    4. Rented, hit two problems. Leased, same two problems. DGX boxes wouldn't hold0.99
    5. reliability, so I went to bare metal and built the whole stack.0.99
    6. IMAGE - bare-metal GPU rack / hands-on the hardware.0.98

  • 6:27 #13 done10 line(s)

    shot 13·sharpness 1808.6

    1. WHAT I MADE ALONG THE WAY0.98
    2. Open sourced a tool1.00
    3. github.com/kalmantic.com/JusTokenMax1.00
    4. Wrote the book0.99
    5. PeakInference Infra Economics of AI Inference1.00
    6. JusTokenMax (MIT)1.00
    7. Open source. If you still rent or lease, it compresses input, cuts waste, adds1.00
    8. security.1.00
    9. The book0.95
    10. If you're an enterprise that has to build its own, the whole how-to is in here.1.00

  • 6:58 #14 done10 line(s)

    shot 14·sharpness 1006.5

    1. BEFORE MONDAY0.97
    2. Everyone sells one story.1.00
    3. Find your own answer.1.00
    4. Jensen, Nvidia0.98
    5. Token factory. He sells the factory.1.00
    6. Satya, Microsoft0.98
    7. Unmetered intelligence. Folded into a seat you already pay for.1.00
    8. Lin Qiao, Fireworks0.99
    9. Own your intelligence. On her infrastructure.1.00
    10. For enterprises, the answer is to own it. Find me at Al Engineer / X / LinkedIn.0.97

  • 7:24 #15 skipped

    shot 15·duplicate of #14

  • 7:38 #16 done4 line(s)

    shot 16·sharpness 286.6

    1. Thank you.0.96
    2. JustInfer · JusTokenMax(MIT) · PeakInference0.96
    3. mtrajan.com x.com/mtrajan1.00
    4. IMAGE - an open door, light beyond - the exit. or reuse slide 1.0.99

  • 7:51 #17 empty

    shot 17·sharpness 44.2

Transcript

87 cues· 1,546 words· 8,402 chars

  1. 0:00 One of the largest retailers in the country spent close to $200 million on inference with Anthropic and decided that things got way out of hand and built their own infrastructure.
  2. 0:09 I'm pretty sure most of you have read the news from Uber CTO on how they had planned a budget of their tokens for an entire year and it got over in month four.
  3. 0:19 I'm also confident that half of you in this room have come to a very similar conclusion that as time goes by, the cost of intelligence really builds.
  4. 0:27 Using inference feels like, you know, it's one of the most inexpensive thing.
  5. 0:30 But then this is very different from using a phone where you get a bill once every month and then you have like
  6. 0:36 a specific set of amount that you can actually anchor your mind to but in case of using these rented intelligence platform they are like prepaid you load credits it's almost as if you're loading credits inside a casino you put some and then you pull it and then you are so addicted to it then you end up doing more and more of it and by some time you realize that you've blown past the threshold that you had mentally kept in mind and i had this
  7. 1:01 experience myself i built an app called ultra sono and i experienced the influence cost ballooning here suno.com has anybody heard about suno.com yeah suno.com is this application that allows a user to turn a text prompt into music what i was interested in is doing the reverse which is given a particular song what prompt could have actually generated it this is something that i wanted so i built this
  8. 1:26 And I was having a lot of fun using this application, shared it with a few friends and spread wide around.
  9. 1:31 I had hundreds of thousands of users, but then the cost ballooned way more than what I had anticipated.
  10. 1:38 Hundreds of thousands of dollars had to spend on inference.
  11. 1:41 Now, this happens for many reasons.
  12. 1:43 There are many talks that are there at AI itself where people talk about how you need to manage your context better.
  13. 1:50 And many people forget about doing compression of their input token.
  14. 1:54 And when there are agent loops, then there are many of these calls that are happening, which are very, very wasteful.
  15. 2:00 The inference endpoint that is consuming this is completely unaware of the shape of the workload and which is why this happens.
  16. 2:06 And I have this other issue that had happened.
  17. 2:08 Three weeks ago, my key got stolen.
  18. 2:11 Someone in China got hold of it and then was sucking my endpoint dry.
  19. 2:15 I could see the cost rise up from $7,000 to $7,500 to $8,000 and so on and so forth.
  20. 2:22 Thanks to my co-founder who heads the research and technology, we were able to arrest it at $10,000.
  21. 2:28 Otherwise, it could have been $100,000.
  22. 2:30 Now, many people suggest that the alternative to rented intelligence platform is to use Token Factory.
  23. 2:35 Token Factory is basically saying that why are you paying money to Anthropic and OpenAI?
  24. 2:39 Instead, go open source, have these open source models that are already deployed somewhere on the cloud, and then they are provisioned as tokens per second.
  25. 2:48 There are NeoClouds and then there are inference endpoint providers who actually do this.
  26. 2:52 In fact, there is also an argument saying that you can build this token factory locally.
  27. 2:56 There are AI Twitter influencers who actually talk about building inference in your garage, in your basement.
  28. 3:03 Buy GPU cards, rig them up together, and then you could actually run a local token factory.
  29. 3:08 In fact, I was inspired by that a little bit.
  30. 3:11 I bought my own DGX Sparks, and then I first moved Ultasuno from Anthropic to DGX Sparks.
  31. 3:19 It worked well.
  32. 3:19 I ran into this one issue of memory being the bottleneck, and then it was good enough that I started building my next applications.
  33. 3:26 I started having agents.
  34. 3:28 I have some agents that I need for running my research lab.
  35. 3:30 So these agents started shaping up inside the DGX Sparks, 2, 6, 8, 12, and it worked all right.
  36. 3:37 The issue though is that, you know, it may not be reliable for enterprise, which is what I exactly faced.
  37. 3:43 Three enterprises reached out to me to replicate the same setup for them.
  38. 3:47 But for enterprises, renting and leasing don't cut it.
  39. 3:50 Bill is a problem, but then there are secondary set of problems that makes it extremely ineffective approach.
  40. 3:57 The enterprises that I reached out to me, one was a fund, another a hospital, and the third a tax practice.
  41. 4:03 And each of them had different wall that they had hit.
  42. 4:06 The fund, it was an investment fund.
  43. 4:08 They were running an investment analyst on a Nemo architecture, and they didn't want somebody else to dictate as to what the rate limit that they could consume.
  44. 4:16 So control become a big issue for them to actually go with token factories.
  45. 4:21 Hospital had a different issue.
  46. 4:22 They used the use case, it worked well, but later when they went through an audit,
  47. 4:28 A third party vendor dependency was redlined and then they couldn't go forward.
  48. 4:32 The tax practice was a completely different issue.
  49. 4:35 In a tax practice, what is happening is when intelligent generates a recommendation, you want to be able to recreate it.
  50. 4:42 And when you don't have access to the in-depth of the model, you will not be able to do this.

Open at this second