Videos OXMMN-XbxwA
Research to Reality: Bringing Frontier ML Research to Production - Vaidas Razgaitis, Higharc
Scene timeline
40 shot(s).
keyframes kept every frame deduplicated
What was stored
- cues
- 104
- whisperx 104
- chunks
- 27
- from 104 cues
- keyframes
- 35
- kept of 40 captured
- frames with text
- 35
- 682 lines read
- chapters
- 0
- from the source metadata
- keyframe bytes
- 2.7 MB
- word timings on 104 cues
Provenance
| stage | state | model | started | took |
|---|---|---|---|---|
fetch |
done | — | 2026-08-11 07:54 | 1m 14s |
stt |
done | — | 2026-08-11 07:56 | 15s |
chunk |
done | — | 2026-08-11 07:56 | 0s |
text_embed |
done | — | 2026-08-11 07:56 | 0s |
keyframe |
done | — | 2026-08-11 07:56 | 40s |
ocr |
done | — | 2026-08-11 07:57 | 17s |
frame_embed |
done | — | 2026-08-11 07:57 | 6s |
Frames, and what the machine read
-
- 回0.85
- AI ENGINEER WORLD'S FAIR 20261.00
- ONLINE TRACK1.00
- Research to Reality0.98
- Turning frontier ML research into real, shipped features.0.99
- Vaidas Razgaitis1.00
- Senic0.98
- ineer, Labs·Higharc0.99
-
- 回0.93
- 0:021.00
- 国0.99
- 中0.87
-
- 回0.88
- Higharc1.00
- This is Al for homebuilding.0.99
- Slide 2/180.98
-
- 日0.53
- Higharc Al0.99
- Higharc, please generate1.00
- this home with pricing,1.00
- renders, and build ready plans.1.00
- slide 2/180.91
- D ×0.58
- End presentation0.92
-
- Higharc Al1.00
- Building data model..0.96
-
- Higharc Al0.98
- Show it to me in a0.98
- community.1.00
- Rendered home in a1.00
- community.1.00
-
- Higharc Al0.99
- How about spring?0.99
- 合合0.97
- Rendered home in spring.0.99
- Side 2/18 -0.85
-
- Frontier Al / ML Research:0.96
- Researchers ≠ production engineers0.99
- Computer Vision1.00
- ML Engineer1.00
- up-to-date on papers / research0.99
- Computer Vision0.98
- Model training & fine-tuning0.99
- Generative Building1.00
- Evals & Observability0.97
- Production-grade code0.98
- Model (GBM)0.96
- Vector DBs & RAG1.00
- LLMs (tokenization, prompting, context management)0.99
- Agent building (ReAct, tool-use, workflows, multi-agent)0.99
- spatial intelligence1.00
- Model routing (fal.ai, OpenRouter)0.99
- Building robust non-deterministic features (testing &0.98
- fault-tolerance)1.00
- AI Engineer1.00
- Backend system design1.00
- Streaming architectures (SSE)1.00
- Reactive UIs1.00
- In-memory state management1.00
- ents1.00
- Full-stack SWE1.00
- End presentatio0.87
-
- Frontier Al / ML Research:0.94
- Researchers ≠ production engineers0.99
- Computer Vision1.00
- ML Engineer1.00
- up-to-date on papers / research0.98
- Computer Vision1.00
- Model training & fine-tuning1.00
- Generative Building1.00
- Production-grade code0.98
- Evals & Observability0.98
- Model (GBM)1.00
- Vector DBs & RAG1.00
- LLMs (tokenization, prompting, context management)0.99
- Agent building (ReAct, tool-use, workflows, multi-agent)0.99
- spatial intelligence0.98
- Model routing (fal.ai, OpenRouter)0.99
- Building robust non-deterministic features (testing &1.00
- fault-tolerance)1.00
- AI Engineer1.00
- Backend system design0.97
- Streaming architectures (SSE)1.00
- Reactive UIs0.98
- In-memory state management1.00
- ing agents0.99
- Full-stack SWE1.00
- A & D × O0.52
- End presentatio0.99
-
- 回0.53
- Researchers ≠ production engineers0.98
-
- 回0.53
- THE THESIS1.00
- Getting research to reality is a1.00
- systems & process0.94
- problem.1.00
- 011.00
- 021.00
- 031.00
- Make research legible1.00
- A monorepo that can0.99
- Decomposition as a1.00
- receive new research1.00
- design problem1.00
- A handoff contract maps out a0.98
- Modularized code and templates1.00
- Prototype decomposition + review1.00
- project so an engineer can see the1.00
- that give every prototype a clean1.00
- needs to be planned so that giant1.00
- shan0.93
- ork1.00
- home1.00
- PRs ship without big-bang risk1.00
-
- Scaling Engineering Teams1.00
- via RFCs: WritingThings1.00
- Down1.00
- by GergnlyOrosz0.89
- Homme0.83
- I have recently been talking at small and mid-size companies, sharing1.00
- Nomaktuar0.57
- engineering best practices I see us use at Uber, which I would recommend0.99
- Popular Atticien0.84
- The Software Engineer's0.93
- Guadeboo0.63
- My Booka0.86
- the most raised eyebrows, as well the mos "aha!" moments is the one on0.99
- how the planning process for engineering has worked since the early0.99
- any tech company adopt as they are growing. The one topic that gets both0.99
- Eadsuends0.81
- years of Uber.0.99
- Reallng Lat0.75
- Ethicsstanment0.75
- When working at large comanies like Microsoft or smaller ones like0.98
- Writr a guest article0.86
- Seanagrs0.64
- imatngg0.50
- Now0.62
- Gontactmme0.86
- built the same thing as my team. Second, the tech and architecture debt1.00
- approach-wise and quality-wise.1.00
- Skyscanner, there have been two things related to planning that have0.99
- always bugged me. First, the lack of visibility on others building or having0.99
- accumulated due to different teams building things very differently, both0.99
- BSS.Fend0.79
- hlansky0.68
- What if I aid there is a way to tackle both these issues pretty well, using a0.98
- twitts0.65
- few simple steps? A word of warning, one of the steps will sound a little0.99
- crazy. Here they are:0.98
- Subscribe xia.omall0.76
- 1. Do planning before building something new. This can be in-person0.99
- S Subscribe in a prader0.86
- whiteboarding or just talking it through with the team members, as1.00
- long as you're all clear on how you will get things done.0.99
-
- LEVER 01 THE TEMPLATE0.95
- What an RPT forces you to answer1.00
- Domain context1.00
- Business goal1.00
- Type safety1.00
- Parti diagrams, masks, latent spaces0.98
- The consumplion model: headless0.98
- Input/output contracts and the guardrails0.98
- explained for a great SWE who knows0.99
- microservice. standalone agent, or0.98
- preventing client/server schema dift0.94
- nothing about homebuilding.0.99
- embedded Studio tool? It reorganizes the0.99
- whole build.0.96
- Persistence1.00
- Architecture1.00
- Decomposition1.00
- DBs, model weights, checkpoints — how0.98
- they're versioned and where they live.0.99
- Map to existing serving pattemns. Describe0.97
- only where you depart from them.0.99
- prototype into reviewable, mergeable0.98
- A roadmap for slicing a monolthic0.97
- modules.1.00
- training.0.95
- inference trade offs stay with speclalists.0.94
-
- 回0.72
- LEVER 01 THE TEMPLATE0.95
- What an RPT forces you to answer1.00
- Domain context1.00
- Business goal1.00
- Type safety1.00
- Parti diagrams, masks, latent spaces0.97
- The consumption model: headtess0.98
- Input/output contracts and the guardrails0.97
- nothing about homebulding.0.99
- explained for a great SWE who knows0.99
- microservice, standalone agent, or0.99
- embedded Studio tool? It reorganizes the0.99
- preventing client/server schema drit.0.87
- whole buld.1.00
- 41.00
- Persistence1.00
- Architecture1.00
- Decomposition1.00
- DBs, model weights, checkpoints — how0.97
- Map to existing serving pattems. Describe0.99
- A roadmap for slicing a monolithic0.98
- they're versioned and where they live.0.99
- only where you depart from them.0.96
- prototype into reviewable, mergeable0.99
- modules.1.00
- inference trade ofts stay with specialiats.0.93
-
- 回0.68
- LEVER 01 THE TEMPLATE0.97
- What an RPT forces you to answer1.00
- Domain context1.00
- Business goal1.00
- Type safety1.00
- Parti diagrams, masks, latent spaces0.94
- The consumption model: headless0.99
- Input/output contracts and the guardrails0.98
- explained for a great SWE who knows0.97
- microservice, standalone agent, or0.99
- preventing client/server schema drit.0.90
- nothing about homebulding.0.99
- embedded Studio tool? It reorganizes the0.99
- whole buld.0.95
- 41.00
- Persistence1.00
- Architecture1.00
- Decomposition1.00
- DBs, model weights, checkpoints — how0.98
- Map to existing serving pattems. Describe0.99
- A roadmap for slicing a monolthic0.98
- they're versioned and where they live.1.00
- only where you depart from them.0.98
- prototype into reviewable, mergeable1.00
- modules.1.00
- training.selection,0.94
- inference trade offs stay with apeclalists.0.91
-
- 日0.53
- LEVER 01 · THE TEMPLATE0.94
- What an RPT forces you to answer1.00
- Domain context1.00
- Business goal1.00
- Type safety1.00
- Parti diagrams, masks, latent spaces0.96
- The consumption model: headless0.98
- Input/output contracts and the guardrails0.99
- nothing about homebuilding.0.98
- explained for a great SWE who knows0.98
- microservice, standalone agent, or0.98
- embedded Studio tool? It reorganizes the1.00
- preventing client/server schema drift.0.99
- whole build.0.97
- Persistence1.00
- Architecture1.00
- Decomposition1.00
- DBs, model weights, checkpoints — how0.97
- Map to existing serving pattems. Describe1.00
- A roadmap for slicing a monolithic0.96
- they're versioned and where they live.0.99
- only where you depart fom them.0.99
- prototype into reviewable, mergeable0.99
- 41.00
- modules.1.00
- training.0.98
- inference trade offs stay with speclaliate.0.96
-
- THE THESIS0.98
- Getting research to reality is asystems & process1.00
- problem.1.00
- 011.00
- 021.00
- 031.00
- Make research legible1.00
- A monorepo that can1.00
- Decomposition as a0.98
- receive new research1.00
- design problem1.00
- A handoff contract maps out a0.99
- Modularized code and templates1.00
- Prototype decomposition + review0.99
- project so an engineer can see the1.00
- that give every prototype a clean0.99
- needs to be planned so that giant1.00
- shape of the work1.00
- home1.00
- PRs ship without big-bang risk0.99
- 40.99
-
- Architecture & Services1.00
- Client requests from the Higharc application are routed through an Al Gateway that behaves as a reverse proxy,0.98
- performing JWT authentication before routing the request to the appropriate backend Al/ML microservice.1.00
- Microservices are built as isolated Docker containers, which join the same bridge network called higharc-network.0.99
- The bridge network allows the gateway to use Docker's internal DNS resolution, referring to other containers by1.00
- their service name rather than their IP address.1.00
- DDEP1.00
- Embeddings1.00
- Higharc1.00
- Al Gateway0.99
- web clients1.00
- Auto-translate0.98
- portB00.97
- (future-services)0.99
- docker network: higharc-network0.98
- stide/180.61
- ×0.53
- End presantatio0.89
-
- 回0.50
- Architecture & Services1.00
- Client requests from the Higharc application are routed through an Al Gateway that behaves as a reverse proxy,0.99
- performing JWT authentication before routing the request to the appropriate backend Al/ML microservice.1.00
- Microservices are built as isolated Docker containers, which join the same bridge network called higharc-network.0.99
- The bridge network allows the gateway to use Docker's internal DNS resolution, referring to other containers by1.00
- their service name rather than their IP address.0.99
- DDEP0.99
- Embeddings1.00
- Higharc1.00
- Al Gateway0.98
- web clients1.00
- Auto-translate1.00
- portB800.86
- (future-services)0.99
- docker network: higharc-network0.99
-
- Architecture & Services0.98
- Higharc1.00
- Client requests from the Higharc application are routed through an Al Gateway that behaves as a reverse proxy,0.99
- web clients0.96
- performing JWT authentication before routing the request to the appropriate backend Al/ML microservice.0.99
- Microservices are built as isolated Docker containers, which join the same bridge network called higharc-network.0.99
- The bridge network allows the gateway to use Docker's internal DNS resolution, referring to other containers by1.00
- their service name rather than their IP address.0.99
- DOEP0.89
- Embeddings1.00
- Higharc1.00
- Al Gateway0.98
- web clients1.00
- Auto-translate1.00
- (future-services)0.97
- docker network: higharc-network1.00
- 8×0.53
- End presentatic0.82
-
- 回0.71
- Higharc1.00
- Microservice API0.99
- web clients1.00
- V0.75
- Pydantic schema1.00
- Service layer1.00
- V0.84
- Data layer1.00
- database1.00
- Microservice Application0.98
- Slide 10/180.99
- End presentatio0.96
Transcript
104 cues· 2,193 words· 12,213 chars
- 0:01 Hey, I'm Vitus and I'm a senior research engineer at HIARC on our labs team.
- 0:06 So at HIARC, our labs team is basically our research and development arm, where we have machine learning researchers who kind of explore the frontier of AI ML and figure out ways to apply it to home building.
- 0:27 Now, because we're in home building and we're in spatial reasoning, we pretty much end up needing to use a lot of what's out there in AI.
- 0:34 So computer vision to scan hand sketched floor plans and parse them into our internal data model.
- 0:43 Reasoning agents to kind of carry the user through these agentic experiences.
- 0:47 We have custom transformers.
- 0:50 We use diffusion models for image gen and basically a lot of kind of what's out there in AIML we end up needing to use because of the kind of multidisciplinary nature of our product.
- 1:06 So hopefully that video gives you some idea of the research areas that we focus on that I've mapped out here.
- 1:13 And that kind of leads us to the problem, which is
- 1:18 In this challenge of getting frontier research into production, we need to start working with software engineers, let's say platform engineers, infrastructure engineers, back end engineers who are very familiar with building robust and production grade code.
- 1:36 but are likely not familiar with the methodology and research in computer vision in training your own llms and even some top secret uh topics that i can't reveal here now you kind of have the flip side problem with our ml researchers who are very up to date with the latest papers and can pull together these concepts in novel and creative ways to develop new features
- 2:02 But they have not really worked as software engineers typically where they've been responsible for production grade APIs.
- 2:12 That's kind of what I want to get into is this this handoff, this this baton pass of how do we facilitate that and how do we do it productively?
- 2:22 And we look at this basically as a systems and process problem.
- 2:27 And I want to kind of zero in on three main focus areas that you can use to improve this the velocity of teams that are bringing research into production.
- 2:42 So the first thing we'll look at is research legibility.
- 2:45 So let's say you have a ML researcher who has produced a prototype concept.
- 2:52 How can they map that out and make it digestible and understandable for these different software engineers and product managers who are going to be jumping into the project?
- 3:02 The second is how to structure your code base.
- 3:05 So we use a monorepo.
- 3:07 How do you arrange your code and modularize it so that it's ready to receive these new prototype concepts and turn them around and stand them up quickly?
- 3:17 And then the third thing we'll look at
- 3:20 is basically making the jump from a mapped out, proven out research prototype that's going to be going into this repo.
- 3:29 How do you decompose that prototype and stand it up on best practices in software engineering?
- 3:37 So taking a look at the first step, there's a really good article that I want to link, a blog from the pragmatic engineer.
- 3:45 where he talks about software engineering teams as they grow and start to scale, how important it is to write out technical design documents, request for comment, whatever you want to call it, specifications before building software as a way to align teams.
- 4:04 So we have a very analogous document that we require from all research prototypes that we call the research prototype taxonomy document.
- 4:16 So it's really just a technical design document from software engineering with some specific twists that make it more digestible given its nature in machine learning.
- 4:30 So the first thing in that document, we use Notion for this, but obviously any written document will work, is that we start with the kind of domain context and like, what are the domain specific, you know, we're in the architectural domain in home building.
- 4:44 So what are these kind of novel ways to represent data?
- 4:47 Maybe it's a party diagram.
- 4:49 Maybe it's a graph to represent the kind of circulation graph through a home.
- 4:55 Maybe it's embedding models or latent space representations.
- 5:00 I like to say kind of picture a software engineer who just we just hired from JP Morgan.
- 5:06 What are the kind of specific lingos and data representations that they might need to know before jumping into this project?
- 5:14 The second is kind of mapping out the business goal, right?
- 5:17 What's the why to solving this problem matter and what's the value in this ML tool?
- 5:23 And then the four remaining parts of this document are pretty kind of conventional software engineering principles you might see in a TDD.
- 5:32 So the type safety.
- 5:35 So we'll see later what our machine learning repo looks like.
- 5:40 But what is the type contract between our core product repository
- 5:44 and this machine learning repo?
- 5:47 How are those types shared and how do they stay in sync?
- 5:51 Then kind of mapping out the persistence layer.
- 5:54 Is there a database?
- 5:55 This is an area where
- 5:58 We think it's probably best not to have our researchers spend too much time in the persistence layer and just map out how far they got.
- 6:07 And this is a great first entry point once we start bringing in software engineering help on the project.
- 6:15 Then kind of mapping out the overall system architecture.
- 6:19 Is this a workflow?
loading