Videos 4TxOBhDRRCM
OpenRAG: An open-source stack for RAG — Phil Nash
Scene timeline
67 shot(s).
keyframes kept every frame deduplicated
What was stored
- cues
- 173
- whisperx 173
- chunks
- 28
- from 173 cues
- keyframes
- 55
- kept of 67 captured
- frames with text
- 55
- 2,966 lines read
- chapters
- 0
- from the source metadata
- keyframe bytes
- 8.3 MB
- word timings on 173 cues
Provenance
| stage | state | model | started | took |
|---|---|---|---|---|
fetch |
done | — | 2026-08-10 19:06 | 1m 26s |
stt |
done | — | 2026-08-10 19:07 | 17s |
chunk |
done | — | 2026-08-10 19:07 | 0s |
text_embed |
done | — | 2026-08-10 19:56 | 0s |
keyframe |
done | — | 2026-08-10 19:07 | 1m 13s |
ocr |
done | — | 2026-08-10 19:09 | 52s |
frame_embed |
done | — | 2026-08-10 19:56 | 9s |
Frames, and what the machine read
-
- Phil Nash0.99
- Developer relations engineer1.00
- IBM1.00
- x.com/philnash1.00
- linkedin.com/in/philnash1.00
- philna.sh/links1.00
-
- RAG is dead0.99
-
- RAG is solved0.97
-
- RAG is hard1.00
-
- One RAG1.00
- doesn't fit all0.99
-
- OpenRAG1.00
- 公0.61
- OpenSearch1.00
-
- Ingestion1.00
-
- Docling1.00
- Document parsing1.00
- Docling1.00
- {0Oc}0.81
- • PDFs and images0.99
- • HTML, Markdown, DOCX, XLSX,0.99
- PPTX, CSV0.96
- • Audio and video0.96
- • ...and many more0.89
- ASCII0.94
-
- ASR0.76
- Audio/1.00
- Movie1.00
- Audio1.00
- Document1.00
- Assemble1.00
- VWLM0.65
- pdf1.00
- images1.00
- Page images0.96
- Document1.00
- Assemble1.00
- ##0.99
- (}0.78
- Adobe0.97
- Standrd0.73
- PDF0.75
- PFipeline0.65
- Export JSON,1.00
- Markdown,1.00
- Figures,1.00
- HTML,1.00
- pdf1.00
- images1.00
- Parse PDF1.00
- pages1.00
- OCR1.00
- Analysis1.00
- Layout1.00
- Structure1.00
- Table1.00
- Document1.00
- Assemble1.00
- Docling1.00
- Document1.00
- Create1.00
- datasets1.00
- XML1.00
- ascii0.92
- Pommmatc0.73
- md1.00
- html1.00
- Patents..1.00
- AsciiDoc1.00
- Piupeline0.66
- Chunking1.00
- Parse Markup1.00
- Format1.00
- Document1.00
- Assemble1.00
- for RAG1.00
- docx1.00
- pptx1.00
- xlsx0.99
-
- OpenSearch1.00
- Indexing and search1.00
- • Vector0.98
- • Keyword0.94
- OpenSearch1.00
- • Filters0.99
- • Multiple embedding models0.97
-
- OpenSearch1.00
- Indexing and search1.00
- • Vector0.98
- • Keyword0.94
- OpenSearch1.00
- • Filters0.99
- • Multiple embedding models0.97
-
- JVector1.00
- No fancy logos0.98
- C0.84
- # github.com/datastaxjvector0.85
- 回☆0.83
- U0.88
- datastax / jvector0.95
- Q Type to search0.97
- +.0.87
- one0.60
- <) Cade0.80
- Issues 410.89
- I' Pull requests0.94
- 110.99
- Actions0.93
- Projects0.98
- ⑦ Security0.86
- 区Insights0.95
- os jvector Pubic0.91
- © Watch 340.91
- ￥ Fork 1490.94
- ☆ Star 1.7k0.85
- P main P Branches Tags0.91
- Q. Go to fie0.84
- Add file0.93
- <) Code0.83
- About1.00
- Markwolters Fix for bug causing tests to se FP vectors in earch (8645)0.92
- 18488b8 - 3 days ago0.88
- 923 Commits0.94
- embedded vector search engine1.00
- JVector: the most advanced1.00
- github0.98
- virtualize DataSet (r580)0.80
- 3 months ago0.98
- .mwn0.63
- Add mvn wrapper0.99
- 3 yeears ago0.90
- benchmarks-jmh1.00
- Refactor parallel writer (608)0.95
- last morth0.98
- Readme0.97
- docs0.99
- Added Bink to JV tutorials in the README (4644)0.96
- 3 weeks ago0.96
- Apache-2.0 license0.96
- ivector-base0.97
- Retactor parallel writer (e608)0.94
- last month0.99
- Custom properties0.97
- Ar Activity0.90
- ivector-examples0.94
- Fix for bug causing tests to use FP vectors in search (#645)0.98
- 3 days ago0.98
- ☆1.7k stars0.99
- ivector-multirelease0.97
- Release 4.0.0-rc.10.95
- 9 months ago0.98
- ② 34 watching0.92
- 149 foks0.84
- ivector-native0.88
- Move buffer position in Bytettufferindewiternferiefloats..0.80
- last month0.99
- Audit log0.95
- vector-bests0.97
- add DataSetinfo and some guards around loading errors (..0.98
- 3 days ago1.00
- Report repository0.97
- vector-twenty0.96
- Introduce run.ymi policy and run-scoped experiment loggi..0.95
- last month0.97
- Releases1.00
- siftsmall1.00
- SiftSmall example0.99
- 3 years ago0.94
- 54 tags0.96
- gitignore0.90
- Add quantization timing metrics to YAMI. benchmarking (#....0.95
- last manth0.95
-
- Langflow1.00
- Drag, drop, done0.98
- d782831-478c-4264-b3c8-128br700.95
- "0.61
- OpenSearch Ingestion Flow0.99
- a0.92
- • Visual0.97
- • Customizable0.99
- D0.87
- • Extensible0.98
-
- Langflow1.00
- Agentic retrieval1.00
- -6649-4e1d-aed1-b77249fb8dd00.95
- 图☆0.70
- Starter Project0.99
- OpenRAG OpenSearch Agent Flow1.00
- O108k0.64
- m 200.70
- Puayground0.94
- Share1.00
-
- Langflow1.00
- Agentic retrieval0.99
- -6649-4e1d-aed1-b77249fb8dd00.96
- 回☆0.65
- Starter Project0.99
- OpenRAG OpenSearch Agent Flow0.99
- O108k0.73
- m 2a0.69
- Puayground0.94
- Share1.00
-
- 王0.96
-
- Chrome1.00
- File1.00
- Edit1.00
- View1.00
- History1.00
- Bookmarks1.00
- Profiles1.00
- Tab1.00
- Window1.00
- Help1.00
- 90.97
- 口0.71
- 只0.93
- è0.61
- 40.83
- 8.0.66
- Wed 1 Apr 3:59 pm0.99
- OpenRAG1.00
- localhost:3000/chat0.99
- ☆0.95
- 白0.64
- P Work0.96
- OpenRAG1.00
- □ Chat0.83
- o0.83
- What is OpenRAG?1.00
- \ Knowledge0.94
- Settings1.00
- Function Call: search_documents (tool_call)0.98
- completed1.00
- Function Call: get_current_date (tool_call)0.99
- completed1.00
- Conversations1.00
- +0.99
- OpenRAG is an open-source package designed to build advanced Retrieval-Augmented Generation (RAG) systems. It integrates with key1.00
- What is OpenRAG?1.00
- components such as:1.00
- 1. Langflow - Tools for constructing and deploying Al agents.0.99
- 2. OpenSearch - An open source search engine that organizes unstructured data at scale.0.99
- 3. Docling - A document processing system offering advanced PDF understanding, OCR support, and seamless integration with LLMs.0.99
- These integrations enable powerful, scalable applications of RAG technology while leveraging proven open-source projects.0.98
- Explore Langflow's role in Al agent construction.1.00
- Discover OpenSearch capabilities for data search and indexing.1.00
- Investigate how0.99
- Y0.59
- Ask a question...0.99
- +0.92
-
- Chrome1.00
- File1.00
- Edit1.00
- View1.00
- History1.00
- Bookmarks1.00
- Profiles0.99
- Tab1.00
- Window1.00
- Help1.00
- 口0.73
- 只0.59
- è0.83
- X0.52
- Wed 1 Apr 3:59 pm0.96
- OpenRAG1.00
- +0.94
- localhost:3000/chat1.00
- ☆0.97
- 白0.69
- P Work0.94
- 20.54
- OpenRAG0.99
- 110%0.91
- Reset1.00
- Chat1.00
- o0.77
- What is OpenRAG?1.00
- li\ Knowledge0.87
- Function Call: search_documents (tool_call)0.99
- completed1.00
- Settings0.96
- Function Call: get_current_date (tool_call)0.98
- completed1.00
- Conversations1.00
- +0.95
- OpenRAG is an open-source package designed to build advanced Retrieval-Augmented Generation (RAG) systems. It integrates with key1.00
- Whatis OpenRAG?0.99
- components such as:1.00
- 1. Langflow - Tools for constructing and deploying Al agents.1.00
- 2. OpenSearch - An open source search engine that organizes unstructured data at scale.0.99
- 3. Docling - A document processing system offering advanced PDF understanding, OCR support, and seamless integration with LLMs.0.99
- These integrations enable powerful, scalable applications of RAG technology while leveraging proven open-source projects.1.00
- Explore Langflow's role in Al agent construction.1.00
- Discover OpenSearch capabilities for data search and indexing.1.00
- Investigate how1.00
- Ask a question...1.00
- +0.99
-
- Chrome1.00
- File1.00
- Edit0.99
- View1.00
- History1.00
- Bookmarks1.00
- Profiles1.00
- Tab1.00
- Window1.00
- Help1.00
- 只0.96
- è0.79
- 00.67
- 4x0.52
- Wed 1 Apr 3:59 pm0.95
- OpenRAG1.00
- x0.58
- +0.87
- localhost:3000/chat0.98
- a0.64
- ☆0.92
- 白0.53
- P Work0.95
- OpenRAG0.99
- ※0.63
- Chat1.00
- Function Call: search_documents (tool_call)1.00
- completed1.00
- lil\ Knowledge0.87
- Settings0.93
- Arguments:1.00
- Type:1.00
- tool_call1.00
- 40.95
- Conversations1.00
- +0.96
- "search_query": "OpenRAG"0.98
- Whatis OpenRAG?0.97
- Result:1.00
- 00.85
- Function Call: get_current_date (tool_call)0.99
- completed1.00
- OpenRAG is an open-source package designed to build advanced Retrieval-Augmented Generation (RAG) systems. It integrates with key1.00
- components such as:0.99
- 1. Langflow - Tools for constructing and deploying Al agents.0.99
- 2. OpenSearch - An open source search engine that organizes unstructured data at scale.0.98
- 3. Docling - A document processing system offering advanced PDF understanding, OCR support, and seamless integration with LLMs.0.98
- These integrations enable powerful, scalable applications of RAG technology while leveraging proven open-source projects.1.00
- Explore Langflow's role in Al agent construction.1.00
- Discover OpenSearch capabilities for data search and indexing.1.00
- Investigate how0.98
- Ask a question...0.99
- +0.97
Transcript
173 cues· 2,729 words· 15,012 chars
- 0:01 Hi there, my name is Svom Ash and I'm a Developer Relations Engineer at IBM.
- 0:05 I've been working on tools around AI and RAG for the last couple of years and I've got something I'd like to show to you today.
- 0:14 Now, first things first, I've heard that RAG is dead many a time, and I'm sure you have too.
- 0:20 Context windows are huge these days, so you might as well just dump all of your information into there.
- 0:25 I don't take this kind of thing very seriously.
- 0:28 If every business has less than a million tokens worth of data, then sure, RAG is dead and probably so are all those businesses.
- 0:36 And of course, not everyone is happy paying for a million input tokens every time you want to ask a question as well.
- 0:44 Instead I sort of hear that the RAG is dead claims as more of RAG is solved, right?
- 0:49 We think we understand the process and we can just apply RAG when we need to.
- 0:53 You just, you know, gather up all your unstructured data, extract the text, chunk it up, embed it, throw it into a vector database, and then when you want to ask your agent a question you just embed that question, search the database, pick the top K results and pass them to a model as context.
- 1:07 It's just a footnote in context engineering these days.
- 1:13 But it turns out that the rag is actually hard.
- 1:15 And it's hard for different reasons for different projects.
- 1:19 You know, PDFs are a pain.
- 1:21 Chunking strategies are a hassle and changing them and testing them is difficult.
- 1:25 Embeddings keep improving, which is great for the industry, but not very great when you've used something from six months or a year ago.
- 1:32 There are new search techniques all the time.
- 1:34 And further tweaks that you can add to your pipeline to improve the results, like adding summaries to chunks, performing chunk expansion, using a cross-encoder to re-rank results, query rewriting, there's so much more.
- 1:46 RAG is quite complex.
- 1:47 In fact, everyone's documents are different.
- 1:50 Every system will have different users, different questions, different interaction patterns, and different expectations.
- 1:57 While every RAG system will ultimately be different, there are definitely some core components that are required.
- 2:04 When building a RAG system, it's useful to have a high-quality baseline to build from.
- 2:09 And so that's what we've been working on at IBM.
- 2:11 We've brought together three existing open-source projects to create a RAG stack that is powerful, easy to use, and easy to extend.
- 2:19 And the project's called OpenRAG.
- 2:22 And it uses the open source Dockling for document processing, OpenSearch for search indexing, and Langflow for visual orchestration and agents.
- 2:32 OpenRAG is an open source project that you can try out today to build your own powerful, customizable, and easy to use RAG system.
- 2:40 But I just want to break down the stack for you so that you understand the components and how they work together and how they create a stack that is flexible enough for your modern RAG requirements.
- 2:51 Let's start by looking at the ingestion side of RAG.
- 2:54 Let's start where it all begins, document processing.
- 2:57 Ingesting PDFs, HTML, Word docs, slides, and more can be a pain.
- 3:02 But the biggest pain of all is, of course, PDFs.
- 3:05 Docling is an open-source project that was built out of IBM Research in Zurich, and it processes and parses all sorts of documents, from HTML, Markdown, and Word documents, through to slides and spreadsheets, audio and video, and even that enemy of all RAG systems, PDFs.
- 3:23 DocLink has a number of different pipelines that handle different file types.
- 3:27 This allows it to be flexible in the way it takes in documents and accurate in its output.
- 3:35 So there is a simple pipeline that handles those mostly straightforward text documents like Markdown, HTML and Word.
- 3:41 That just extracts the text, turns it into a hierarchy and outputs a document.
- 3:47 For audio and video, there is an ASR, an automatic speech recognition pipeline.
- 3:52 And for PDFs, there's two available pipelines.
- 3:55 The standard pipeline has a number of small focused models that do different things like extracting text, tables and images from PDFs.
- 4:03 You can even choose an OCR backend to read text, which is particularly useful for scanned documents that don't have actual real text in them.
- 4:11 So this collection of small models in a pipeline perform things like layout analysis, table extraction, image extraction, descriptions.
- 4:19 This gives you a wide array of options to get the best out of those documents.
- 4:23 There's also a VLM, a vision language model pipeline that uses the Granite Dockling 258 million vision model to extract all of that in one go.
- 4:32 This is a newer pipeline, but it is simpler as it is this just all-in-one model that's trained specifically for this task.
- 4:42 Dockling extracts text and then produces an intermediate representation, a Dockling document, which models the structure of a document in an XML-ish format called Docktags.
- 4:53 Those Docktags can then be converted to a number of formats, including Markdown, HTML, and JSON.
- 4:59 And then Dockling also has a chunker that uses the hierarchy generated by the parsers and built into those Docktags to produce hierarchically understood chunks of text.
- 5:14 Moving on to embeddings, OpenRAG actually isn't very prescriptive with embeddings at all.
loading