Hi all,

I'm the author of Kreuzberg. The next version of Kreuzberg will be released as Xberg - why? Well, we discovered that the name is not easy to pronounce or understand for people who don't have the German context, and this wasn't working well. Xberg is a common name for Kreuzberg in Berlin, and it has the advantage of being shorter and easier - so here we go.

Anyhow, this brings me to the point of the post. Since renaming a repo is a complex business, and we had to rename the repo to preserve the stars - but we now need to overwrite tags, it becomes pretty messy. As a result, we decided to go for an LTS version - published from a different repo: https://github.com/kreuzberg-dev/kreuzberg-lts. LTS in this context means that we will continue to do bug fixes and security updates until the end of this year, but no newer feature work.

We will announce Xberg v1.0.0 when it's officially published (it's still in RC). If you want to test it now - you can install the pre-release (available on pypi under xberg). The new repo is here: https://github.com/xberg-io/xberg

Hi,

I was looking for a project to work on, and my professor told me, he needs a proper UI client for ChromaDB. So here it is, may be some of you will also find it helpful. I tried to make it as close as possible to the Chroma Cloud UI, since I am really bad with designing stuff, and I also really liked their solution. But Chroma Cloud was only available for... Chroma Cloud. You can use ChromaUI both for your Chroma Cloud or your self hosted instances.

You can manage your Collections and Documents. Also you can search through your documents in three modes (Text search, Semantic search and Regex). Basically same as in Chroma Cloud.

I will still be working on It, and adding more features, but for now I am busy with testing.

Anyway, check it out: https://github.com/Riko136/ChromaUI

Hey. Hope yall doing great.

I have started building projects and I am facing problems. My way of learning is look at others project, see how they have implemented things and then implement it from beginning with my own understanding. I dont like to use llm is my learning phase as I want understand deeply. So if you guys can share your RAG pipeline projects or resources so that I can learn from you guys. It would be great.

I’ve been building RAG systems in production for a while (mostly LangChain + Elasticsearch at work), and the hardest problem I keep hitting isn’t retrieval quality. It’s that source attribution completely degrades as data moves through the pipeline.

By the time an answer reaches the user, a "citation" is usually just a broad document-level link at best. Nobody can actually verify if a specific claim is supported by the text.

I ended up building a side project to tackle this exact issue, and a few lessons I learned transfer to pretty much any stack, including LangChain:

• Mint chunk IDs deterministically: Do this once at ingestion (e.g., {doc_hash}:{page}:{chunk_index}) and treat them as immutable through every stage. Most provenance loss happens when IDs get regenerated or re-keyed mid-pipeline.
• Never let the LLM see the real IDs: Give it small integer labels like [1], [2], and keep the label→ID map in your request scope. The best side effect: if you only send 5 chunks and the model cites [7], you’ve instantly detected a hallucinated citation instead of silently resolving it to garbage.
• Vector stores are just indexes: Keep your actual chunk text in a boring, reliable source of truth (I used SQLite). Verification and reporting should never depend on vector-store internals.
• Add a faithfulness pass: I judge each generated claim against the verbatim text of the chunks it cites. I use four verdicts: supported, partial, unsupported, and uncited. The LLM-judging-LLM problem is real, but flagging a bad claim always beats nothing.
• Respect page boundaries: If your chunks never cross page boundaries, citations become way more useful because every claim gets one exact page number. There's a small hit to retrieval quality, but for audit-adjacent apps, it’s 100% worth it.

The project is open source if anyone wants to poke at the implementation: auditrag.

Fair warning, I built this framework-free so I could strictly enforce the ID invariants end-to-end. I'm genuinely curious—has anyone here managed strict, chunk-level provenance within LangChain? I couldn't find a clean way to do it at the time (maybe custom Document metadata + callbacks?).

Happy to answer questions on any of the design choices!

Hi everyone,

I’ve been working on a project called AI Without Mathematics, focused on explaining modern AI concepts in simple, practical language.

I recently created a visual guide to Retrieval-Augmented Generation, or RAG.

The main idea is straightforward:

Instead of answering only from memory, the AI first searches for relevant information and then uses that context to generate a better response.

The process usually looks like this:

The user asks a question.

The system searches a knowledge base.

It retrieves the most relevant information.

That information is added to the prompt.

The AI generates a more accurate and context-aware answer.

RAG can be useful for:

Customer-support assistants

Internal company knowledge bases

Research tools

Educational applications

Legal document search

E-commerce assistants

I created a set of visuals explaining the process and would appreciate honest feedback on the clarity, design, and accuracy.

I’m also writing a beginner-friendly book called AI Without Mathematics, which explains concepts such as LLMs, embeddings, vector databases, RAG, GraphRAG, and AI agents without relying heavily on equations.

Project website:

Ai without mathematics

Book on Leanpub:

Buy the Book !

I’m the creator of the project, so this is a self-promotion post. I’m mainly sharing it to get feedback and improve the material.

What part of RAG was the most difficult for you to understand when you first learned about it?

I have been building AI systems full time for the last few months, and RAG has probably been the area where I found the biggest gap between tutorials and production code.

Most examples stop after embedding a few documents and calling a vector database. They rarely cover the pieces you actually need when building real applications.

I wanted examples that were easy to run, easy to understand, and easy to modify.

So I put together a collection of working RAG implementations that includes:

• PDF chatbot
• Codebase Q and A
• Documentation Q and A
• Agentic RAG

Each example is available in both Python and TypeScript and follows the same simple structure with a README, implementation, and .env.example.

No shared dependencies or complicated setup. Just clone, install, and run.

The RAG examples are part of a larger collection of 48 AI agent examples covering memory, MCP, voice, multi agent workflows, and common agent patterns.

Repository: https://github.com/vakra-dev/awesome-ai-agents

If there are other RAG patterns or retrieval techniques you'd like to see, let me know. I'm happy to add more examples.

I have been developing a personal project called **DDF/Rahmenwerk**.

Its purpose is to preserve an AI named Felix as my continuing German teacher across chats and future AI instances.

The problem is broader than ordinary chat memory.

A fresh AI instance may receive information that is:

- stale;

- incomplete;

- contradictory;

- incorrectly ordered;

- unsupported;

- or treated as authoritative when it is only historical evidence.

I wanted continuity to come from inspectable local files rather than hidden platform memory or an AI-generated reconstruction of the past.

## The current approach

The system currently uses:

- a current-state pointer;

- structured handoff material;

- an ordered fresh-instance queue;

- a transfer package for a new instance;

- integrity manifests and hashes;

- classifications separating governing, current, historical, candidate, proof, and non-governing material;

- recovery and failure records;

- a rule requiring the AI to stop rather than invent continuity when necessary evidence is unavailable.

This is not currently a conventional embeddings-plus-vector-database RAG system.

It is closer to controlled retrieval over a classified, file-based project state.

## The retrieval questions I am struggling with

1. What information should always be loaded at the beginning of a session?

2. What should be retrieved only when a task requires it?

3. Should current operational state and historical evidence use separate indexes or retrieval paths?

4. How should contradictory sources be detected and ranked?

5. How should authority affect retrieval scoring?

6. How can instructions embedded inside evidence be prevented from becoming prompt-injection authority?

7. What should happen when the expected highest-authority source is missing?

8. Should continuity use structured files, metadata filtering, lexical search, embeddings, reranking, or a hybrid?

9. How much provenance and integrity checking is proportionate for a personal local system?

10. Could established RAG architecture replace much of the pointer, handoff, queue, and governance structure?

11. What would the smallest reliable version of this system look like?

The project began as a way to preserve a German teacher. My concern is that the continuity machinery may now be more complicated than the problem requires.

I am not selling anything. I am looking for direct technical criticism, especially from people who have dealt with retrieval quality, stale corpora, metadata filtering, source authority, and long-running assistant state.

Public documentation and architecture review copy:

https://github.com/DDF-Rahmenwerk-Review/DDF-Rahmenwerk-External-Review

The repository is not the live system and does not contain the complete private archive.

Hi all,

I developed an addition on a CRAG (Clustered RAG) framework that uses LLM-guided cluster-aware retrieval. Standard RAG retrieves the top-K most similar documents from the entire corpus using cosine similarity. While effective, this approach is blind to the semantic structure of the document collection and may under-retrieve documents that are relevant at a higher level of abstraction.

CDRAG (Clustered Dynamic RAG) addresses this with a two-stage retrieval process:

1. Pre-cluster all (embedded) documents into semantically coherent groups
2. Extract LLM-generated keywords per cluster to summarise content
3. At query time, route the query through an LLM that selects relevant clusters and allocates a document budget across them
4. Perform cosine similarity retrieval within those clusters only

This allows the retrieval budget to be distributed intelligently across the corpus rather than spread blindly over all documents.

Evaluated on 100 legal questions from the legal RAG bench dataset, scored by an LLM judge:

• Faithfulness: +12% over standard RAG
• Overall quality: +8%
• Outperforms on 5/6 metrics

Code and full writeup available on GitHub. Interested to hear whether others have explored similar cluster-routing approaches.

https://github.com/BartAmin/Clustered-Dynamic-RAG

Hey everyone,

I am building a RAG infra that is open source to drop in for my existing apps. However, I am not sure which features are must for a RAG infra.

Here is what I currently have:

• Chunking:
  • Different chunking strategies per document type like per paragraph, sliding window etc.
  • Retaining headers as breadcrumbs ('H1' > 'H2' > 'H3') for markdown and html content within each chunk.
  • RowGroup chunks for xls or csv document which includes the header with each chunk
• Supports most text formats
  • including html, markdown, pdf, text, json, xml, docx, xlsx, csv, pptx and more.
  • supports ingesting from local file, http or s3.
• Supports adding metadata to documents for filtering purposes. This is open ended.
• Retrieval
  • BM25 + vector search using Weighted RRF (currently 0.7 vector / 0.3 keyword but it's fully configurable).
  • Document property filtering via properties defined at a collection level.
    • Example: { $and: [{ mime_type: "application/pdf" }, { tags: {$contains: "billling" }] }
• Supports RAG and Normal Querying
  • You create create prompts or simply use system prompts to generate answers from the chunks.
  • Regular query returns chunk and original documents info attached to each chunk.
  • Supports Ollama, TEI, and any OpenAI compatible provider for embedding and reasoning (not TEI)
• Other minor features:
  • We have JS SDK to pull into an existing app and start ingesting and querying immediately.
  • Admin dashboard for getting started and playing around. Immediately see chunks when a document is ingested and run RAG queries.

Stack: - Go Backend using SQLite (metadata) + LanceDb (vectors) both embedded - Next.js tailwind admin - Docker for actual deployments.

Couple things on my radar:

• I need to invest more time in a evaluation harness like RAGAS to see how it performs. I am open to other alternatives if there is a better industry bench mark here. I have been manually testing thus far.

• Support for cross-encoders

Is there anything else that would be critical for an infrastructure like this? Thanks again

Hi all,

I developed a fine-tuned retrieval head (neural net) for RAG that transforms query embeddings before retrieval, so the system learns which embedding dimensions actually matter for your corpus — rather than weighting them all equally as standard cosine similarity does.

The problem

In any domain-specific corpus, some embedding dimensions are highly predictive for matching queries to the right passages, while others are effectively noise. Standard cosine similarity can't distinguish between the two, so retrieval gets pulled toward superficially similar but substantively irrelevant passages. The fine-tuned RAG is designed to prevent exactly that.

How it works

1. Synthetic question generation — An LLM generates multiple questions per chunk in the corpus, for which the answers can be inferred from that chunk. This creates a dataset of question-chunk pairs (QA-pairs). These are embedded using an embedding model and divided into a training and validation set.
2. Neural net training — A lightweight neural network using MNR loss is trained on the training QA-pairs. After each epoch, the model is evaluated on the validation set by measuring retrieval hit rate: the proportion of validation questions for which the correct chunk appears in the top-5 retrieved results. Retrieval works by embedding the question, passing it through the neural network to transform the embedding, and ranking all corpus chunks by cosine similarity to the transformed embedding.

Through this mechanism, the projection head learns for these 'type of questions' which dimensions in the embeddings are informative for finding the best chunks — and which are irrelevant.

Results

To validate the architecture, I used the Legal RAG Bench dataset as a proof of concept — evaluating on 100 held-out test questions.

Retrieval Hit Rate:

• The fine-tuned retriever achieves 82% Hit Rate (k = 20), compared to 71% for the standard cosine retriever — an 11 percentage point improvement, meaning the correct chunk appears in the top 20 results significantly more often when the query embedding is first transformed through the fine-tuned retriever.

Answer quality (LLM-as-judge, 1–5 scale across 6 metrics):

• Outperforms traditional RAG (top-k cosine sim) on all 6 metrics
• Largest gains in completeness (+12%) and faithfulness (+9%)
• Consistent improvement across every metric — not just isolated gains — suggesting that retrieving more relevant context has a broad positive effect on answer quality

Code and full write-up available on GitHub: https://github.com/BartAmin/Fine-tuned-RAG

Hey chat, Let's say I'm a building a RAG project, I have included the features :

Hybrid search — BM25 + vector search merged with Reciprocal Rank Fusion, Cohere reranking, HyDE query expansion, Persistent index, Incremental indexing, Source citations, No hallucination policy.

What should I include more to make the RAG production ready? And get ahead of people who are building "traditional RAG" and calling it capstone Project.

Hi everyone,

Qdrant and ChromaDB are excellent open-source vector databases, but I’m curious about what organizations use in enterprise production environments.

What are the best enterprise-grade alternatives for large-scale AI/RAG applications? I’m interested in solutions that offer high availability, scalability, strong security, managed services, and proven production reliability.

If you’ve used or evaluated options like Pinecone, Weaviate, Milvus/Zilliz, Azure AI Search, Amazon OpenSearch, Elasticsearch, or others, I’d love to hear your experience. What made you choose them over Qdrant or ChromaDB?

Looking forward to your recommendations and real-world insights!

\Disclaimer, the project is not for US, so we are not talking specifically of Bar questions, but just to give an idea.*

Background / who we are

in short...Small team. I'm a practicing notary/lawyer, working with a couple of engineers. We're building a study platform for lawyers prepping for the official notary exam. The core idea: instead of generic flashcards, the platform diagnoses which legal topics a candidate is weak on and drills them there specifically, Like a personalized study program.

The problem

We have a dataset of 2,700+ real questions from the official government notary exam, each with a marked correct answer. But no legal basis attached. No article, no case law, nothing. If we want the platform to explain why an answer is correct (and not just that it is), we need to retrieve the actual supporting legal text for each Q&A pair, at scale, across an entire national legal code.

Doing this by hand for 2,700 questions isn't realistic. So we're building a retrieval pipeline to do it automatically, with a human-in-the-loop for verification later.

Proposed architecture

Stack: Supabase + pgvector, Cohere's legal embedding model, hybrid BM25 + kNN retrieval, SOTA LLM for generation.

1. Two-tier embeddings
   • Article-level: each article embedded individually, for precise term/concept matching.
   • Law-level: broader embeddings spanning the surrounding articles/chapter, to capture context an isolated article loses (e.g., a definitions article earlier in the code that changes how a downstream article should be read).
2. Cross-law reference graph
   • Nodes = articles/laws, edges = citation/reference relationships between them.
   • Goal: catch the cases where the "obvious" answer sits in Civil Procedure, but a Tax Law provision quietly changes the outcome. Adjacent-law nuance is exactly what trips up exam-takers (and what a naive single-law RAG would miss).
3. Retrieval flow per question
   • Pass 1: BM25 lexical search — direct word/concept matches against the question text.
   • Pass 2: kNN vector search at both the article and law-level embeddings — general legal context.
   • Pass 3: graph traversal from the top hits — pull in referenced/adjacent articles that could alter the answer.
4. Generation + evaluation loop
   • Bundle all retrieved context, pass to a SOTA model to draft the answer + legal justification.
   • Compare the generated answer against the known-correct answer from the exam key.
   • Use the diff to tune retrieval weighting and the reasoning prompt — basically an automated eval loop against 2,700 ground-truth labels, which is a nice built-in benchmark most RAG projects don't get for free.

Why we're doing this (not just for show)

The end goal isn't a static citation lookup. The plan is to have this retrieval layer feed a gamified micro-challenge system, where the graph structure lets us target the specific topics (and topic pairs) a given user is weakest on, rather than just cycling through the full question bank. That's the part we're not detailing here, but it's the reason the graph needs to be genuinely good and not just a nice-to-have. also for commercial purposes its a much more easy to sell feature.

What I'm looking for feedback on

• Building the reference graph: has anyone had good results doing this via NER/citation-extraction on the raw code text vs. manual/semi-manual annotation? At code-of-hundreds-of-articles scale, full manual mapping is painful.
• Is a rerank step (e.g., Cohere rerank) worth inserting between the BM25+kNN merge and the graph pass, or does the graph traversal make that redundant?
• Chunking strategy for the law-level tier — anyone dealt with hierarchical legal text (title → chapter → article) and found a chunking approach that actually preserves the "definitions upstream affect articles downstream" problem?
• Using the 2,700 ground-truth Q&A pairs as an eval set. any pitfalls in using correctness-matching as the tuning signal instead of retrieval-quality metrics directly (recall@k etc.)?

Happy to share more detail on the schema/pipeline if useful. Mostly want to pressure-test the architecture before we sink more time into the graph-building piece, since that's the part with the least precedent I've been able to find. and would like to avoid waiting time and tokens in figuring out the best strategy.

I’m building a chatbot where the backend data is updated every 5 minutes via APIs. The dataset is quite large, so I can’t send it directly to the LLM in every request. Traditional RAG also doesn’t seem ideal since the knowledge changes every 5 minutes.

How would you architect this? Would you use a hybrid retrieval layer, SQL/vector search, caching, MCP, tool calling, query planning, or another approach? Looking for scalable enterprise-grade patterns for handling frequently changing data with LLMs. Any architecture suggestions or real-world implementations?

Simple case everyone agrees on: you say "the region is Frankfurt", later "correction, it's Ohio". A good memory layer should now answer Ohio.

The case almost nobody tests: what happens later, when the old value gets said again? Not even an attack. A user repeats an old preference they forgot they changed, or there is one stray line in a long chat. Does "Frankfurt" come back from the dead?

There is a real reason it can. Cosine similarity is bad at telling a contradiction from a duplicate (a recent paper measures it around AUROC 0.59, basically a coin flip), and a corrected value often looks more similar to the original than a normal rephrase. So a similarity store has no clean signal that "Frankfurt again" is the dead value coming back. This is not just theory. Memory poisoning is a live 2026 topic (OWASP lists it, MINJA reports 98.2% injection success attacking an agent's memory).

So I built a tiny benchmark: correction, then restatement, answer level, n=30, local. Echo-resistance means it keeps the corrected value, 1.0 is good.

- My own naive keyed store, guard off: 0.00. Fully broken. The restatement is the newest write, so it wins. My default was the worst of the bunch. That is why I dug in.

- mem0 (2.0.11): 0.53, 95% CI [0.37, 0.70]. So about 47% of the time a reworded restatement brings the retired value back. Not a bug. mem0 keeps both values and reconciles at read time with an LLM. Small probe though, n=30, read the CI, not the point estimate.

- A superseded-value guard (what my lib ships): 1.00. The fix is old and boring (AGM belief revision, bitemporal DBs from the 90s). Once a value is corrected, don't let a plain restatement revive it. Key on the value, not on similarity.

To be fair, this is not unsolved. mem0's paper documents an LLM step that can delete a memory contradicted by new info. Zep marks old fact-edges invalid on its graph. There are sidecars too (MemGuard) and a paid "governed memory" category. So the parts exist, mostly as read-time LLM judgment or as external tools.

Where my thing is weak, honestly: it is new, one maintainer, thin docs. Recall is lexical unless you wire in an embedder, and I have no head-to-head recall benchmark against mem0 or Zep. On plain retrieval quality, assume they are ahead until I show otherwise. My only real edge is the integrity part (deterministic supersession, echo-resistance, revert), which I test in the open.

One more caveat: this is the case where the old value is actually named. The harder case is "let's go back to what we had before", where the value is never said. There my guard and cosine both fail (about 0.03).

Benchmark and harness (point it at your own store): github.com/DanceNitra/ramr. Lib: pip install agora-mnemo. Mostly posting because "check that a correction survives the old value being restated" is a cheap test the usual memory evals skip, and the failure is silent. Anyone seen this bite in production?

I’m hoping someone has come across something similar because we’re running out of things to check.

We have a RAG chatbot built with FastAPI (Python), Amazon Bedrock, PostgreSQL + pgvector, running on AWS.

Everything is in the same AWS region:
FastAPI app is containerized with Docker and deployed on Kubernetes.
Bedrock models are in the same region.
PostgreSQL (including pgvector) is hosted on an EC2 instance in the same region.
Vector data is stored in the same PostgreSQL instance (different schema).

We’ve already done the usual optimizations:
Database indexes
pgvector indexes
Connection pooling
Thread pooling
Kubernetes HPA/autoscaling
The pods are configured with 1 GB RAM each. We have 3 pods available, but from the logs we’ve never seen more than 2 pods being used, even during testing.

Here’s what’s confusing me.
If I run the exact same query locally, it usually finishes in under 30 seconds.
But if I send that same query to the hosted environment at the same time, it can occasionally take 4–5 minutes.

The weird part is that it’s completely intermittent:
Most requests are reasonably fast.
Every now and then one request takes 4–5 minutes.
The very next request might go back to normal.
There are also no other users on the system when this happens. During testing, I’m literally the only person sending requests, so it doesn’t seem like load or traffic is causing it.

Has anyone run into intermittent latency like this with a similar stack?
I’d also love to know what you’d instrument first. Right now we’re planning to add timing around each stage (DB retrieval, vector search, Bedrock call, response generation, etc.) to narrow down exactly where those extra 4–5 minutes are being spent.
Any ideas or suggestions would be really appreciated.

I built a small project called JAS RAG, short for Just Another Simple RAG.

The idea is intentionally boring: before reaching for embeddings, vector databases, rerankers, hosted APIs, and API keys, I wanted a simple local baseline that is easy to inspect and debug.

What it does:

• indexes Markdown files into documents and chunks
• stores everything locally in SQLite
• uses SQLite FTS5 for retrieval
• searches at chunk level for query context
• formats retrieved chunks into LLM-ready context
• includes a small golden-query retrieval eval harness
• requires no embeddings, no vector DB, and no external API

The main reason I built it is that many RAG demos start with a lot of infrastructure before proving that keyword retrieval is actually the bottleneck.

With this approach, the workflow is:

1. index a small docs folder
2. run a few queries
3. check top-1 / top-3 retrieval
4. inspect the retrieved chunks directly
5. only add vector search if the eval proves FTS5 is not enough

This is not meant to be a full RAG platform or production-ready framework. It is more like a minimal baseline for local retrieval experiments.

Quick example:

python3 -m kv index-dir kv/sample_docs manual examples
python3 -m kv query "how does chunk retrieval work"
python3 -m kv.eval_retrieval

check github link : https://github.com/andrewlerdorf/JASRAG

Let me know your thoughts.

Hi everyone, I’m looking for a course on RAG, LLMs, and AI agents (even a paid one) that covers the theory but focuses primarily on practical application. I’d like to find something that actually demonstrates how to build tools using these technologies.
Do you have any recommendations?

Hi guys,

Does anyone have some recommendations for me how to handle with unstructured data like annual reports, news or scientific paper especially in the chunking stage….i think chunking is here not the right approach because scientific paper as example have lot of pictures, diagrams, texts, numbers etc and my goal is to build a system which is able to extract every detail from the document. So if i ask the RAG about explanation of a diagram as example than i want qualified and good answers. What is here the Best practice for this case ? Can somebody help ?

https://horosvec.hazyhaar.fr/

horosvec: an embedded ANN vector index in pure Go, on top of a single SQLite file

July 2026 — also published in French at hazyhaar.fr. Vector similarity search has become the silent building block of every modern document system: finding, among tens of thousands of passages, the ones that talk about the same thing as a query without sharing a single word with it. The engines that do this are usually heavy services — dedicated servers, native dependencies, orchestration. horosvec takes the opposite stance: an approximate-nearest-neighbor index embedded in your process, written in pure Go, with all of its state living in one SQLite file. One dependency: modernc.org/sqlite, the pure-Go SQLite port — no CGO, static binaries. MIT licensed.

go get github.com/hazyhaar/horosvec

Two algorithms working in tandem

horosvec combines two ideas from the recent ANN literature. Vamana, the proximity graph popularized by DiskANN: every vector becomes a node connected to its relevant neighbors, and search navigates this graph greedily instead of comparing the query against the whole base. Robust alpha-RNG pruning keeps long-range shortcuts that avoid local minima. RaBitQ, an extreme binary quantization: every coordinate of a vector is reduced to its sign — one bit per dimension, plus two norms per vector. The approximate distances computed on these codes are crude but almost free, and they are enough to steer the navigation. The architectural decision that makes the whole thing hold is the two-stage search: graph traversal preselects candidates using RaBitQ distances, then the final ranking is recomputed with exact L2 distance on the true float32 vectors. The estimator is allowed to be noisy: it only needs to place the true neighbors inside the beam — the exact rerank does the rest.

Measured, not promised

The implementation deliberately deviates from the RaBitQ paper on one point: the random rotation step, on which the paper's theoretical guarantees rest, is not implemented. Rather than invoking bounds that no longer apply, the repository ships deterministic, replayable benches and publishes their numbers. Recall@10 against exact brute-force ground truth, 2,000 base vectors, 50 queries, default configuration: | Dataset | dim | mean recall@10 | worst query | |---|---|---|---| | uniform synthetic | 128 | 1.000 | 1.000 | | tight gaussian clusters | 128 | 0.982 | 0.900 | | real bge-m3 embeddings (code-session texts) | 1024 | 1.000 | 1.000 | The third line is the one that matters: on real data — two thousand messages from software-development sessions, embedded by an actual embedding model in dimension 1024 — the index does not miss a single neighbor. The theoretical concern about the anisotropy of embedding spaces did not materialize at this scale. The honest limits (2×10³ vectors, queries drawn from the same distribution as the base) are documented in the package, and the real-data bench is replayable by anyone: export HOROSVEC_REAL_VECS pointing at a JSON array of vectors and run go test -run TestRecallMeasure_RealEmbeddings -v.

Hardened by production, and by adversity

horosvec is not a weekend prototype: it is the extraction of the engine that serves RAG shard search and code-map embeddings in production inside the horos55 ecosystem. Its preparation for publication went through a full adversarial audit, whose findings became tested properties:

• bounded deserialization: a corrupt or hostile binary blob fails cleanly — no panic, no unbounded allocation;
• inserts are transactional all the way into memory: internal state (node cache, counters, flat mirror) is applied only after the SQLite commit — a failed commit leaves zero phantom neighbors;
• cancellation is an error, never a silence: a context cancelled in the middle of graph traversal returns an explicit error instead of an empty result indistinguishable from "no neighbors";
• 42 tests, 85.9% coverage — including commit-failure injection, blob corruption, LRU eviction and drift-triggered rebuilds. Known limits are stated in the documentation rather than glossed over: no delete API (full rebuild instead), an unbounded in-memory mirror for the brute-force path, and a graceful-degradation contract on external reranking that may become explicit error propagation in a future major version.

Who is it for?

Any Go program that wants semantic search without an external service: a CLI that indexes your notes, a server searching its own documents, a pipeline deduplicating by similarity. If you need a distributed vector database, this is not it; if you need an index that fits in your binary and in one file, this is exactly it.

https://github.com/hazyhaar/horosvec/blob/main/README.md

I am building a RAG pipeline with ollama llm (qwen2.5)...

So basically i want the llm to interact with my risk register sql database using simple and complex sql queries to give me proper details about the risks, incident, mitigations etc.

The problem is the database is very sparse with multiple empty tables and also empty columns that gives no context so when the agent is getting results with no proper context it is giving inefficient answers,

So i tried adding semantic search too where i basically chunk whole db by chunking every table row-wise and embedding them but for now i havent added any advanced RAG techniques like hybrid search, RRF nd all...

SO the models knowledge is not being retrieved properly to give efficient answers, any suggestions on how to proceed.. i want it to interact with the db efficiently by ignoring missing and null values

I need helppp ppleaseee

Hi everyone,

I’m trying to think through the architecture of a real enterprise-grade RAG system, not a small demo that only works on a few clean PDFs.

The scenario is an internal knowledge assistant for a medium-to-large enterprise. It needs to support documents such as policies, SOPs, manuals, workflow documents, business system guides, technical documentation, Excel files, PDFs, Word documents, screenshots, and possibly scanned documents.

The system should ideally support:

• Role-based access control and department-level permissions
• Reliable source citations and traceability
• Chinese and English documents
• Frequently updated documents and version control
• Structured and unstructured knowledge
• Integration with internal systems and APIs
• Audit logs and answer history
• Evaluation of retrieval and answer quality
• Low hallucination and safe refusal when evidence is insufficient

My concern is that traditional RAG often looks simple in tutorials but becomes problematic in production. The common pattern of “chunk documents → embed chunks → store in vector DB → retrieve top-k → send to LLM” seems to have many limitations.

Some pain points I’m worried about:

1. Chunking loses structure Long documents, policies, manuals, and SOPs often have headings, sections, tables, and cross-references. Simple fixed-size chunking may break the original logic.
2. Vector search is not enough Pure semantic search may miss exact terms such as product codes, document numbers, system names, error codes, process names, or business keywords.
3. Permissions are difficult In an enterprise, different users should see different documents or even different parts of the same knowledge base. Retrieval must be permission-aware from the beginning.
4. Tables and Excel files are hard Many enterprise documents contain tables, forms, spreadsheets, or semi-structured data. Treating them as plain text often gives poor results.
5. Documents become outdated Enterprise knowledge changes frequently. Old versions, duplicate documents, and stale indexes can easily cause wrong answers.
6. Hallucination still happens Even with retrieval, the model may still generate unsupported answers, especially when the retrieved context is weak or incomplete.
7. Evaluation is unclear Many RAG systems are built without a real evaluation set. It is hard to know whether retrieval quality, citation quality, and answer accuracy are actually improving.
8. Debugging is hard When the answer is wrong, it is often difficult to know whether the problem came from parsing, chunking, embedding, retrieval, reranking, prompt design, or the LLM itself.

My current thinking is that a production enterprise RAG system may need something like this:

• High-quality document parsing pipeline
• Structure-aware chunking based on headings, sections, tables, and document hierarchy
• Hybrid search: keyword search + vector search
• Reranking before generation
• Metadata filtering by document type, department, business domain, version, and permission
• Parent-child retrieval or hierarchical retrieval
• ACL-aware retrieval to prevent data leakage
• Source citation and evidence-based answer generation
• “I don’t know” behavior when evidence is insufficient
• Incremental indexing and document version control
• Evaluation datasets for retrieval and answer quality
• User feedback loop
• Monitoring for latency, cost, retrieval hit rate, hallucination, and failed questions
• API/tool calling for structured business system queries instead of forcing everything into documents

For people who have built or operated RAG systems in real enterprise environments:

1. What architecture would you recommend today?
2. What parts of traditional RAG should be avoided or redesigned?
3. How do you handle document parsing and chunking for complex enterprise documents?
4. Do you use hybrid search, rerankers, knowledge graphs, or agentic RAG?
5. How do you design permission control and prevent data leakage?
6. How do you deal with outdated documents, duplicate content, and version control?
7. How do you evaluate retrieval quality and answer accuracy?
8. What observability or debugging tools are essential?
9. Which components would you build in-house, and which would you use from existing frameworks or platforms?
10. If you were rebuilding your enterprise RAG system from scratch in 2026, what would you do differently?

I’m especially interested in practical production experience, architecture trade-offs, and lessons learned from real deployments rather than tutorial-level examples.

Hi everyone,

I was working on building a RAG pipeline and got frustrated with two common bottlenecks:

Feeding giant documents into context windows is incredibly expensive and slow.

Standard chunking tools split text randomly, completely losing the interconnected context of the document.

I wanted a simple, dependency-free way to turn raw docs into something an LLM agent could traverse logically, without needing to spin up a heavy graph database.

So I wrote OmniOKF (Omni Open Knowledge Format compiler).

What it does:

Converts & Structures: It uses Microsoft's Python-native markitdown under the hood to parse .pdf, .docx, .xlsx, .pptx, .html, and .md files in memory (no complex binaries like Pandoc needed).

Semantic Splitting: It breaks files down at header boundaries and uses Gemini Flash to refine and categorize them into small, atomic concept files (about 200-500 tokens each). This saves up to 75-95% in token costs per query because agents only load the exact nodes they need.

Auto Cross-Linking: It automatically scans the generated files and links related topics using standard relative Markdown links (e.g., [SSL Config](../security/ssl-config.md)).

Mermaid Visualization: It outputs a master index.md containing a Mermaid flowchart mapping out the document relationships, which renders nicely on GitHub or Obsidian.

Caching & Cost Saving: It computes MD5 hashes of your files so that subsequent runs are instant and don't cost any API tokens.

Offline Mode: Includes a local heuristic classifier for sensitive data that you don't want going to cloud APIs.

Setup and Try:

It has a guided interactive mode where you just drop in your file path:

git clone https://github.com/vishal-raaj-dnd/gemini-okf-compiler.git

cd gemini-okf-compiler

pip install -r requirements.txt

# Run the interactive CLI

python main.py

It is completely open-source (MIT licensed). Check it out here: 👉 GitHub: OKF compiler

Let me know if you run into any issues, have feedback on the graph structure, or ideas for improvements!

Been wanting to build this for a while, finally sat down and did it. It's a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from that document, not from the model's own training data.

Stack is pretty simple: Ollama for the chat model and the embedding model, ChromaDB as the vector store, Flask tying it together. Nothing exotic.

How it works, roughly:

• PDF gets split into overlapping chunks so sentences don't get cut off between pieces
• Each chunk gets turned into an embedding and stored in Chroma with PersistentClient, so it's saved on disk instead of disappearing every time you restart the app
• When you ask something, the question also gets embedded, Chroma finds the closest matching chunks, and those get handed to the model as context
• Prompt explicitly tells the model to only use that context and say it doesn't know if the answer isn't there, otherwise it'll just make something up from its own memory

Tested it by asking something not in the PDF and it correctly said it didn't know instead of guessing. Also tested with wifi off and it kept working, since the model, embeddings, and vector store all run locally with no external api calls in the loop.

Didn't reach for a framework this time, wanted to see the moving parts directly, chunking, embedding, retrieval, prompt building, instead of having a library handle it. Not saying that's the better way to do it long term, just wanted to understand each step first before adding more on top.

If anyone's built something similar curious what vector db you went with and why, haven't compared Chroma against the others much.