AIOHM

Appearance

Feature Idea: Evaluate LlamaIndex For The RAG Pipeline ​

Idea ​

Consider replacing or wrapping the current custom RAG ingestion/retrieval pipeline with LlamaIndex.

The current app is growing its own source registration, chunking, embedding, retrieval, entity extraction, graph attribution, retry, and cleanup behavior. LlamaIndex may give us a more standard abstraction for document ingestion, node parsing, vector indexing, retrieval, and future graph-aware workflows.

This remains a feature idea, not a bug fix. It is related to the open source replacement and cleanup questions, but it should be evaluated separately from the immediate review fixes.

Why It Might Help ​

  • Reduce custom code around chunking, document/node identity, and retrieval orchestration.
  • Make source replacement, re-indexing, and cleanup easier to reason about through explicit document/node abstractions.
  • Support richer retrieval strategies, such as hybrid retrieval, reranking, metadata filters, and graph-aware retrieval.
  • Provide a clearer boundary between app-level knowledge source management and indexing implementation details.
  • Potentially make Ollama/local-model support easier to standardize across embeddings and LLM calls.

Questions To Answer ​

  • Should LlamaIndex own chunk/node identity, or should the app continue to own KnowledgeSource, VectorEntry, and graph attribution tables?
  • Can LlamaIndex cleanly support the current tenant isolation model?
  • Should we use LlamaIndex only for ingestion/chunking, only for retrieval, or for the full RAG pipeline?
  • How would LlamaIndex integrate with Laravel queues and the current sequential indexing chain?
  • How would document deletion, replacement, failed indexing, and manual re-index cleanup work?
  • Can it preserve the current browser-facing Vector Entries and Retrieval QA workflows?

Proposed First Spike ​

Build a narrow proof of concept outside the production path:

  • Use one tiny Markdown document.
  • Run LlamaIndex chunking/node parsing.
  • Generate embeddings through local Ollama.
  • Store enough metadata to map nodes back to a KnowledgeSource.
  • Run a retrieval query and compare the result shape with the current RagEngine output.
  • Document what tables would remain app-owned versus LlamaIndex-owned.

Proposed Longer-Term Direction ​

If the spike is promising, introduce LlamaIndex behind an internal adapter rather than replacing the whole RAG system in one step.

The adapter should expose app-level operations such as:

  • Register or replace a knowledge source.
  • Index a source version.
  • Retrieve relevant chunks for a query.
  • Return source attribution for retrieved chunks.
  • Delete or supersede source-scoped indexing artifacts.

This keeps the Laravel app in control of tenants, permissions, UI state, jobs, and cleanup policy while allowing LlamaIndex to handle lower-level RAG mechanics.