Vector Databases

In modern AI Engineering, applications cannot rely solely on static model parameters or brute-force keyword matching. They require high-speed semantic retrieval across proprietary knowledge bases. The core mental model of vector databases is:

Connecting this directly to our previous RAG lesson:
Documents → Chunks → Embedding Model → Vector Database → Retrieval → Context Prompt → LLM. The vector database serves as the high-speed infrastructure layer that makes query-time RAG possible at production scale.

Suppose your application has 10 vectors. Finding the closest vector to a query requires calculating 10 dot products. On a modern CPU, this takes less than a microsecond. But what happens as your knowledge base expands?

In an exhaustive scan, the time complexity is O(N · d), where N is the number of stored vectors and d is the vector dimension. A vector database solves this by constructing specialized multi-dimensional indexing structures (such as hierarchical graphs or clustered inverted lists) that reduce candidate evaluations to O(log N) or sub-linear time.

A vector is simply an array of numbers (e.g., [0.024, -0.912, 0.145, ...]). If a database only stored numbers, retrieving the closest vector would yield no actionable information for an AI application. A production vector record consists of five coordinated pillars:

{
  "id": "doc_9481a_sec_4",
  "vector": [0.0142, -0.0521, 0.0891, "...1536 floats total..."],
  "text": "All learners enrolled in individual course modules are eligible for a 100% full refund within 14 calendar days...",
  "metadata": {
    "source_file": "refund_policy_2026.pdf",
    "document_id": "DOC-POL-2026",
    "tenant_id": "enterprise_org_42",
    "department": "Finance & Compliance",
    "access_role": "public_customer",
    "effective_year": 2026,
    "language": "en",
    "chunk_index": 4
  }
}

Every vector database uses its own taxonomy to represent container hierarchies. They are conceptually similar (a container with a fixed vector dimension and distance metric), but their terminology differs:

How does a database determine which vectors are "closest" to a query? It applies a geometric distance function. Three metrics dominate the industry:

In vector retrieval, you have two fundamentally different architectural paths:

The two most popular index algorithms supported natively in production vector databases like pgvector are:

In vector infrastructure engineering, you cannot optimize all variables simultaneously. You must pick the index that aligns with your application's operational SLA:

In real-world applications, users rarely search unconstrained corpora. They ask:
"Find Python functions, but ONLY in documents where language = 'Python' AND category = 'AI Engineering'."

Pure semantic vector search has a notorious blind spot: rare exact strings. When an engineer queries for "ERR_CONNECTION_RESET", an embedding model might map the query near general networking concepts (like "TCP timeouts" or "network sockets"), completely bypassing the exact troubleshooting guide with that specific error code. Hybrid Search solves this by querying two parallel representations:

RRF_Score(doc) = (1 / (60 + Dense_Rank(doc))) + (1 / (60 + Sparse_Rank(doc)))

Where:
• Dense_Rank(doc): Position of the document in the vector similarity candidate list
• Sparse_Rank(doc): Position of the document in the BM25 keyword matching candidate list
• 60: Smoothing constant (k) that prevents extreme outlier ranks from dominating

In production enterprise search, relying on a single retrieval step introduces a painful dilemma: if Top-K is too small (e.g. 3), you miss relevant context; if Top-K is too large (e.g. 50), you flood the LLM with distracting noise. The production solution is Two-Stage Retrieval:

There is no single "best" vector database. The optimal selection depends strictly on whether you are augmenting an existing relational application, deploying a standalone microservice, or operating a multi-tenant enterprise cluster:

In production AI Engineering, PostgreSQL with the official pgvector extension is the most widely deployed vector storage layer because it keeps vectors, relational foreign keys, and audit logs within a single ACID-compliant database.

-- 1. Enable the official pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- 2. Create documents table with vector(1536) and JSONB metadata
CREATE TABLE documents (
    id BIGSERIAL PRIMARY KEY,
    tenant_id VARCHAR(64) NOT NULL,
    content TEXT NOT NULL,
    metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
    embedding VECTOR(1536) NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- 3. Create HNSW index using cosine distance operator class
CREATE INDEX ON documents 
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

-- 4. Execute Top-5 nearest neighbor search with metadata pre-filtering
SELECT 
    id, 
    content, 
    metadata->>'source_file' AS source,
    embedding <=> '[0.014, -0.052, 0.089, ...]'::vector AS cosine_distance
FROM documents
WHERE tenant_id = 'enterprise_org_42'
  AND (metadata->>'effective_year')::int >= 2026
ORDER BY embedding <=> '[0.014, -0.052, 0.089, ...]'::vector ASC
LIMIT 5;

In a multi-tenant enterprise system, User A (Company Alpha) and User B (Company Beta) query the same vector database. A catastrophic security failure occurs if Company Alpha's query vector returns Company Beta's confidential internal documents:

SELECT id, content, metadata FROM documents
-- [VULNERABILITY: Missing tenant_id constraint!]
ORDER BY embedding <=> query_vector ASC LIMIT 5;

Managing vector data over time involves specific lifecycle rules that differ from relational databases:

Test this working simulated vector retrieval engine. Enter search queries to compute real-time cosine similarities and Euclidean distances against our indexed roadmap knowledge base (without invoking an LLM, highlighting the retrieval layer itself).