Metadata Filtering Strategies for RAG Systems

Every RAG conversation eventually turns into an argument about embedding models. Which one’s best, how big should chunks be, cosine vs dot product. Fine, those matter. But there’s a decision that quietly breaks more RAG systems in production than any of that, and almost nobody writes about it: how do you apply your metadata filters?

If your retrieval needs to respect tenant boundaries, price ranges, stock status, document permissions, or basically any “and only show me results where X” condition — and most real systems do — you have three ways to combine that filter with your vector search. Pick the wrong one and your system will look fine in a demo and then quietly return garbage, or nothing, once it hits real data and real constraints.

The three approaches are pre-filter, post-filter, and on-the-fly (integrated) filtering. Here’s how each one actually works, where it breaks, and how to build it with pgvector — including a couple of gotchas that trip people up even when they think they’re doing it right.

Pre-filter: narrow first, then search

The idea: apply your metadata constraints before the vector similarity search runs, so the ANN search only ever looks at a subset of the data.

How it works, step by step:

• Metadata scan — a normal database index finds every row matching your condition (tenant ID, category, price cap, whatever)
• Vector isolation — the embeddings for that matching subset get pulled out
• Semantic search — the ANN search (usually HNSW) runs only on that isolated subset

Why people like it:

• 100% accuracy on your constraints — nothing that fails the filter can ever leak into the results
• It’s the only real option for hard isolation requirements like multi-tenancy, where a customer’s data must never even be considered for another customer’s query

Where it bites you:

• If the filter still matches millions of rows, you haven’t actually narrowed much, and the “isolated” vector search is nearly as slow as searching everything
• If a naive implementation tries to restrict the same sharedHNSW graph to just the allowed nodes (instead of truly isolating them), you get a fragmented graph — the traversal runs into dead ends because the neighbors it needs to hop through got filtered out. This is a real and well-documented failure mode, and it’s specifically what more advanced “on-the-fly” filtering techniques exist to fix.

Use it when:the filter is highly selective (a small, well-defined slice of your data) and you don’t actually need to search the whole corpus most of the time — tenant isolation is the textbook case.

Pre-filter with pgvector — and the gotcha that catches people

Here’s the trap: just tacking a WHERE clause onto a normal ANN query does not automatically give you true pre-filtering.

# Looks like pre-filtering. Often isn't.
cur.execute(
    """
    SELECT id, content, category, price, in_stock,
           embedding <=> %s::vector AS distance
    FROM items
    WHERE in_stock = true AND price <= %s
    ORDER BY distance
    LIMIT %s
    """,
    (query_embedding, max_price, top_k),
)

If items has an HNSW index and there's no supporting index tuned for this predicate, pgvector's default behavior is to run the approximate index scan first and apply your WHERE clause as it comes off that scan — the exact same recall risk as post-filtering, just hidden inside one SQL statement instead of two steps.

To get genuine pre-filtering out of pgvector, you’ve got two real options:

1. A partial index, when the predicate is a fixed, low-cardinality value known ahead of time (a boolean flag, a tenant ID, a category):

CREATE INDEX items_in_stock_hnsw
    ON items USING hnsw (embedding vector_cosine_ops)
    WHERE (in_stock = true);

This bakes the filter into the index itself, so the ANN search only ever traverses matching rows. It doesn’t help for a runtime-supplied range like price <= $X, though — you’d need a different index per price bucket, which usually isn’t worth it.

2. Iterative index scans(pgvector 0.8.0+), for filters that aren’t a fixed value — like our price cap:

cur.execute("SET hnsw.iterative_scan = relaxed_order;")
cur.execute(
    """
    SELECT id, content, category, price, in_stock,
           embedding <=> %s::vector AS distance
    FROM items
    WHERE in_stock = true AND price <= %s
    ORDER BY distance
    LIMIT %s
    """,
    (query_embedding, max_price, top_k),
)

With this setting on, pgvector automatically scans more of the index and keeps going until it’s actually found top_krows that satisfy the filter, instead of quietly returning fewer. It’s not the same mechanism as a partial index — it’s closer to a smart, self-widening scan — but it solves the same underlying problem: don’t stop scanning until the filter is satisfied.

Post-filter: search everything, filter after

The idea:run the vector search across the entire dataset first, then throw out anything that doesn’t meet your business rules afterward.

Steps:

• Semantic search — ANN search runs against the whole index, no metadata involved yet
• Metadata scan — filter the returned rows using a normal index or just application code

Why people like it:

• No fragmented graph, no filter-aware index to build — you’re just running vector search the normal way
• Predictable, consistent latency, since the ANN search always does the same amount of work
• Nothing special required from your vector index

Where it bites you:

• Recall starvation — if your filter is selective and the top-k vector matches happen to be mostly filtered out, the genuinely relevant documents never even entered the candidate pool
• Empty results — worst case, you filter your top-k down to nothing, even though relevant matching documents exist further down the ranking

Use it when:the filter is broad (most rows pass it) and you’re usually searching close to the full corpus anyway. If 95% of your inventory is in_stock = true, filtering after search barely costs you anything.

Post-filter with pgvector

This is close to what you’d write instinctively — run the plain vector query, then filter in your application code:

def main() -> None:
    openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    query_embedding = embed(openai_client, QUERY)

    conn = psycopg2.connect(os.environ["DATABASE_URL"])
    register_vector(conn)
    try:
        with conn.cursor() as cur:
            # Pure vector search: no metadata WHERE clause, so results may include
            # items that are out of stock or over budget.
            cur.execute(
                """
                SELECT id, content, category, price, in_stock,
                       embedding <=> %s::vector AS distance
                FROM items
                ORDER BY distance
                LIMIT %s
                """,
                (query_embedding, TOP_K),
            )
            rows = cur.fetchall()
    finally:
        conn.close()

    # Post-filter: discard matches that don't meet business rules. The vector
    # search already ran; this only narrows down the results we already fetched.
    filtered = [
        row
        for row in rows
        if row[4] is True and row[3] is not None and row[3] <= MAX_PRICE
    ]

The catch is visible right there in the code: TOP_K has to be big enough that, after filtering, you still have something left to show the user. If MAX_PRICE is aggressive and most of your catalog is over budget, this snippet can quietly hand back an empty list even when plenty of matching products exist. Instrument that — log how many rows survive the filter, and alert when it trends toward zero.

On-the-fly: check the filter while you’re walking the graph

The gold standard, and the reason dedicated vector databases like Qdrant and Weaviate exist. Instead of filtering before or after the ANN search, the filter check happens during graph traversal:

[Start Graph Traversal]
     │
     ├──► Inspect Item A ──► in_stock = true? ──► YES ──► Calculate Vector Distance ──► Keep moving
     │
     └──► Inspect Item B ──► in_stock = true? ──► NO  ──► Skip Vector Math ──► Follow graph links

Why it’s the best of both worlds:

• You get pre-filter’s guarantee (nothing invalid ever makes it into results) and post-filter’s speed, without either failure mode
• You get your requested top-K results back, assuming enough matching documents exist — no recall cliff
• No disconnected-graph problem, because the traversal is filter-aware from the start, not restricted after the fact
• Skipping the distance calculation for nodes that fail the filter saves real CPU

What it costs you:

• Keeping the vector graph and the metadata index aligned in memory takes meaningfully more RAM
• Writes get more expensive — every insert or update has to keep graph edges and metadata payloads in sync

Who actually does this well:Qdrant’s filterable HNSW is the reference implementation — it builds extra graph edges around indexed metadata fields so filtered traversal doesn’t fall apart. Weaviate’s ACORN does something similar with two-hop expansion, and it’s the default filter strategy for new collections. Pinecone’s serverless architecture merges the vector and metadata indexes into a single-stage system rather than treating them separately.

Worth being honest here: pgvector doesn’t do this natively. Its iterative-scan feature (shown above) solves the symptom — not enough filtered results — with a smarter, self-widening post-filter loop, not true single-stage in-graph filtering. If your access-control or tenancy requirements are strict enough that you need this pattern specifically, a purpose-built engine is currently the more mature answer.

Use it for:RBAC and strict multi-tenant systems where filters are the norm, not the exception. A common pattern: store a document’s allowed roles directly in its metadata payload, then pass the requesting user’s roles as the filter on every search. The filtering isn’t an afterthought — it’s baked into every single query.

Picking one

• Tight, stable filter that must never leak (tenancy, permissions) → pre-filter, ideally backed by a partial index or a real filter-aware engine
• Loose, broad filter that most rows pass anyway → post-filter, with monitoring on your post-filter result count
• Strict filters that run on every query, at scale, where correctness and latency both matter → on-the-fly, and that’s usually the point where it’s worth evaluating a dedicated vector database instead of stretching pgvector further

None of this shows up in a demo with fifty rows and no access control. It shows up the first time a real customer runs a real query against real data with real permission boundaries — and that’s exactly the moment it’s most expensive to redesign around.