Agentic Keyword Search vs. Vector Databases: What the 2026 Data Actually Shows

In early 2026, a debate that had been simmering in agent-engineering Slack channels went mainstream: do AI agents need vector databases at all? The trigger was an Amazon Science paper presented at AAAI 2026, which measured agentic keyword search — an agent that iteratively calls grep-style and file-search tools instead of querying an embedding index — at 94.5% of RAG faithfulness, using zero vector store. That's not a rounding error. It's a real result, and it forced a real reckoning about when embeddings are doing load-bearing work and when they're just overhead. This post walks through the finding, where it holds, where it breaks, and what the practical 2026 architecture looks like.

The Amazon Science finding: 94.5% faithfulness, zero vector store

The core result, as reported here, is that an agent given iterative search tools — the equivalent of running grep, narrowing a file list, and re-reading a smaller candidate set — can match 94.5% of the faithfulness score of a traditional RAG pipeline built on vector retrieval, without building or querying an embedding index at all. Faithfulness here means the agent's answers stay grounded in the source material it retrieved, rather than hallucinating around gaps.

Why does this work? Vector retrieval does one thing well: it collapses a semantic question into a single nearest-neighbor lookup. But it also does one thing poorly: it commits to a fixed top-k result set before the agent has had a chance to look around. An agent with tool access doesn't have that constraint. It can search, read, notice it grabbed the wrong file, search again, and narrow down — the same way a competent engineer greps a codebase instead of trusting a single fuzzy match. When the corpus has exploitable structure (file names, function signatures, headers, exact keywords), iterative keyword search can recover almost everything a vector index would have surfaced, and it can correct its own mistakes mid-task in a way a single embedding lookup cannot.

Why coding agents dropped embeddings first

The clearest evidence isn't academic — it's shipped product. Several coding-agent tools, including Windsurf, Cline, Devin, and Sourcegraph Amp, have moved away from vector embeddings for parts of their code-search workflow, favoring tool-driven, agentic search instead, per the same reporting.

Codebases are close to a best case for this approach. They have exact-match anchors everywhere — identifiers, imports, file paths, type names — that keyword and structural search exploit directly. They're also usually small-to-medium in size relative to an agent's context and tool budget, which means the cost of a few extra grep-and-read round trips is trivial next to the cost of maintaining a fresh embedding index that has to be re-synced on every commit. A vector index over a codebase is solving a semantic-similarity problem the codebase mostly doesn't have; the real problem is precise navigation, and grep-style tools were built for exactly that.

Where vector search still wins

None of this means vector databases are obsolete — and the honest version of this debate has to say so plainly. The 94.5% figure is a floor for a specific class of problem: bounded corpora with exploitable lexical or structural signal, where an agent can afford to iterate. It does not generalize to every retrieval workload, and three cases in particular still favor a vector layer:

• Semantic similarity across large, unstructured corpora. When the relevant match doesn't share vocabulary with the query — a support ticket phrased nothing like the KB article that resolves it — keyword and grep-style tools have nothing to grab onto. Iterating doesn't help if there's no lexical thread to pull. This is the case vector embeddings were built for, and it's still common in support, legal, and research corpora.
• Low-latency, single-shot retrieval. Agentic search trades latency and token spend for iteration. Every extra search-read-refine loop costs tool calls, round trips, and tokens. A well-tuned ANN index returns a ranked result in under a millisecond — Feather DB, for example, holds p50 ANN latency of 0.19ms at 500K vectors with 97.2% recall@10. For a single-shot answer at that speed, no amount of grep iteration competes.
• Long-term, cross-session memory. This is a different problem from single-query document retrieval, and it's worth separating cleanly. An agent that needs to recall something from a conversation 40 sessions ago can't re-scan the full history per query the way it can re-grep a small codebase — the corpus of past interactions grows unboundedly, and iterative search cost grows with it. That's why industry analysis on agent memory expects contextual, agentic long-context memory to surpass RAG in usage specifically for agentic workloads — but that memory layer is still, in most implementations, backed by compact vector recall rather than full re-scanning. On LongMemEval_S, Feather DB paired with GPT-4o scores 0.693 versus 0.640 for a full-context GPT-4o baseline that re-reads everything — evidence that a memory layer built for selective recall beats brute-force re-scanning as history grows, even against a model with no retrieval bottleneck at all.

A decision matrix: when to use what

The 2026 synthesis: a small vector layer, plus lots of tools

The framing that loses in 2026 is "pure agentic search" versus "pure vector search" as competing architectures. The framing that wins, per multiple industry analyses and echoed in broader 2026 enterprise-AI forecasting, is a small vector layer plus a lot of tools. Vector search didn't die in 2026 — it got demoted from default to fallback. Agents reach for grep-style, structural, and API-based tools first, because they're cheap, precise, and self-correcting on structured data. The vector index stays in the loop for the narrower set of jobs it's actually good at: fast single-shot semantic lookups over large unstructured corpora, and durable cross-session memory that can't be re-scanned on every query. That's a smaller footprint than the "vector database as the retrieval layer for everything" architecture common in 2023-2024 RAG stacks — and it's a more honest one, because it stops asking embeddings to solve problems that a grep loop solves better and cheaper.

For teams building agents today, the practical takeaway is architectural, not ideological: give the agent tools first, and reserve the vector layer for the retrieval problems where iteration genuinely can't substitute — semantic drift between query and source, latency budgets that can't absorb multiple round trips, and memory that spans more history than any context window or search budget can re-read.

FAQ

Does the 94.5% faithfulness result mean vector databases are unnecessary?

No — it means they're unnecessary for the specific class of problem the paper tested: bounded, structured corpora where an agent can iterate cheaply. It doesn't generalize to large unstructured semantic corpora or to latency-critical single-shot retrieval, where a vector index still outperforms iterative search.

Why did coding agents drop embeddings before other domains?

Code is close to a best case for keyword and structural search: exact identifiers, predictable file structure, and corpora small enough that iterative search stays cheap. Most other domains — support content, research literature, long conversation history — don't share those properties.

Is agentic memory the same thing as RAG?

No. RAG typically answers a single query against a document corpus. Agentic memory recalls information across sessions over an unbounded and growing history, which is why re-scanning per query becomes impractical as history accumulates — that's a case where compact vector recall keeps outperforming brute-force context stuffing, as seen in LongMemEval_S results.

What does the "small vector layer + lots of tools" architecture actually look like in practice?

An agent equipped with file-search, API, and structural tools as its first line of retrieval, with a lightweight vector index reserved for semantic lookups and long-term memory — not the default retrieval path for every query.

If you're deciding where a vector layer earns its place in an agent stack, Feather DB is free and MIT-licensed to try locally with pip install feather-db — no server, no infrastructure commitment, just a single .feather file to test against your own workload.