Jev in Depth: Can It Reshape Agent Search?

Over the past few days, Jev has been all over my social feeds. It does something unusual: give it context and tell it what to judge, and it returns a choice, a score, or a true-or-false decision. Skipping long-form generation makes decisions faster, with lower token prices as well.

The excitement is easy to understand: millisecond-scale responses, and output that costs almost nothing.

An illustrative replay based on our experiment, not live API calls.

My first thought was search. Online search is sensitive to latency, and search systems now make more model calls than ever.

Is this document useful? Which result should come first? After a search round, should the agent keep looking? Every one of these is a decision. Making them faster opens up more room in the design of a search system.

How much can Jev change the way we use models in search? I wanted to answer that with real tests.

Our team works on search, and we already had several open-source projects with datasets and baselines. So I integrated Jev into three of them for an in-depth, quantitative evaluation.

Agentic Search: Matching Retrieval Quality with Faster Decisions

In 2025, OpenAI's Deep Research brought autonomous search into the spotlight. An agent searches, checks whether it has enough information, and keeps looking until it can answer. This is agentic search, now a standard pattern for search agents.

Suppose the question is: In which city was the director of a particular film born?

The first round identifies the director. The agent still needs to search.

Once it finds the birthplace, it can stop.

Every round comes back to the same question: Do we have enough information, or do we need another search?

Previously, this meant calling a large model each round and waiting, or training a specialized small model and taking on the work of collecting data, training, and maintenance.

This was the first place I wanted to try Jev. It looked like a natural fit.

Our open-source project DeepSearcher follows this pattern: an agent repeatedly searches private knowledge, gathers evidence, and answers complex questions.

We reused its evaluation data, taking 100 multi-hop questions and asking Jev and our DeepSeek V4 Flash service to inspect the same search histories and decide whether to continue.

The result: matching evidence recall, decisions about four times faster, and estimated decision cost reduced to about one-seventh.

Both reached 93.25% Recall@5, with nearly identical average search rounds. On this test, replacing the decision model preserved retrieval quality.

Decision response time fell from 2.23 seconds to 0.55 seconds, roughly a fourfold improvement.

For this task, Jev is ready to do the job. It can take over the search-stopping decision from a general-purpose model, with lower latency and cost.

Still, this is a binary decision. It did not seem to stretch Jev very far. The next experiments were more demanding.

Memory Reranking: A Specialized Reranker Still Leads

We had previously tested Jev on public retrieval datasets, where it performed close to dedicated rerankers.

Would that hold when the material moved away from general-purpose retrieval?

This time, we used agent memory retrieval to explore an out-of-domain setting: Can Jev fully replace a state-of-the-art dedicated reranker?

Our open-source project MemSearch provides persistent memory for coding agents such as Claude Code and Codex. Development conversations and lessons are stored in Markdown, so a new session can recover earlier context. The question here is whether Jev can put the memories that actually help with the current coding task first.

For example, you ask your coding agent: Last time this project's integration tests could not connect to the database, how did we fix it?

The retrieved Markdown memories include test commands, database configuration, and notes from the previous debugging session.

The passage that should come first is the one recording the cause and the fix that worked.

Mentioning tests and databases does not make every passage useful for this problem. Reranking needs to put the right development memory first, so the agent does not repeat a debugging detour it has already taken.

We tested the existing 2,172 questions in both English and Chinese. BGE-M3 dense embeddings provided the original candidates, and the dedicated Voyage rerank-3 model served as the reranking comparison.

Here, Jev could not fully replace a leading specialized reranker.

Jev improved Recall@5 from 74.71% to 79.41%, but the dedicated model reached 81.87% and was also better at moving the correct result toward the top. The gap appeared in both languages. Jev had no cost advantage in this test either.

I suspect two factors:

This is a domain shift. Jev may have trained on public retrieval datasets, but that would not guarantee the same performance on newer, specialized distributions. We do not know its training data; this is a hypothesis.
Some reranking questions require reasoning through implicit steps. Jev may not yet handle these cases as well. The next experiment makes this limitation more visible.

Multi-Hop Graph Retrieval: Useful Relationship Reranking, but a Reasoning Gap

The third project was Vector Graph RAG, our open-source approach combining vector retrieval with entity relationships. It follows connections to find answers that no single document contains.

Suppose the question is: Where was the author of Book A born?

Search retrieves several candidate relationships: Book A was published by Publisher C, Author B won an award, Book A was written by B, and B was born in City D.

All are related to the book or author, but they are not equally useful for answering the question.

Relationship reranking puts the useful connections first. In this simplified example:

Original candidate order	Illustrative reranked order
Book A → Publisher C	Author B → born in City D
Author B → won an award	Book A → Author B
Book A → Author B	Book A → Publisher C
Author B → born in City D	Author B → won an award

The birthplace relationship supplies the answer directly. The book-to-author relationship connects the question to the right person. Both belong near the top. Publication and award relationships can move down or be filtered out.

The system then retrieves the source passages associated with those relationships, carrying their order into the evidence presented downstream. This relationship reranking and filtering step is what we asked Jev to take over from a generative model.

Relationship filtering had been one of the slower steps: many candidates, long inputs, and a wait for generated output. That made it an appealing target for Jev.

We compared it with the existing approaches, ultimately running 500 questions each from MuSiQue and HotpotQA.

Jev uses our 500-question samples; historical baselines use their published 1,000-question results. This is a comparison across those evaluation scopes, not a paired test.

Jev beat GPT-4o-mini on both datasets, but fell short of GPT-5-mini.

On HotpotQA, it trailed GPT-5-mini by just 1 percentage point. On the harder MuSiQue dataset, the gap widened to 4.13 points. HippoRAG 2's historical results were also higher.

My takeaway: Jev can handle relationship filtering, but I would still choose a stronger generative model for difficult multi-hop cases. I would also consider moving from Vector Graph RAG to an agentic search approach with Jev.

These are complicated reranking problems. The candidate set can be large, with intertwined relationships. Getting the order right in one pass is genuinely difficult.

One plausible explanation is the benefit of explicit intermediate reasoning. A generative model can write down intermediate steps and attend to them later, much like using a scratchpad. Jev's decision interface does not produce that kind of reasoning trace. That may help explain the gap on complex cases, though this experiment does not prove the architectural cause. Its strengths look closer to fast, direct judgments.

On estimated cost, Jev came to about $3.15 per 1,000 questions: close to GPT-4o-mini, and below GPT-5-mini's $6–9. An API-based HippoRAG 2 setup could be cheaper still.

These are cost and latency estimates, not a controlled speed benchmark across providers.

Jev's appeal here is the shorter wait for a decision. The stronger alternatives all involve generative-model calls. If response speed matters enough to accept some retrieval-quality loss, Jev remains a useful option.

Where This Could Lead

I see a strong role for models like Jev in search: making frequent semantic decisions with short outputs faster. Beyond the tasks we tested, data curation, quality evaluation, semantic caching, query routing, and reranking with business preferences are all worth exploring.

Complex reasoning still exposes a gap, but the agentic search result shows that some steps can already be replaced. Search decisions that used to be too slow or too expensive deserve another look. More affordable semantic judgments could also reduce our reliance on rough heuristics.

Looking beyond search, Jev's generality is just as appealing. Avoiding large-model latency used to mean training and maintaining separate small models for different tasks. Now, decisions about tools, skills, or browser actions may be handled by one model, adapted through context and criteria. Community browser and game demos are already exploring this direction. One general decision model for many tasks could simplify the architecture itself. That is a direction I want to keep watching.