Benchmark · November 6, 2026

OpenEZ and Graphify, measured

OpenEZ leads most code retrieval suites on three codebases, and answers documentation questions without an LLM. It is not a clean sweep: on the one question set never tuned against, Graphify has the better recall, and both tools miss most questions phrased differently from the code.

How to read these numbers

Across the nine code suites, OpenEZ leads Recall@5 in 6, Graphify in 1, and 2 tie. OpenEZ leads MRR in 8 and 1 ties. Two of the three codebases are optimistic for OpenEZ, because its ranking was improved using their questions. The fairest row is the desktop app's concept suite: Recall@5 .40 for OpenEZ and .53 for Graphify, a gap of two questions out of 15, and MRR .37 and .27.

Retrieval quality

Each cell is Recall@5 / MRR, with no embeddings. Recall@5: is an expected file in the top five? MRR: how high is the first one? Both tools get the same questions and answers, and results count per file. Highlighted cells lead on both metrics.

Rust + TypeScript desktop app · 187 filesQuestions written before either tool ran. The concept row was never tuned against.
QuestionsOpenEZGraphify
Concept (15).40 / .37.53 / .27
Identifiers (10)1.00 / 1.001.00 / 1.00
Relations (8)1.00 / .94.50 / .16
Documentation (7).86 / .78N/A
Python FastAPI service · 1,876 filesOptimistic for OpenEZ: its ranking was improved using these questions.
QuestionsOpenEZGraphify
Concept (11).91 / .71.45 / .31
Identifiers (10)1.00 / 1.00.80 / .80
Relations (4).75 / .75.50 / .28
Documentation (6).50 / .50N/A
TypeScript Next.js frontend · 933 filesOptimistic for OpenEZ: its ranking was improved using these questions.
QuestionsOpenEZGraphify
Concept (9).89 / .69.78 / .37
Identifiers (9)1.00 / 1.00.89 / .89
Relations (4).75 / .75.75 / .25
Documentation (5).80 / .80N/A

N/A: Graphify 0.9.103 ran with --code-only, which indexes no documentation.

Speed and size

Query speed is comparable. OpenEZ indexes faster but excludes its call graph, built on the first graph query, and it uses more disk on the larger codebases. Graphify's times include its graph and community detection.

Query, median / p95 (warm, in-process)
CodebaseOpenEZGraphify
Desktop app7 / 13 ms6 / 27 ms
Python service66 / 122 ms41 / 486 ms
Next.js frontend11 / 17 ms11 / 51 ms
Index time · disk
CodebaseOpenEZGraphify
Desktop app0.5 s3.9 s
Python service15.8 s · 262 MB81.2 s · 121 MB
Next.js frontend2.9 s · 31 MB8.2 s · 7.4 MB

Through its CLI, Graphify costs about 290 ms to 2.2 s per query, because each call starts Python and reloads the graph.

What the numbers leave out

The tools return different things. OpenEZ returns code an agent can use directly. Graphify returns graph nodes and edges, and the agent then reads the files. Graphify also covers more languages and adds community clustering, reports, and LLM-based extraction from documents, PDFs, and images.

Limits of this benchmark

  • OpenEZ's authors wrote the questions. The answers were checked against the source with grep, but this is not an independent evaluation.
  • Two of the three codebases and all question sets are private, so these runs cannot be reproduced publicly.
  • The desktop app's identifier and relation suites exposed three OpenEZ defects that were then fixed, so those rows are after-the-fix numbers.
  • Suites are small: one question moves a score by .07 to .25.
  • Graphify ran with --code-only and no LLM. Its optional LLM pass, community clustering, and wider language coverage were not measured.

Full method in BENCHMARK.md. Back to the overview