Benchmark · November 6, 2026
OpenEZ and Graphify, measured
OpenEZ leads most code retrieval suites on three codebases, and answers documentation questions without an LLM. It is not a clean sweep: on the one question set never tuned against, Graphify has the better recall, and both tools miss most questions phrased differently from the code.
How to read these numbers
Across the nine code suites, OpenEZ leads Recall@5 in 6, Graphify in 1, and 2 tie. OpenEZ leads MRR in 8 and 1 ties. Two of the three codebases are optimistic for OpenEZ, because its ranking was improved using their questions. The fairest row is the desktop app's concept suite: Recall@5 .40 for OpenEZ and .53 for Graphify, a gap of two questions out of 15, and MRR .37 and .27.
Retrieval quality
Each cell is Recall@5 / MRR, with no embeddings. Recall@5: is an expected file in the top five? MRR: how high is the first one? Both tools get the same questions and answers, and results count per file. Highlighted cells lead on both metrics.
| Questions | OpenEZ | Graphify |
|---|---|---|
| Concept (15) | .40 / .37 | .53 / .27 |
| Identifiers (10) | 1.00 / 1.00 | 1.00 / 1.00 |
| Relations (8) | 1.00 / .94 | .50 / .16 |
| Documentation (7) | .86 / .78 | N/A |
| Questions | OpenEZ | Graphify |
|---|---|---|
| Concept (11) | .91 / .71 | .45 / .31 |
| Identifiers (10) | 1.00 / 1.00 | .80 / .80 |
| Relations (4) | .75 / .75 | .50 / .28 |
| Documentation (6) | .50 / .50 | N/A |
| Questions | OpenEZ | Graphify |
|---|---|---|
| Concept (9) | .89 / .69 | .78 / .37 |
| Identifiers (9) | 1.00 / 1.00 | .89 / .89 |
| Relations (4) | .75 / .75 | .75 / .25 |
| Documentation (5) | .80 / .80 | N/A |
N/A: Graphify 0.9.103 ran with --code-only, which indexes no documentation.
Speed and size
Query speed is comparable. OpenEZ indexes faster but excludes its call graph, built on the first graph query, and it uses more disk on the larger codebases. Graphify's times include its graph and community detection.
| Codebase | OpenEZ | Graphify |
|---|---|---|
| Desktop app | 7 / 13 ms | 6 / 27 ms |
| Python service | 66 / 122 ms | 41 / 486 ms |
| Next.js frontend | 11 / 17 ms | 11 / 51 ms |
| Codebase | OpenEZ | Graphify |
|---|---|---|
| Desktop app | 0.5 s | 3.9 s |
| Python service | 15.8 s · 262 MB | 81.2 s · 121 MB |
| Next.js frontend | 2.9 s · 31 MB | 8.2 s · 7.4 MB |
Through its CLI, Graphify costs about 290 ms to 2.2 s per query, because each call starts Python and reloads the graph.
What the numbers leave out
The tools return different things. OpenEZ returns code an agent can use directly. Graphify returns graph nodes and edges, and the agent then reads the files. Graphify also covers more languages and adds community clustering, reports, and LLM-based extraction from documents, PDFs, and images.
Limits of this benchmark
- OpenEZ's authors wrote the questions. The answers were checked against the source with grep, but this is not an independent evaluation.
- Two of the three codebases and all question sets are private, so these runs cannot be reproduced publicly.
- The desktop app's identifier and relation suites exposed three OpenEZ defects that were then fixed, so those rows are after-the-fix numbers.
- Suites are small: one question moves a score by .07 to .25.
- Graphify ran with --code-only and no LLM. Its optional LLM pass, community clustering, and wider language coverage were not measured.
Full method in BENCHMARK.md. Back to the overview