Knowledge Graph Audit: Six of Mine, Twelve Capabilities, No Complete Column

I had built the same layer six times without noticing. Over the past few years a knowledge graph kept appearing inside otherwise unrelated work. It turned up as a routing viewer for one client, as a layout explorer for another, as a word map I use for teaching, as a live add-in that watches a Revit model while somebody edits it, and as a governance layer over a personal vault of notes. Each one was built for its own reason on its own schedule, and not one of them borrowed a line of code from the others.

Last week I stopped and opened all of them at the same time. The trigger was a complaint I had about my own tooling, and the complaint had three parts: none of these tools handled bulk, none of them offered more than one way of looking at the same data, and none of them showed the process by which an answer was reached. I wanted to know whether the fix was a new tool or something I already owned without having collected it. This post is the audit, and the short version is that the specification had already been written six times over.

Twelve capabilities, six tools, no complete column

I scored six implementations against twelve capabilities. A filled circle means the capability is present and I hold a measurement for it. A triangle means partial or single-purpose. A dash means absent. Three of the tools were built under client contracts, so they appear as A, B and C, and their figures are given as ranges rather than exact counts.

Capability	A	B	C	KMAP	Add-in	Vault
Live update, model to graph	—	△	—	—	●	—
3D geometry linked to nodes	△	△	—	—	●	—
Deterministic query engine	●	●	—	△	●	—
LLM connected, choosing only	●	△	—	●	●	—
Ground check on every number	●	△	—	●	●	—
Axis views, position encodes meaning	—	—	—	●	△	—
Two-subject comparison	—	—	—	●	—	—
Process made visible, trace and Q&A	●	●	—	△	△	—
Approval and provenance	●	—	—	△	—	●
Large scale, over 10k nodes	—	—	—	—	△	△
Written data contract	△	△	△	△	●	△
Packaging and release gate	●	●	●	△	△	—
filled, out of 12	6	3	1	4	6	1

● present and measured · △ partial or single-purpose · — absent. A is a routing viewer, B a generative-design-to-Revit add-in and C a layout explorer, all three under client contracts. KMAP is my word map, Add-in is the live Revit graph and Vault is a personal store of 1,639 notes.

Here is the same set of columns as pictures, so that the table has something to point at. Five of the six draw a graph, only the layout explorer has none at all, and no two of the five look remotely alike.

Two results fall out of the table on sight. Every one of the twelve capabilities exists somewhere in the set, so nothing on my wish list was unbuilt research waiting for a breakthrough. At the same time no column holds more than six of the twelve, and the two strongest columns belong to a client tool and a personal add-in that share no code whatsoever. A core assembled from these six would not be an invention. It would be a harvest.

Six principles I arrived at more than once

The more useful finding sits underneath the circles. When I lined the design decisions up side by side, six of them turned out to be identical across implementations written months apart and never compared against each other. Independent arrival is a stronger signal than a design document, because nobody involved was trying to be consistent.

Principle	What it rules out
Engine computes, model chooses	Arithmetic done inside a language model
Ground check	Any sentence holding a number the tables do not hold
Graceful degradation	A tool that stops working when the model is off
Join by identifier	Coordinates and geometry inside the prompt
Aggregate nodes	Drawing every individual element at once
Candidate before approved	Unreviewed vocabulary treated as evidence

The sixth one is the most embarrassing. I built a two stage vocabulary in a client project, where terms sit in a candidate folder until a person promotes them, and then months later I built the same thing again over my notes with different words for the same idea. Neither implementation knew about the other.

Do not let the model calculate, let it choose

The rule carrying the most weight is also the narrowest one. A small local model, in my case an eight gigabyte one running on the same machine as Revit, is never asked to compute anything. It receives a numbered list of skills the engine knows how to execute, and it answers with a number. Every figure in the final sentence comes from code, and the model contributes phrasing and classification only.

The test that convinced me was not one of the six domain questions that passed. It was the seventh question, which asked how many bananas were in the model. The answer came back as a row of nulls rather than as a plausible number, because no skill returns bananas and the engine had nothing to hand over. A system willing to invent a banana count will also invent a duct length, and the only difference between those two failures is that one of them is embarrassing enough to catch.

Measurement	Result
Choosing from a list against free generation	about 2.5× faster, zero schema deviations
First response, before and after prewarming	10.1 s → 2.8 s
Room area total against an independent recount	760.7 m², exact match
Six domain questions	all passed
Seventh question, asking for bananas	all nulls, nothing invented

Position has to mean something

The word map taught me the visual half of the lesson, and it taught me by failing. In a review session with eighteen people, all eighteen touched the map and not one of them read a value off it. My first assumption was that the graphics needed work. The actual cause was that position carried no information at all, because a force layout drops each node wherever the physics happens to settle, and readers correctly learn that location is noise.

The repair was to make the axes carry meaning. In the first view vertical position encodes abstraction, so the root sits at the top and individual terms hang below their cluster.

In the second the terms group into horizontal bands by kind and the connecting lines are suppressed, so the vocabulary itself dominates instead of the web.

The third splits the canvas into three zones so that two subjects can be compared directly, with what belongs to each one on the outside and what they share in the middle.

Same data, four layouts, and only the last three are readable. I wrote that map up separately in KMAP: a BIM knowledge map in two languages, and the tool is public at github.com/onehojoe/kmap if you would rather read the code than the argument.

The 84 second bug I first diagnosed by guessing

One export was taking 92 seconds and I was certain I knew why. My explanation involved geometry extraction, which was plausible, expensive looking and entirely wrong. When I finally attached timers instead of opinions, 84,225 milliseconds of those 92 seconds turned out to be sitting somewhere I had never considered.

The add-in was comparing document wrappers for equality in order to decide whether an element belonged to the open model. In the Revit API that comparison can return false even when both wrappers point at the same document, so the check failed for every element and each one triggered a fresh view query.

Export	Guessed cause	Measured cause
92 s → 6.3 s a factor of 14.6	Geometry extraction	Document wrapper equality failing, 84,225 ms of per-element view queries. No geometry code was touched.

The lesson generalises well past this one API. A performance guess is not evidence, and the cost of measuring is almost always lower than the cost of rewriting the wrong component. I now treat any optimisation started before instrumentation as a rewrite of something that was probably fine to begin with.

What none of the six can do

The empty row is the one worth reading. Large scale holds no filled circle anywhere, and that absence has a history. An early version of one explorer tried to draw eighteen thousand nodes at once and simply fell over, which is where the aggregate node rule came from. Nodes in the later tools stand for levels, disciplines, zones and equipment types, and individual items surface only while a query is being traced.

The live add-in loads 29,461 nodes from a public sample model in 2.7 seconds, which sounds like the problem is solved until you notice that loading is not interaction. I hold no frame rate curve for any of the six, and every one of them draws on a two dimensional canvas. Measuring that curve is the first task, before any decision about rendering technology, because the 84 second bug taught me what happens when I skip that order.

Gap	What already exists	What is missing
Bulk	Aggregate nodes, 2.7 s load of 29,461 nodes	Any interactive frame rate measurement at all
View variety	Four working axis views in the word map	Views are hardcoded, so no other tool can reuse them
Process visibility	Question, skill and answer are nodes; the trace lights the path taken	Provenance is scattered and is nowhere a first class field

The contract had already been written

The clearest evidence that this is a harvest rather than a project sits in a source file I wrote in August and then forgot about. The Revit add-in carries an explicit graph contract: a node type with identity, label, kind, bounding box, properties, numeric facts and mesh, an edge type with source, target and relation, and a delta type carrying additions, edges, removals and statistics. Above it sits a comment I left for myself, telling me not to change the contract because the data has to keep flowing when the viewer is eventually swapped out. The first step of any shared core is therefore not a blank sheet. It is the promotion of that file.

The cost of having waited is visible in the port numbers. Five copies of what is essentially the same adapter server run on five different ports across these projects, and four separate graph renderers exist because each tool grew its own. The vault shows the same pattern in governance rather than in code. Out of 1,639 notes only 223 carry the canonical layer that makes them usable as evidence, and the built in graph view cannot tell those 223 apart from the rest, so the picture it draws is decoration.

What I have not verified

No interactive frame rate has been measured for any of the six tools, so every statement about scale in this post concerns load time rather than rendering. The speed figures come from one machine and one model, and I would expect them to move on different hardware. The audit establishes that the twelve capabilities exist somewhere across the set, and it does not establish that merging them costs less than maintaining six tools separately. That question needs the frame rate curve and a real second consumer before it can be answered honestly.

What the table does settle is narrower and still worth having. The specification is not open. It was written six times, in six places, by somebody who kept forgetting he had written it before.