Design review in Omniverse Kit: change the design, get an AI answer in 0.1 s and GPU physics in 1.7 s

In short

• Omniverse Kit can be more than a viewer for generative-design results. Inside one Kit session I raised a flooded plant step by step. At each step a U-Net surrogate painted the flood on the USD stage in 125 ms, and NVIDIA Warp confirmed it on the GPU in the same session, in 1.7 s once its solver loop was replayed as a CUDA graph. Warp cleared the plant from +0.4 m; the U-Net, trained only on GA layouts, kept false alarms on the raised platform. That is the case for marking it unconfirmed.
• Across 115 what-if layouts it had never seen, the U-Net layer missed 1 of 58 real dangers. An untrained physics proxy shown just as fast missed 9. It raised more false alarms than the proxy (8 against 5), the cheaper kind of mistake while Warp confirms.
• Everything lives in OpenUSD: per-request values, a confirmed flag, 6,810 values read back with 0 mismatches. CUDA graphs cut a Warp run inside Kit from 59 s to 2.0 s.

Synthetic site, assumed unit costs, no client data.

This is the second companion note to the plant-layout case. It uses the same frame as the rest of the series: GD alternatives, a simulation on what each alternative builds, OpenUSD and Omniverse Kit for review, and AI measured against a baseline.

The demo: raise the plant, watch the answer

Plan C is the GA optimum that flooded at the clearwell and the sludge basin. Inside Kit, I raised its platform (every facility and its 3 m apron) by 0.2 m at a time. At each step:

1. Kit updates the finished terrain on the USD stage.
2. The U-Net, running in PyTorch inside Kit's own Python, paints its depth layer. It is marked unconfirmed, and the facilities it thinks are in danger are outlined in red.
3. Warp runs the 2D flood on the GPU in the same session and replaces the layer.

Platform raise	U-Net answer	U-Net: in danger	Warp in Kit	Warp: in danger	Sludge basin apron, U-Net / Warp
+0.0 m	125 ms	clearwell, sludge basin	1.7 s	clearwell, sludge basin	0.77 / 0.68 m
+0.2 m	113 ms	sludge basin	1.7 s	sludge basin	0.60 / 0.48 m
+0.4 m	131 ms	sludge basin	1.7 s	none	0.50 / 0.28 m
+0.6 m	110 ms	sludge basin	1.7 s	none	0.46 / 0.08 m
+0.8 m	132 ms	clearwell, sludge basin	1.7 s	none	0.40 / 0.01 m
+1.0 m	125 ms	clearwell, sludge basin	1.6 s	none	0.35 / 0.00 m

Read back from the USD stage: 180 values, 0 mismatches.

• The instant answer was right while the design looked like its training data: at +0.0 and +0.2 m it named the same facilities as Warp.
• Off its training data it erred on the safe side: from +0.4 m Warp showed no danger, but the U-Net kept the sludge basin red (0.35–0.50 m against Warp's 0.00–0.28 m), and from +0.8 m the clearwell too. No danger was missed.
• The confirmation is now fast enough to keep up: 1.7 s per step inside Kit, against 48 s before CUDA graphs.

The latency here (125 ms median) is higher than in the 115-request test below (99 ms) because each step also repaints the terrain on the stage.

This is a simplified repair: the design tool's real repair raised the inlet level by 0.95 m and regraded the slopes, and costs are not computed here. The point is the loop a reviewer gets: change, instant answer, physics confirmation, all on one stage.

The question behind it

The method post sets a condition: a surrogate is worth using when one evaluation is expensive, or when a person is waiting for the answer in milliseconds. A review in Kit is the second case. A reviewer asks "what if the layout were this one?" Recomputing it took about 48 s inside Kit (49 s in the main post's run) and 4.7 s outside. That gap is now closed with CUDA graphs (59 s to 2.0 s); see the appendix.

Criteria fixed before the run (file hash logged):

• Latency: request → depth layer written to the USD stage → one Kit update. The median must be ≤ 0.5 s and the 95th percentile ≤ 1 s.
• Verdicts before confirmation: count them over 115 requests × 14 facilities. The surrogate must miss at most half as many dangers as the untrained physics proxy, and make fewer wrong verdicts overall. Danger means an apron (the 3 m strip around a facility) at least 0.30 m deep.
• Map agreement: the median overlap (IoU) between the surrogate's flooded area and Warp's must be at least 0.80.
• Confirmation: Warp inside Kit must reproduce the stored reference within 0.001 m.
• USD: every value written to the stage must read back with 0 mismatches.

The file also records what was already known: the offline scores of the same models on the same 115 plans. Three things were not known:

• whether PyTorch would run inside Kit at all,
• how long a painted answer would take,
• whether a retrained model with saved weights would reproduce those scores.

GD: what-if requests, and where the training data came from

The surrogate in this note is a product of the extended GD–AI loop. The GD tool produced 574 plant layouts. Warp flooded each one on its own finished ground on the GPU, so every layout arrived with a label: its max-depth map and the depth at 14 facility aprons. Nobody surveyed or labelled anything by hand. The surrogate was trained on 459 of them. The other 115 (every fifth plan) are the requests. They stand for alternatives that are not in the USD file yet, so nothing has been computed for them.

Simulation: the instant layers and the confirmation

Each request runs three things on the same Kit stage:

1. Physics proxy (no training). It fills the terrain's depressions (Planchon–Darboux, in Warp) and paints the fill depth. It measures how deep each depression is, not how much water flows through.
2. U-Net surrogate. It takes the proxy's terrain features plus the finished ground and predicts the maximum depth map. This is the model from the plant case: 1.9 M parameters, trained for 60 epochs. I retrained it here so the weights could be saved, and it gave the same verdict counts and F1 (0.927) as before.
3. Warp 2D, the same flood kernel as in the main post. It runs inside Kit and replaces the instant layer when it finishes.

Two of the requests, the three layers side by side: request #20, where the U-Net raised false alarms, and request #400, where the proxy missed a danger.

The timings below are from the 115-request test, run before CUDA graphs: Warp then took about 48 s.

• The U-Net layer appears in 99 ms (median); 95 % of requests take under 107 ms, and the slowest took 416 ms. The terrain features take 80 ms, the U-Net 4 ms and writing the USD 5 ms. The remaining 10 ms covers the Kit update and everything else. The proxy shares the 80 ms feature step, so both layers appear at the same moment. The question is which one to show.
• Warp inside Kit took 47–59 s per request (before CUDA graphs) on the 5 requests I confirmed. Four of them matched the stored reference within 0.0006 m. Request #20 missed my 0.001 m tolerance by 0.00003 m. The reference is stored as float16, which rounds by up to 0.00098 m at that depth (2.2 m). The rest is within the 0.0002 m difference measured earlier between Kit and standalone. My tolerance was finer than the reference can hold above 2 m, so I record the confirmation criterion as not met.

AI: how often is the instant answer wrong?

Before Warp confirms	Physics proxy	U-Net
Missed dangers (of 58 real ones)	9	1
False alarms	5	8
Requests with at least one wrong verdict	12 of 115	7 of 115
Requests with a missed danger	8 of 115	1 of 115
Flooded-area IoU with Warp (median, lowest)	not computed	0.983, 0.92

• The proxy errs both ways.
  • In its 9 misses it showed 0.05–0.29 m where Warp had 0.32–0.67 m. Flowing water does not appear in a depression fill: request #400 above is one of these misses.
  • Its 5 false alarms go the other way: depressions filled to 0.45–3.6 m where Warp has 0.2 m or less.
• The U-Net leans towards alarms.
  • It predicted 65 dangers against 58 real ones.
  • Its one miss was 0.32 m by Warp and 0.15 m by the U-Net.
  • Its 8 false alarms were 0.12–0.29 m by Warp and 0.32–0.44 m by the U-Net; three of them fell on request #20 above.
  • In a review, that means a few red facilities that turn grey when Warp finishes, which is the cheaper kind of mistake.
• Against the proxy, the U-Net is the better instant layer. Latency, verdicts and map agreement met their criteria; confirmation missed by 0.00003 m (above). The layer is marked unconfirmed on the stage and replaced by Warp.
• It is not the best model overall. Offline, a gradient-boosting model (GBM) got the facility verdicts right more often: 3 missed and 1 false alarm, 4 wrong in all against the U-Net's 9. GBM predicts facility values, not a map, so it cannot paint the layer. A U-Net map with GBM-coloured facilities is the next test; I have not run it.

The 8 requests that matter most, as maps:

• Which ones: the 7 where the U-Net got a verdict wrong, plus #400, where the proxy missed a danger.
• What the error maps show: the U-Net's error is small away from the channel. The exception is #295, where it over-reads the channel by about 0.2 m.

OpenUSD: what the stage carries

Each request is a prim under /World/Requests. For each of the 14 facilities it stores:

• the surrogate's apron depth and verdict,
• the proxy's apron depth and verdict,
• once confirmed, Warp's apron depth and verdict.

It also stores the request latency and a confirmed flag.

The terrain and the three depth layers (Flood_Proxy, Flood_AI, Flood_Warp) are fixed-topology meshes. Only their points are updated per request, which is why writing to USD takes 5 ms. Read back in Kit: 6,810 values, 0 mismatches.

The confirmed flag is the contract. A reviewer, or a script, can tell from the file alone whether a number on screen has been simulated yet.

Here are 32 of the requests laid out in one stage, each U-Net row above its Warp row:

• Which ones: all 7 with a wrong U-Net verdict and all 8 where the proxy missed a danger, plus others.
• Red pad: a wrong U-Net verdict.
• Bar height: the U-Net's largest apron error in that request.
• Warp rows: the stored reference, from the same kernel I ran inside Kit for the 5 confirmations.

What this adds to the method

• The simulation is the teacher. The instant layer is only as good as the labels behind it, and those came from Warp on what each GD layout builds. Every confirmation in a review can become one more labelled layout.
• AI gets a second job, and only one more. In the plant case its only job was to suggest which layouts to simulate first. In a review session it also fills the wait. Its layer is marked unconfirmed and is always replaced.
• Compare against the cheapest instant answer, not against waiting. Anything beats a 48 s wait. With CUDA graphs the wait on this 2 m grid is about 1.7 s, so here the instant layer matters less than it did; I expect it to matter again where one run is slower, such as finer grids, longer storms or many requests at once. What matters is whether the trained model beats an untrained proxy shown just as fast. Here it does, mainly on missed dangers.
• Put the confirmation state in the file. Without a confirmed attribute, a screenshot of the instant layer looks exactly like a result.

Appendix: traps when running Warp and PyTorch inside Kit

Setup: Kit 110.3 headless (--no-window --exec script.py) on an NVIDIA GB10 (ARM64, Ubuntu 24.04). Kit bundles Warp 1.14; pip has 1.18.

1. About 10× slower inside Kit. This is the reason the note exists.
   • Measured: 4.7 s standalone, 48–49 s inside Kit, and 74 s as a separate process while Kit runs.
   • Tried, no effect: render throttling, pausing the viewport, CPU affinity, environment variables, fewer host syncs.
   • Fixed with CUDA graphs. I moved the time step, rain switch and inflow onto the GPU and captured 200 solver steps as one CUDA graph, replayed until the storm ends. Inside Kit one run fell from 59 s to 2.0 s, faster than the original loop outside Kit (6.3 s); the answer stayed within 0.0002 m. Criterion fixed beforehand: at most 2× the standalone time.
   • What it says about the cause: moving the control onto the GPU without a graph changed nothing (58 s). The cost was most likely in launching about ten small kernels per step while Kit runs, not in reading the time step back (I did not profile it).
   • One trap: capturing the graph once and replaying it across calls failed twice. First, arrays referenced only by the captured step were garbage-collected, and the graph read freed memory: depths of tens of metres. Second, after keeping them alive, the replay hit CUDA error 719 once Kit's renderer had started. Capturing per call works and costs little. I have not found the cause of the 719.
2. Kit that never exits. A streaming Kit app held 3.4 GB of GPU memory for 19 days. In --exec scripts, an exception before post_quit() leaves Kit running. Put everything in try/finally and wrap the launch in timeout.
3. "Kit's Warp gives different answers."
   • My first comparison differed by about 2 m (1.68 m became 3.75 m).
   • It was not the version: Kit's Warp, loaded from plain Python, gave bit-identical results to 1.18.
   • The real cause was my solver at its stability limit next to vertical walls. After the fix, the difference was 0.0002 m.
   • To separate "version" from "environment", add Kit's extscache/omni.warp.core-* to PYTHONPATH.
4. PyTorch inside Kit. Kit's Python is 3.12, the same as my venv.
   • Appending the venv's site-packages to sys.path imports torch 2.14 on CUDA in 2.3 s. Appending rather than prepending keeps Kit's own NumPy.
   • wp.from_torch works across the two.
   • The first CUDA call takes about 0.3 s, so warm up before measuring.
5. import omni.timeline inside a function makes omni a local name and breaks omni.kit.app on an earlier line. Use from omni import timeline.
6. Blank captures and sticky colours. Headless captures stay blank until about 200 warm-up frames. Colour overrides persist across variants, so shoot the unpainted variant first.
7. Fixed topology for live layers. Re-creating a 36,000-cell mesh per request is slow. Define it once and update only points, sinking dry cells below the terrain.

Process and tools

Step	Tool	What it does
Design	Plant-layout GD tool (Python/NumPy)	574 layouts; 115 held out as what-if requests
Criteria	JSON with hash	Fixed before retraining and before the Kit run
Instant layers	NVIDIA Warp (fill and flow accumulation), PyTorch U-Net	Proxy and surrogate depth maps in about 0.1 s
Confirmation	NVIDIA Warp 2D inside Kit	One flood run per request: 47–59 s before CUDA graphs (5 requests); 1.7 s per step with CUDA graphs
Review	OpenUSD, Omniverse Kit 110.3 (headless)	Three depth layers, per-request attributes, confirmed flag, 6,810 values read back
Scoring	NumPy	Verdicts, IoU and latency against the pre-set criteria
Figures	matplotlib, Kit viewport captures	Generated from the saved results

Site, terrain and unit costs are synthetic or assumed; no real client or site data.

In this series: The "optimal" plant layout flooded on its own ground · Linking generative design to simulation (method) · Cross-checking against HEC-RAS 2D · Case 2: the road alignment · Inside the plant · Every alternative, at once · LinkedIn newsletter (all seven posts)