Measure the boundary you ship.
A million elements can still be a poor offload workload. Submission, synchronization, transfer, memory traffic, and arithmetic intensity determine whether a physical backend pays.
Element count is not work
T_cpu = N C_cpu
T_backend = T_submit + T_sync + T_transfer + N C_backendOffload wins only when the per-element advantage exceeds fixed and transfer costs. A light 32-bit map may move roughly eight bytes per element for only a few integer operations, so its arithmetic intensity remains low even at a large count.
One checked profile, three cost boundaries
Resident creation + input copy/upload
| Path | Setup median | Interpretation |
|---|---|---|
| CPU | 86.542 µs | Resident job creation and input copy. |
| Metal | 477.500 µs | Resident job creation and input upload. |
| Vulkan/MoltenVK | 561.000 µs | Apple translation path, not native Vulkan evidence. |
Host: checked M4 Pro profile, N = 1,048,576. The timer encloses
Program::resident(input). Compilation, Job::run(), and result
read are outside it. These numbers describe setup, not CPU-versus-GPU compute speed.
Compute-heavy warm execution
| Path | Warm run median | CPU / path |
|---|---|---|
| CPU | 324.959 µs | 1.000× |
| Metal | 123.666 µs | 2.628× |
| Vulkan/MoltenVK | 190.542 µs | 1.705× |
Workload: dense expensive map, N = 262,144. Resident creation and one
validated warmup happen before sampling; each sample times only Job::run().
The suite accepts the result only when its graph and output hashes match the CPU reference.
Submission amortization
| Path | 64 serial jobs | One Batch | Serial / Batch |
|---|---|---|---|
| Metal | 7,347.062 µs | 227.645 µs | 32.274× |
| Vulkan/MoltenVK | 9,136.105 µs | 583.146 µs | 15.667× |
Workload: 64 independent jobs of 64 elements on the same checked M4 Pro profile. This proves the structural benefit of one submission boundary for this case. The serial wall path also includes per-job stats and host-loop work, so the ratio is not a pure kernel or submission speedup.
Optimize shape without changing meaning
- Program removes repeated graph construction and compilation.
- Resident removes warm allocation and host transfer.
- Pipeline removes intermediate host materialization and extra submissions.
- Batch amortizes submission and synchronization across independent jobs.
Measure the boundary on your host
The single-Flow abstraction does not create a portable performance ranking. A new device, runtime, driver, lowering, data shape, or execution surface requires its own admitted evidence.
These commands are repository engineering harnesses, not programs shipped in the binary SDK. Run them from a runD source checkout after building and installing the matching Release SDK.
tools/measure/compute/run --resident cpu
tools/measure/compute/run --resident metal
tools/measure/compute/run --resident vulkan
tools/measure/compute/run --pipeline metal
tools/measure/compute/run --pipeline vulkanThe checked baseline stores observed values and separate regression upper bounds. A row is admitted only after semantic identity and output evidence pass; a passing upper bound is not a speedup claim. For exact methodology and additional routes, inspect the optional engineering measurement reference ↗.