The XGS 136's receive rings on the coprocessor's doorbells, 2026-10-20¶
Until today the XGS 136's host emptied its receive rings on a clock: every millisecond since
2026-10-17, and only then. The coprocessor has
four target-to-host doorbells for the GIU - MSI-X vectors the driver allocated and counted - and they
never rang, because every receive queue was added with msix_id 0. The vendor's own host driver runs
its receive on them by default (msix_mode 1, "Rx"), giving queue q the facility's doorbell q,
whose id is that vector's place in the order the vectors are allocated in.
contrib/npuep/npugiu.c now does the same when the loader tunable hw.npuep.giu.rx_msix is 1: each
receive queue is added on its doorbell, and a doorbell schedules one pass over the rings on a task of
its own, its frames handed up as the poll's are. The poll stays underneath at whatever
dev.npuep.0.giu.poll_ms says. New counts: giu.rx_msix, giu.rx_irqs (per queue),
giu.rx_task_runs, giu.rx_task_frames.
Measured¶
The same module, npuep.ko d34004f6, reloaded once with the tunable at 1 and once at 0, so the one
thing that changed is the doorbells. One TCP stream of 10 s from the PC's adapter on F1 into the 136's
own host and one back, then twenty pings, at poll intervals of 1, 10 and 100 ms. The first run after
each reload is left out: the PC's first stream after it has been idle is slow, as
531 found.
| the PC to the 136's host | the 136's host to the PC | ping | frames the doorbells' passes took | |
|---|---|---|---|---|
| doorbells off, poll every 1 ms | 947, 947 Mbit/s | 915, 908 | 0 ms | - |
| doorbells off, poll every 10 ms | 270, 271 Mbit/s | 548, 583 | 5, 4 ms | - |
| doorbells on, poll every 1 ms | 892, 843 Mbit/s | 948, 947 | 0 ms | 98.7, 99.3 % |
| doorbells on, poll every 10 ms | 948, 948, 948 Mbit/s | 946, 887, 915 | 0 ms | 99.9 % |
| doorbells on, poll every 100 ms | 840 Mbit/s | 949 | 0 ms | 100.0 % |
So the doorbells carry the receive: with them nearly every frame is taken by their passes, and the rate and the ping no longer depend on the clock - at a poll ten times slower the stream ran at the gigabit link's rate, where the poll alone fell to a quarter of it and the ping to five milliseconds. At a 1 ms poll both ways reach the link.
Script: images/deploy/lab-136-rx-msix.sh, its report lab-136-rx-msix-report.sh, photographed.
What it does not mean¶
- It is off by default. A loader tunable, chosen before the queues are added; no soak has run with it on yet, and that comes before it is anyone's default.
- One stream, to the host. Forwarded traffic in hardware never reaches these rings; what reaches them is the host's own and what the host forwards.
- The poll is still there, under the doorbells, as the net; at 1 ms it competes with them and the stream into the host was a little slower than at 10.
The discussion of this page is held in the project's GitHub Discussions. Showing it here loads giscus from giscus.app and GitHub, and writing in it needs a GitHub account - what that means.