Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture

Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores, propose that modern NVIDIA GPUs express dependency constraints through compiler-encoded control fields, and that the hardware merely performs the corresponding wait and counting mechanisms according to those encodings. Data dependencies of fixed-latency instructions are managed by the Stall counter field, while those of variable-latency instructions are managed by the Dependence counter. They adopt a methodology following Jia et al.'s Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking:

• Construct a producer A and a consumer B that depends on its result.
• For a fixed-latency A, progressively decrease the stall in the control word.
• Until the result used by B becomes incorrect.
• Take the last stall value that still produces a correct result as the latency of A.

They measured the producer Stall Counter values that satisfy producer-consumer data dependencies under different register bank conflict conditions, and found that this value does not vary with bank conflicts, which rules out the existence of a variable-latency Operand Collector stage during instruction execution. In addition, they measured the latency at which the first MOV's result can be correctly obtained by the second instruction in MOV-MOV and MOV-LDG instruction pairs, and found that the two latencies differ, pointing to the possible existence of a data forwarding path inside the pipeline to which MOV belongs among the fixed-latency instructions.

This paper further investigates the producer Stall Counter conditions that satisfy producer-consumer data dependencies between different classes of fixed-latency instructions, and introduces ILP. We find that simply setting the Stall Counter to 4 does not satisfy data dependencies in all cases; this is a problem involving multiple factors, including producer generation timing, data forwarding or write-back paths, consumer source-operand read timing, producer instruction type, consumer instruction type, execution pipeline, and instruction-level parallelism. All measurements in this paper were performed using SASS-Testbench; prior to the microbenchmark snippet, R52-R60 were zeroed to eliminate interference from uninitialized values.

Both producer and consumer are arithmetic instructions

ILP = 1

The following code snippet is chosen:

[B------:R-:W-:Y:S04] FADD R12, R8, R9;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;

By adjusting the Stall field, the minimum value that ensures R12 is correctly read by the FMA is confirmed to be $D_{Producer-Consumer_{ADD-FMA}} = 4$ . At this point the snippet runs in 5 cycles.

ILP = 2/3/4

We progressively increase the number of producer<->consumer pairs to observe the effect of ILP on the result visibility of fixed-latency instructions. The following code snippets are chosen:

• ILP = 2

[B------:R-:W-:Y:S01] FADD R12, R8, R9;
[B------:R-:W-:Y:S03] FADD R14, R8, R9;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;

The minimum value for R12 and R14 to be correctly read by the FMA is $D_{Producer-Consumer_{ADD-FMA}} = 4$ . At this point the snippet runs in 6 cycles.

• ILP = 3

[B------:R-:W-:Y:S01] FADD R12, R8, R9;
[B------:R-:W-:Y:S01] FADD R14, R8, R9;
[B------:R-:W-:Y:S02] FADD R16, R8, R9;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;
[B------:R-:W-:Y:S01] FFMA R56, RZ, RZ, R16;

The minimum value for R12, R14, and R16 to be correctly read by the FMA is $D_{Producer-Consumer_{ADD-FMA}} = 4$ . At this point the snippet runs in 7 cycles.

• ILP = 4

[B------:R-:W-:Y:S01] FADD R12, R8, R9;
[B------:R-:W-:Y:S01] FADD R14, R8, R9;
[B------:R-:W-:Y:S01] FADD R16, R8, R9;
[B------:R-:W-:Y:S01] FADD R18, R8, R9;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;
[B------:R-:W-:Y:S01] FFMA R56, RZ, RZ, R16;
[B------:R-:W-:Y:S01] FFMA R58, RZ, RZ, R18;

The minimum value for R12, R14, R16, and R18 to be correctly read by the FMA is $D_{Producer-Consumer_{ADD-FMA}} = 4$ . At this point the snippet runs in 8 cycles. Overall, when both the producer and the consumer are arithmetic instructions, the "latency is 4" model proposed by the community and the literature is consistent with the observations.

Both producer and consumer are MOV

ILP = 1

The following code snippet is chosen:

[B------:R-:W-:Y:S04] MOV R12, R8;

[B------:R-:W-:Y:S01] MOV R52, R12;

The minimum value that ensures R12 is correctly read is $D_{Producer-Consumer_{MOV-MOV}} = 4$ . At this point the snippet runs in 5 cycles.

ILP = 2/3/4

• ILP = 2

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S03] MOV R14, R8;

[B------:R-:W-:Y:S01] MOV R52, R12;
[B------:R-:W-:Y:S01] MOV R54, R14;

The minimum value for data integrity is $D_{Producer-Consumer_{MOV-MOV}} = 4$ . At this point the snippet runs in 6 cycles.

• ILP = 3

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S01] MOV R14, R8;
[B------:R-:W-:Y:S02] MOV R16, R8;

[B------:R-:W-:Y:S01] MOV R52, R12;
[B------:R-:W-:Y:S01] MOV R54, R14;
[B------:R-:W-:Y:S01] MOV R56, R16;

The minimum value for data integrity is $D_{Producer-Consumer_{MOV-MOV}} = 4$ . At this point the snippet runs in 7 cycles.

• ILP = 4

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S01] MOV R14, R8;
[B------:R-:W-:Y:S01] MOV R16, R8;
[B------:R-:W-:Y:S01] MOV R18, R8;

[B------:R-:W-:Y:S01] MOV R52, R12;
[B------:R-:W-:Y:S01] MOV R54, R14;
[B------:R-:W-:Y:S01] MOV R56, R16;
[B------:R-:W-:Y:S01] MOV R58, R18;

The minimum value for data integrity is $D_{Producer-Consumer_{MOV-MOV}} = 4$ . At this point the snippet runs in 8 cycles. When both the producer and the consumer are MOV, the "latency is 4" model proposed by the community and the literature remains accurate.

Producer is MOV, consumer is an arithmetic instruction

Next we examine the cases where the producer and consumer are not of the same class, starting with the case where the producer is MOV and the consumer is an arithmetic instruction.

ILP = 1

[B------:R-:W-:Y:S04] MOV R12, R8;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;

The minimum value for R12 to be correctly read by the FMA is $D_{Producer-Consumer_{MOV-FMA}} = 4$ . At this point the snippet runs in 5 cycles.

ILP = 2/3/4

• ILP = 2

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S03] MOV R14, R8;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;

We observe that across 10 different inputs, the result of R52 remains correct while the result of R54 is always corrupted. After testing, the nominal distance that ensures neither R52 nor R54 is corrupted is $D_{Producer-Consumer_{MOV-FMA}} = 5$ :

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S04] MOV R14, R8;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;

• ILP = 3 $D_{Producer-Consumer_{MOV-FMA}} = 4$ :

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S01] MOV R14, R8;
[B------:R-:W-:Y:S02] MOV R16, R8;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;
[B------:R-:W-:Y:S01] FFMA R56, RZ, RZ, R16;

At this point R12 and R16 are correct while R14 is corrupted, and the snippet runs in 7 cycles. Likewise, the nominal distance that ensures neither R12, R14, nor R16 is corrupted is $D_{Producer-Consumer_{MOV-FMA}} = 5$ .

• ILP = 4

[B------:R-:W-:Y:S01] MOV R12, R8;
[B------:R-:W-:Y:S01] MOV R14, R8;
[B------:R-:W-:Y:S01] MOV R16, R8;
[B------:R-:W-:Y:S01] MOV R18, R8;

[B------:R-:W-:Y:S01] FFMA R52, RZ, RZ, R12;
[B------:R-:W-:Y:S01] FFMA R54, RZ, RZ, R14;
[B------:R-:W-:Y:S01] FFMA R56, RZ, RZ, R16;
[B------:R-:W-:Y:S01] FFMA R58, RZ, RZ, R18;

R12 and R16 are correct while R14 and R18 are corrupted. Likewise, the nominal distance that ensures data integrity is $D_{Producer-Consumer_{MOV-FMA}} = 5$ .

Producer is an arithmetic instruction, consumer is MOV

Then the case where the producer is an arithmetic instruction and the consumer is MOV.

ILP = 1

[B------:R-:W-:Y:S04] FFMA R22, RZ, RZ, R8;

[B------:R-:W-:Y:S01] MOV R52, R22;

The minimum value for R22 to be correctly read is $D_{Producer-Consumer_{FMA-MOV}} = 4$ . At this point the snippet runs in 5 cycles.

ILP = 2/3/4

• ILP = 2

[B------:R-:W-:Y:S01] FFMA R22, RZ, RZ, R8;
[B------:R-:W-:Y:S03] FFMA R24, RZ, RZ, R8;

[B------:R-:W-:Y:S01] MOV R52, R22;
[B------:R-:W-:Y:S01] MOV R54, R24;

At this point, $D_{Producer-Consumer_{FMA-MOV}} = 4$ . The result of R52 remains correct while the result of R54 is corrupted, and the result of R54 is always the constant 0x3f7fffff. After testing, the nominal distance that ensures neither R52 nor R54 is corrupted is $D_{Producer-Consumer_{FMA-MOV}} = 5$ .

• ILP = 3

[B------:R-:W-:Y:S01] FFMA R22, RZ, RZ, R8;
[B------:R-:W-:Y:S01] FFMA R24, RZ, RZ, R8;
[B------:R-:W-:Y:S02] FFMA R26, RZ, RZ, R8;

[B------:R-:W-:Y:S01] MOV R52, R22;
[B------:R-:W-:Y:S01] MOV R54, R24;
[B------:R-:W-:Y:S01] MOV R56, R26;

The results of R22 and R26 remain correct while the result of R24 is always corrupted, and the result of R24 is still the constant 0x3f7fffff. Likewise, the nominal distance that ensures data integrity is $D_{Producer-Consumer_{FMA-MOV}} = 5$ .

• ILP = 4

[B------:R-:W-:Y:S01] FFMA R22, RZ, RZ, R8;
[B------:R-:W-:Y:S01] FFMA R24, RZ, RZ, R8;
[B------:R-:W-:Y:S01] FFMA R26, RZ, RZ, R8;
[B------:R-:W-:Y:S01] FFMA R28, RZ, RZ, R8;

[B------:R-:W-:Y:S01] MOV R52, R22;
[B------:R-:W-:Y:S01] MOV R54, R24;
[B------:R-:W-:Y:S01] MOV R56, R26;
[B------:R-:W-:Y:S01] MOV R58, R28;

The results of R24 and R28 remain correct while the results of R22 and R26 are corrupted, and the results of R22 and R26 are the constant 0x3f7fffff. The nominal distance that ensures data integrity is $D_{Producer-Consumer_{FMA-MOV}} = 5$ .

Further scanning

We tried a variety of producer-consumer combinations and compiled the minimum $D$ values at which data is not corrupted for several combinations under ILP=1/2/3/4, as follows:

Each cell is formatted as ILP=1/ILP=2/ILP=3/ILP=4, giving the minimum $D$ value at which data is not corrupted. The distance model follows the earlier text: the stall of the last producer in a consecutive ILP group is raised so that the nominal distance of every $P_i \to C_i$ equals $D$ (one can verify that $\text{distance}_j = (N-j) + [D-(N-1)] + (j-1) = D$ holds for any $j$ ). Producers and consumers uniformly adopt a "pass-through" form; the criterion is that all 15 repetitions are bit-exact.

Backend pipeline attribution of each instruction

We measured the execution pipeline attribution of each instruction individually using Nsight Compute's sm__inst_executed_pipe_* metrics.

Several observations:

1. Even for the same producer, the moment at which its result becomes visible within the register file depends on the specific consumer instruction. This phenomenon indicates that the moment at which a fixed-latency instruction's result can be consumed is not simply an intrinsic constant of the producer. Possible explanations include the specific timing at which the consumer side reads its operands, the forwarding path between producer and consumer, and data transfer between different backend pipelines.
2. Apart from the FFMA ILP=1 case, most instructions exhibit the same pairing pattern, which correlates with the pipeline statistics. This suggests that data forwarding or write-back very likely reuses components within the pipeline.
3. Compared with pairings within the same pipeline, pairings between different pipelines generally incur higher latency, which hints that the forwarding path within a pipeline may have greater bandwidth or lower latency.

In summary, this paper reveals a fact: the wait value obtained from an isolated dependency pair cannot be directly used as a general scheduling constraint for all arrangements; the minimum correct nominal interval also depends on the consumer, the operand form, and the surrounding schedule.

Limitations

• In our current microbenchmark, the producer target and consumer target in the dependency chain are all expanded using even-numbered registers. The odd/even alternating case and the all-odd case have not yet been measured.
• The current operand positions for each instruction are:
  • MOV {P}, {SRC}
  • FADD {P}, {SRC}, RZ
  • FFMA {P}, RZ, RZ, {SRC}
  • FMUL {P}, {SRC}, R11
  • IADD3 {P}, PT, PT, {SRC}, RZ, RZ
  • LOP3.LUT {P}, {SRC}, RZ, RZ, 0xf0, !PT
  • IMAD {P}, RZ, RZ, {SRC}
  • SHF.L.U32 {P}, {SRC}, RZ, RZ The effects may differ when the operands are in other positions; this has not yet been measured.
• The assembly of the microbenchmarks was performed entirely with cubit, which is not an NVIDIA-official/publicly endorsed tool; the cubin can be legally decoded by nvdisasm.