How averaging works in GFS model from NOAA?
The GFS is a model, and the model have a given time-step equal or smaller than the time-step of the output, that in this case is 3 hours. Different processes can be computed using different or even multiple time-steps.

Some of the reported variables are instantaneous, others are cumulative, and others are time averaged.

According to the documentation for the GFS radiation driver module in the GFS Operational Physics Documentation, the Downwards shortwave radiation flux correspond to a time averaged output quantity, meaning that for each model grid cell, its value is stored every internal time step (deltim) to then report the time-weighted average of all stored values, which is what you get in the 3-hour output.
However, I don't know what is the exact value used for deltim or how the time weights are computed.

Therefore, what you said seem to be pretty much the case. This is that the value reported for timestamp hour is the average of all the intermediate values computed between hour-3+deltim and hour using some time weights schema that is unclear to me.

In the link above, you can find all the source code for GFS, so it seem that to figure our the details of the time averaging you might need to search in the code where the values of the array fluxr (that store the intermediate time-step radiation flux values) is averaged and see there how it is done.