why HDD is more sensitive to the change of innodb_fsync_threshold than SSD
Might a hardware perspective explain the differences?
The steps of disk I/O are something like:

buffering (for writes)
send data to the device
&quot;Seek&quot; the desired cylinder on the device
Wait for the read/write head to be over the sector that needs read/written
perform the I/O
buffering (for reads)

Steps 3 and 4 are the dominant time consumers for HDD.  For SDD there is no &quot;head&quot; to move nor platter spinning, so both of these steps take no time.
More to the point, 3 and 4 take a variable amount of time, based on, well, the phase of the moon.  And a good controller, etc, can play games that optimize disk activity.
Most benchmarks have &quot;real life&quot; flaws.

They ignore interference from other threads -- that may be fighting for access to the same drive.
They pound on the resource to find the &quot;max&quot; that can be handled.  While this is an interesting metric, in real life if the server is at the max, it is about to keel over.  A butterfly flapping its wings can cause it to fall off the cliff.

Some optimizations:

RAID can cache things -- read and/or write -- to smooth out activity.
RAID can &quot;lie&quot; about when action is finished.  This is because, for example, a &quot;battery backed write cache&quot; can safely hold a write in ram on the controller while returning &quot;successful write&quot; to the user.  This may lead to a puzzling inflection point in the graph -- the device will suddenly slow down when the write cache is full.