I am currently developing a software that needs fast inter-process and temporary I/O communication (images as well as big hdf5 arrays). The lifetime of the data is different and dependent on several things, but in most cases something between seconds to minutes. Only a few files have to be stored longer. None of the data needs to be saved persistently.Hence, I thought /dev/shm should be the way to go. However, I am struggling to benchmark /dev/shm.My first attempt was:sudo dd if=/proc/kcore of=/dev/shm/mem count=1000000...which shows the following result:1000000+0 records in1000000+0 records out512000000 bytes (512 MB, 488 MiB) copied, 0,661147 s, 774 MB/sHowever, when I run dd with the bs flag to specify how many bytes have to be read/written at a time, the result heavily changes:sudo dd if=/proc/kcore of=/dev/shm/mem bs=$((1024*1024)) count=512...results in:512+0 records in512+0 records out536870912 bytes (537 MB, 512 MiB) copied, 0,166003 s, 3,2 GB/sWhy is it so much faster to read/write very big chunks of data (many bytes 512 times) compared to small chunks of data a lot of times (1.000.000 times, like in the first attempt)?Is it legit to benchmark /dev/shm with dd using /proc/kcore as the source? Or is /proc/kcore somehow limiting the benchmark?I can not exactly tell how much performance I really do need, but I would like to have more than 1 GB/s read & write speed (because the largest amount of data I am reading or writing is 1 GB and I would like to do so in less than a second).I am mostly storing big arrays (50 MB up to 1 GB) via HDF5.
Benchmark /dev/shm
linux;io;ram;benchmark;ramdisk