Convert fixed frequency bandwidth samples to time domain
As I suspect, it was an issue with insufficient number of points in freq domain data. I convert time domain data from the green trace to freq domain and got exactly the same values in freq domain as on the first picture with original freq data from network analyzer.