In this paper a new output-only measurement based method is proposed which allows identifying the modal parameters of structures subjected to natural loads such as wind, ocean waves, traffic or human walk. In contrast to the existing output-only identification techniques which model the unmeasured load as white noise process, statistical information about the dynamic excitation, e.g. obtained by measurements of the wind fluctuations in the vicinity of the structure, are taken into account which improve the identification results as well as allow identifying the unmeasured load process exciting the structure. The identification problem is solved on basis of a recently developed method called H-fractional spectral moment (H-FSM) decomposition of the transfer function which allows representing Gaussian random processes with known power spectral density (PSD) function as output of a linear fractional differential equation with white noise input. In the present work the efficiency and accuracy of this method is improved by the use of an alternative fractional operator. Based on the H-FSM decomposition a state space representation of arbitrarily correlated Gaussian processes is proposed which neither requires the factorization of the PSD function nor any optimization procedure. Combined with the state space model of the structure, it leads to an overall model with white noise input, which can be efficiently combined with any state-space model-based parameter identification algorithms such as the (weighted) extended Kalman filter algorithm used here. The method is applied for the identification of the stiffness and damping parameters of a three story shear building subjected to wind turbulences with von Karman velocity PDF function.