The climate belongs to the class of non-equilibrium forced and dissipative systems, for which most results of quasi-equilibrium statistical mechanics, including the fluctuation-dissipation theorem, do not apply. We show for the first time how the Ruelle linear response theory, developed for studying rigorously the impact of perturbations on general observables of non-equilibrium statistical mechanical systems, can be applied to analyze the climatic response. We choose as test bed the Lorenz 96 model, which has a well-recognized prototypical value. We recapitulate the main aspects of the response theory and propose some new results. We then analyze the frequency dependence of the response of both local and global observables to perturbations with localized as well as global spatial patterns. We derive analytically the asymptotic behaviour, validity of Kramers-Kronig relations, and sum rules for the susceptibilities, and related them to parameters describing the unperturbed properties of the system. We verify the theoretical predictions from the outputs of the simulations with great precision. The theory is used to explain differences in the response of local and global observables, in defining the intensive properties of the system and in generalizing the concept of climate sensitivity to all time scales. We also show how to reconstruct the linear Green function, which maps perturbations of general time patterns into changes in the expectation value of the considered observable. Finally, we propose a general methodology to study Climate Change problems by resorting to few, well selected simulations and discuss the specific case of surface temperature response to changes of the $CO_2$ concentration. This approach may provide a radically new perspective to study rigorously the problem of climate sensitivity and climate change.