Transform Function with Non Linearity
Transfer function tries to describe a system using the Fourier Transform (More generally the Laplace Transform / Z Transform).
Yet in order for the Transform function to dependent on the frequency and amplitude only the input system must be LTI system.
Hence there is no Transfer Function in the Fourier Domain to your system which can be described the way you want.
Linear Time Invariant (LTI) System
By Convolution Theorem a convolution of LTI System can be described by a transfer function in Fourier Domain.
What do mean by a transfer function? We mean having $ H \left( \omega \right) $ which is independent of the input.
As for any output and input we can calculate the Fourier Transform and have the ratio of them as the Transfer Function. But only for LTI the result of this ratio will be constant.