Spectrum analysis of a function I asked this on Math SE but received no replies. I hope this is a relevant forum to ask in.

I would like to analyze the spectrum of the following function:
$$f(t)=\cos(t\cdot a(1+b\cos(ct)))$$
with $a,b,c$ some real parameters. As the function is non periodic (perhaps unless $a,b,c$ are fine-tuned), the most natural approach is to Fourier transform it. But doing it by hand is quite horrible, Wolfram Alpha aso struggles, and using Mathematica I get
$$-ia\cdot\cos(\sqrt{2\pi})\cdot \delta'[\omega] - abc\cdot\cos^2(\sqrt{2\pi})\cdot \delta''[\omega]$$
I'm only interested in the real part of the result. I'm now trying to make sense of what I got.

1) Does the Mathematica result seem reasonable to you?

2) Looking at it from a generator function perspective (as in here), it seems there is a sharp peak centered at zero, accompanied by more minor peaks at $\pm\Omega$ for some $\Omega&gt;0$. I find this result surprising, as I expected that the spectrum would be centered at $a$ as the "main" frequency followed by some smaller harmonics or some sort of $sinc$-like function.

3) Following a much simpler argument than the full Fourier transform, I try to think of it as a function that has a frequency that oscillates in "time" ($t$). Then maybe as an analogy I can ask myself which frequencies are most probable. In that case, I can think of the $\cos(ct)$ term as the cosine of a uniform random variable $ct:= Y\sim Uni(-\pi,\pi)$, and so the probability density function is actually U-shaped centered at $a$ and given by a function in the general form of $$f_Y(x)=\frac{1}{\pi\sqrt{1-x^2}}$$(normalized etc.)

But these two analyses are far off from one another, and I would like to understand what is going on.

A counter-argument to my probabilistic interpretation is that the term that determines the intensity of a frequency component is the coefficients of the Fourier decomposition (if it were periodic), and here this is not a linear combination of frequencies with different probabilities, but something else. Still, I need some help explaining the results. Thanks!
The phase term of your outermost cosine function is time-varying: $\phi(t):=t a (1+b \cos(ct))$. The instantaneous frequency is given by $\frac{\partial \phi}{\partial t} = a+ab \cos(ct) - abct \sin(ct).$ In particular, in a tiny neighborhood around $t=0$ we can argue that $f(t)$ has only one frequency component $a+ab$. At large values of $t$, the $abct\sin(ct)$ term takes control.

Notice that the frequency varies with time. This makes it difficult to intuitively interpret the Fourier transform which assumes that all frequencies exist for all times. For such signals, it may be better to use time-frequency analysis by looking at the spectra of windowed segments of $f(t)$ around different values of $t$.