Transfer Function of Bessel Filter
How come that the transfer function in the ti design guide is not a
  bessel polynomial.


Let's look at the transfer function you have written: -

\$H(s) = \dfrac{1}{0.618s^2+1.3617s+ 1}\$

Rearranging: -

\$H(s) = \dfrac{1.6181}{s^2+2.2034s+ 1.6181}\$

The equation is now in standard form : \$H(s) = \dfrac{\omega_n^2}{s^2+2\zeta\omega_ns+ \omega_n^2}\$

And clearly \$\omega_n\$ = \$\sqrt{1.6181}\$ hence 2.2034/\$\sqrt{1.6181}\$ = 1.732. This bit is important because it is \$\sqrt3\$.

For a Bessel 2nd order low pass filter 2\$\zeta\$ = \$\sqrt3\$ hence zeta is 0.866.



Picture source

In the picture I've manipulated R to give me a damping ratio (zeta) of precisely 1.732 - look at the peak in the step response - 1.00433 volts - exactly right for Bessel. Look at the phase delay plotted on the upper graph - maximally flat and gradually becoming 90 degrees at the natural resonant frequency. Fd (the damped frequency) is precisely 0.5 - also indicative of Bessel.


  Can a second order bessel low pass have a different Q factor than
  0.5773?


0.5773 is the reciprocal of \$\sqrt3\$ and no it has tto be that Q for a Bessel LPF.