Maximum power transfer derivation
You don't even need to find the first derivative to find the max of this function (even a plot of the power as a function of \$R_T\$ would show this) but you're still after the mathematical proof since the problem asks for it...

You take expression 3.35 and find the derivative using the quotient rule, that's all:

$$ \dfrac{dP}{dR_T}=-\dfrac{2v_T^2R_L}{(R_L+R_T)^3}$$

Just like you showed in your post. From here on, this becomes one of those math problems...There is clearly no \$R_T\$ in the numerator but as \$R_T\to \infty\$, the derivative goes to zero—that is a critical point. 

The list of possible values for \$R_T\$ are [0,\$\infty\$)—can't consider negative resistance values— and these are the endpoints of the interval. In order to find the relative/absoulte max and min of a function, you need to evaluate the orginal function, \$P(R_T)\$, at the critical points (the ones you get by setting the derivative to zero) and the endpoints, see this.

That is, you now need to find \$P(0)\$ (an endpoint) and \$P(\infty)\$ (this is both an end point and a critical point). That clearly will give you a max value at \$P(0)\$.