Gibbs phenomenon in Hamming's digital filters
I think the main issue is you are jumping ahead of yourself.  You probably remember or read somewhere that the Fourier transform of a $rect$ function is a $sinc$ function.  This is true; however, no where in this section does he mention Fourier transform!  In fact, what he is doing is not Fourier transform.  

What he does in this section is to represent any periodic function as a Fourier series:

(1)    $f(\theta)=\frac{a_0}{2}+\sum_{k=0}^\infty{(a_k \cos k\theta + b_k \sin k\theta)}$

The key here is that this function doesn't contain every frequency.  It contains frequency 0 (the DC) and $\frac{k}{2\pi}$ where $k=1,2,3,...\infty$.  This is actually very important.  This function is always periodic with a period of $2\pi$.  The coefficients $a_k$ and $b_k$ are almost sampled values of the Fourier transform (but not quite, why? homework exercise! :p).  You have countably many of these coefficients.  Later in the book, you will learn that when you sample data in one space (frequency or time) it necessarily makes the counter part in the other space (time or frequency) periodic.  

In the first case, he does the Lanczos smoothing derivation where he averages the function by running a rectangular window through it (convolving with rect).  What he shows is, not surprisingly, that the coefficients get multiplied by this term:

(2)    $\frac{\sin k\pi/N}{k\pi N}$

which should look very familiar to you, of course, because it is the $sinc$ function.  However, what you are missing is that $k$ is discrete!  It is actually a sampled version of the $sinc$ function.

Effectively, he convolves a function with a $rect$ and shows that the coefficients of the resulting Fourier series (read loosely as: sampled Fourier transform) is a sampled $sinc$ function.  No surprise there.  Convolution thm says convolution in time turns into multiplication in frequency.  Fourier transform, which you know, of $rect$ is $sinc$, so convolution by $rect$ is multiplication by $sinc$ in frequency space. 

In the next section, he does something different.  He takes the Fourier series (read: sampled Fourier transform) and removes all the higher frequency coefficients.  In effect, he is taking the Fourier transform, multiplies by a $rect$, and then samples it.  For simplicity, he sets all the Fourier coefficients that did not get discarded to $1$.

What he's left with is this:

(3)    $h(\theta)=\frac{sin(N+1/2)\theta}{sin(\theta/2)}$

And you ask, why isn't this a $sinc$ function?  Can you answer it now?

The quick answer is because what is applied in frequency domain is not just a truncation, it's a truncation and a sampling operator.  What you know is that when you truncate (i.e. multiply by rect) in frequency domain, the time domain gets convolved with a $sinc$ (by Convolution thm and Fourier transform of $rect$), but this is without sampling.  

As for why the formula looks the way it does, there are two ways to look at it.  The first, which he shows, is that you can just sum the Fourier series from $-N$ to $N$, and that's what you get.

The second, which is more profound and may come up later, is that (*) when you sample a function in one space (say frequency), the corresponding function in the other space (say time) becomes the sum of shifted versions of itself.  In fact, it probably won't come up exactly like that.  The typical scenario is that sampling time domain creates periodic replication in the frequency domain (btw: this is the reason for what people call aliasing).  However, you can apply the duality property of Fourier transform to get (*).

Does equation (3) make sense now?  It is periodic.  The closer you get to 0, the more it looks like a $sinc$ function.

So, an exercise for you is to derive equation (3) by sampling and truncating in frequency space and applying inverse transform.