Atmega8 - Max output frequency seems to be devide by 4
Have a look at the assembly being generated, both of your cases will have some extra instructions you haven't thought about. In a while loop, for example, there will be an instruction to jump back to the beginning of the loop.

The timer approach is better, but you are going about it the wrong way. When the timer fires, you are calling a function. There are several instructions associated with that.  Fortunately the 16-bit timers on ATmega* chips can do other things than trigger interrupts, they can also toggle pins directly.  Try:

TCCR1B = 0x08 | 0x01; 
TCCR1A = 0x80; 
OCR1A = 1;
TCNT1 = 0;


This will cause timer1 to count to one, then toggle the associated output pin, then count to one and toggle it back. As written, you'll get a F_CPU/4 clock out, so twice as fast as what you have and without blocking the system up so you can't do anything else.  You can probably get a F_CPU clock out by adjusting the fuses to put the master clock on one of the output pins.

Check the datasheet for which pin is the output - on most ATmegas you can choose one of two pins, you don't get to choose an arbitrary pin if you do it this way.

Here's a handy link explaining what some of those registers do: http://www.avrbeginners.net/architecture/timers/timers.html#tccrb



Now you've posted your disassembled while loop, we can see what was happening:

1e4:    8f ef ldi   r24, 0xFF   ; 255   Load the value 0xFF into a register
1e6:    81 bf out   0x31, r24   ; 49    Write that value to OSCCAL
1e8:    92 e0 ldi   r25, 0x02   ; 2     Load the value 0x02  into a register
1ea:    85 b3 in    r24, 0x15   ; 21    Read the pin state port into a register &lt;-
1ec:    89 27 eor   r24, r25            XOR together the two registers           |
1ee:    85 bb out   0x15, r24   ; 21    Write the result to the pin state port   |
1f0:    fc cf rjmp  .-8 ;               Jump back to  ----------------------------


So your while loop includes the instructions: in, eor, out, rjump, which take 1+1+1+2=5 clock cycles, and toggle the pin once. So 10 cycles to toggle it twice.  Since you're seeing 1.6MHz out, that implies that by setting OSCCAL to 0xFF you have obtained a clock speed of 16MHz. Note that although this works, the datasheet recommends against it.