What happens when code is uploaded using the bootloader?
When you reset a Uno running the Optiboot loader, the bootloader first flashes pin 13 three times.



Top line (gray) is sent to the Arduino, middle line (orange) is sent from the Arduino.

While that is happening, the program avrdude running on your computer is sending a query to the device:

STK_GET_SYNC / CRC_EOP  (0x30/0x20)


The Arduino doesn't notice the first "get sync" because it is busy flashing pin 13. Once it is done it notices the "get sync" (it would be buffered by the serial hardware) and replies:

STK_INSYNC / STK_OK (0x14/0x10)


It looks like avrdude got a bit impatient, and timed out, because it tries again with the "get sync" query. This time Optiboot responds immediately.



The rest of the uploading is described in the next image. Example produced uploading the stock "Blink" program.



(Click on the image above for a larger version)



The steps are:


Query: Get Sync? Reply: In Sync.
Query: Get parameter? (major version) Reply: version 4.
Query: Get parameter? (minor version) Reply: version 4.
Set device parameters. The following device parameters are sent to the chip:

0x42  // STK_SET_DEVICE
0x86  // device code
0x00  // revision
0x00  // progtype: “0” – Both Parallel/High-voltage and Serial mode
0x01  // parmode: “1” – Full parallel interface
0x01  // polling: “1” – Polling may be used
0x01  // selftimed: “1” – Self timed
0x01  // lockbytes: Number of Lock bytes.
0x03  // fusebytes: Number of Fuse bytes
0xFF  // flashpollval1
0xFF  // flashpollval2
0xFF  // eeprompollval1
0xFF  // eeprompollval2
0x00  // pagesizehigh
0x80  // pagesizelow
0x04  // eepromsizehigh
0x00  // eepromsizelow
0x00  // flashsize4
0x00  // flashsize3
0x80  // flashsize2
0x00  // flashsize1
0x20  // Sync_CRC_EOP


Optiboot ignores all those and replies with In Sync/OK. :)
Set extended device parameters:

0x45  // STK_SET_DEVICE_EXT
0x05  // commandsize: how many bytes follow
0x04  // eeprompagesize: EEPROM page size in bytes.
0xD7  // signalpagel: 
0xC2  // signalbs2: 
0x00  // ResetDisable: Defines whether a part has RSTDSBL Fuse 
0x20  // Sync_CRC_EOP


Optiboot ignores all those as well and replies with In Sync/OK.
Enter program mode. Reply: In Sync/OK.
Read signature. Optiboot replies with 0x1E 0x95 0x0F without actually reading the signature.
Write fuses (four times). Optiboot does not write the fuse but just replies In Sync/OK.
Load address (initially 0x0000). The address is in words (ie. a word is two bytes). This sets the address for where the next page of data will be written.
Program page (up to 128 bytes are sent). Optiboot replies "In Sync" immediately. Then there is a pause of about 4 ms while it actually programs the page. Then it replies "OK".
Load address (now 0x0040). This is address 64 in decimal, ie. 128 bytes from the start of program memory.
Another page is written. This sequence continues until all the pages are written.
Load address (back to 0x0000). This is for verifying the write.
Read page (up to 128 bytes are read). This is for verifying. Note that even if the verify fails, the bad data has already been written to the chip.
Leave programming mode.




What does "not in sync" mean?

As you can see from the above, every step through the programming sequence the Arduino is expected to reply with "In Sync" (0x14), possibly followed by some data, followed by "OK" (0x10). 

If it is "not in sync" that means that avrdude did not get the "in sync" response. Possible reasons could be:


Wrong baud rate used 
Wrong serial port selected in IDE
Wrong board type selected in IDE
No bootloader installed
Wrong bootloader installed
Board not configured to use the bootloader (in the fuses)
Some device plugged into pins D0 and D1 on the Arduino, interfering with serial commications
The USB interface chip (ATmega16U2) not working properly
Wrong clock for the board
Wrong fuse settings on the Atmega328P (eg. "divide clock by 8")
Board/chip damaged
Faulty USB cable (some USB cables provide power only, and are not for data, eg. cheap cables for USB fans)




What is "in sync"?

As mentioned above, the response "In sync" means that the Arduino (bootloader) is synchronised with the uploading program.



What protocol is being used?

The protocol is the STK500 protocol as documented by Atmel. See the references below.



References


Optiboot source code - GitHub
AVR061: STK500 Communication Protocol
AVR068: STK500 Communication Protocol version 2
Have I bricked my Arduino Uno? Problems with uploading to board
AVR Downloader/UploaDEr - avrdude


Note: STK500 Version 2 is not used in Optiboot, but it is included for information in case you are using boards like the Mega2560.





STK500 constants

/* STK500 constants list, from AVRDUDE */
#define STK_OK              0x10
#define STK_FAILED          0x11  // Not used
#define STK_UNKNOWN         0x12  // Not used
#define STK_NODEVICE        0x13  // Not used
#define STK_INSYNC          0x14  // ' '
#define STK_NOSYNC          0x15  // Not used
#define ADC_CHANNEL_ERROR   0x16  // Not used
#define ADC_MEASURE_OK      0x17  // Not used
#define PWM_CHANNEL_ERROR   0x18  // Not used
#define PWM_ADJUST_OK       0x19  // Not used
#define CRC_EOP             0x20  // 'SPACE'
#define STK_GET_SYNC        0x30  // '0'
#define STK_GET_SIGN_ON     0x31  // '1'
#define STK_SET_PARAMETER   0x40  // '@'
#define STK_GET_PARAMETER   0x41  // 'A'
#define STK_SET_DEVICE      0x42  // 'B'
#define STK_SET_DEVICE_EXT  0x45  // 'E'
#define STK_ENTER_PROGMODE  0x50  // 'P'
#define STK_LEAVE_PROGMODE  0x51  // 'Q'
#define STK_CHIP_ERASE      0x52  // 'R'
#define STK_CHECK_AUTOINC   0x53  // 'S'
#define STK_LOAD_ADDRESS    0x55  // 'U'
#define STK_UNIVERSAL       0x56  // 'V'
#define STK_PROG_FLASH      0x60  // '`'
#define STK_PROG_DATA       0x61  // 'a'
#define STK_PROG_FUSE       0x62  // 'b'
#define STK_PROG_LOCK       0x63  // 'c'
#define STK_PROG_PAGE       0x64  // 'd'
#define STK_PROG_FUSE_EXT   0x65  // 'e'
#define STK_READ_FLASH      0x70  // 'p'
#define STK_READ_DATA       0x71  // 'q'
#define STK_READ_FUSE       0x72  // 'r'
#define STK_READ_LOCK       0x73  // 's'
#define STK_READ_PAGE       0x74  // 't'
#define STK_READ_SIGN       0x75  // 'u'
#define STK_READ_OSCCAL     0x76  // 'v'
#define STK_READ_FUSE_EXT   0x77  // 'w'
#define STK_READ_OSCCAL_EXT 0x78  // 'x'