Serial data received in wrong bits
First, let's look at the data you mention:

0x55AA  0101010110101010
0xD56A  1101010101101010
0xB5CA  1011010111001010


As we can see, the data is close, but slightly off.  The reasons for serial data being "slightly off" are many, but an obvious one in your particular case is the serial data rate.  In order for asynchronous serial data to be received correctly, the original clock must be recreated at the receiving end.  A rule of thumb for serial links is that if the clock rates are off by more than 3%, you'll get errors much like the ones you describe.  

So at a rate of 460,800 bits/second, that's a little over 2.27\$\mu\$s per bit.  Here's some troubleshooting advice:


check the bit rate clocks on both ends and make sure the frequencies are within 3% of each other.
keep your wiring as short as possible.
check the line termination on both ends to make sure it's got the same impedance as the line.
check the received waveform with an oscilloscope.  Look for ringing or slow transitions.


You might also consider if there is a CRC in the packet that would at least help you detect an error.

How to check clocks

First, let me state my assumptions:


each side is able to send arbitrary data
data is being sent in 8-N-1 form (8 data bits, no parity, 1 stop bit) 
you can set up either end to send continuously


With that said, let's look at how serial data is typically sent.  Let's look at the bytes 0x55, 0x55 as they might be sent on a single-ended (RS-232) data link.

+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---
|   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   
+---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   +---+   
  S   1   0   1   0   1   0   1   0   P   S   1   0   1   0   1   0   1   0   P 


In this diagram, S is a start bit and P is a stop bit and the 8 data bits are sent least significant bit first, as is the usual.  It should be clear from looking at this that if you measure the frequency of this waveform, it should be exactly half the bit rate.