Code Smell in Unit Testing Serialized Classes
I think the bigger issue is that you have an operation with side-effect in the property setter SerialPortConnection,  which means that if your unit test fails, it will not be clear whether it is due to a problem with deserialisation or with the operation. 

I would suggest moving the operation (i.e. writing "R" to the port) in a separate Initialize method that you can test individually.