How to hide field if blank
Then dont exclude special price from display and customize that field. There's an option @ No results behavior, where you need to check Hide if empty.

THEN, all you need to do is to style the two fields where they're both present.

That's the simplest way. There is another way, where you exclude both from display and add another field at the end "Global text area", where you can play with the tokens and rewrites.