Transmission Line and capacitive effect
The source termination resistor value should be equal to the impedance of the transmission line minus the output impedance of the driving IC \$\left(R_\text{source}=R_\text{TL}-R_\text{out}\right)\$. It should be placed as close as possible to the driving IC.  Don't be surprised if you have to try a couple different values to get the signal to be perfect.

When using source termination you are actually cutting the signal that goes out in half. It looks full scale because when you match the source termination to the line impedance you get a perfect reflection, hence a doubling and the signal looks full size when scoped.

So, yes, matching the impedance of the line to the source termination impedance is critical for operation at 10MHz over several feet.


  I have some transitions on the trace that I cannot make 50 ohms.


What do you mean by transitions?  Cable couplers?  You're not splitting this signal off to more than one place I hope?  That will really not work, especially with source termination.

In the case that you're dealing with things that aren't perfectly 50 ohms (or whatever impedance cable you're using) you should first avoid them if at all possible.  Use couplers that are closely matched to 50ohm (they'll never be perfect) etc.

Any time the impedance changes in a transmission line there will be reflections -- the larger the change in impedance the larger the problem is and they can occur on both sides of the change (secondary reflections on the far side).  If the transitions in the signal patch are just the cable connectors on either end you can minimize their effect by placing the driving and receiving ICs as close as possible to them.  They usually do influence the overall TL impedance some, which is why you may have to tweak the source termination resistor a few ohms either way to get it perfect.

In response to the OP's comment:
There could be excess capacitance due to a faulty cable or connector.  

However more likely, you can not, ever, tap off a high frequency signal with a source termination.  You will get nasty reflections.  If you need to tap off a signal like this it needs to be end terminated PAST the final tap, and each tap needs to be as close as possible to the transmission line.

I assume that this interface board is tapping into the signal somehow. If not don't use it and just figure out how to plug the output of the ATE directly into your device board.  If you provide some models for these parts I can probably be of more help with this issue.