Properly tensioning a belt drive with two tensioner
To try and get the wheel perfectly straight you could try to fully loosen each nut on each tightener. Then put each nut back on its respective tensioner and whilst tightening count the number of turn on each side. 

Make sure there are an equal number of turns on both sides. I usually do this by tightening one side first whilst counting number of turns, then tighten the other side. Make sure the nuts/quick release with which you tighten the wheel are not loosened too much, if they are the chain tension might increase quite significantly when tightening the wheel nuts/ the quick release (because in some cases the frame bends inwards resulting in a somewhat different chain stay angle which causes the extra tension). 

Usually if the chain (not sure if this is the same for belt drive but it should be ) tension is too high you will feel the 'steps' in the wheel /freewheel/casette bearing when turning the pedals by hand (you will feel little bumps every so often whilst rotating the pedals by hand. This could also indicate somewhat worn bearings but chain tension definitely plays a role in it as well. 

Another sign of the tension being too high:
Spin the pedals to make the rear wheel rotate (whilst the wheel is off the ground), release pedals and see if they stop turning immediately or if they continue to rotate with the back wheel (I am assuming you have a freewheel and are not using a fixed gear), if they do keep turning this can be caused by chain tension being too high (or bad bearings),  this could potentially also indicate misalignment of belt relative to rear cog (wheel not aligned with frame) so if your bearings are in good condition and chain tension is good the wheel might be misaligned.

Try loosening the belt a bit and see if it helps (if aligning the wheel didn't help, as long as the belt doesn't slip/run off the cogsyou shouldn't need you could also measure the 'stickout' of the threads of each tensioner using a caliperor other measuring tool and compare both sides and make sure they are (approximately) equal to try and align the wheel.