Multimeter shows different voltages for positive and negative voltage(interchanging multimeter probes) for same source
It would be reasonable to expect a DMM to read the same to within one count, maybe two, when reversing the leads. There are factors like linearity, and contact potentials with the leads, that could give rise to not exactly the same reading without cause for concern.

The difference you report is too big to be ignored. It's possible your meter is faulty. Measure a battery both ways round to check. Use another meter (borrow one, but at £3, always keep a second meter around as a spare) to check. Is the battery in the meter OK? It may say it's OK, measure it to check (with your 2nd meter!)

It's also possible that your power supply is oscillating, and the different connection of your meter is, through placement of the leads affecting the loading capacitance, affecting how or whether it oscillates. Try changing the length/placement of the leads to investigate this possibility. Any unexpected results are worth investigating.