TS317 voltage regulator falling voltage
It sounds like the regulator may be going into thermal shutdown (or battery is sagging if small 9V PP3 type)
This is probably due to a short on the output. Are you sure R1 = 212 ohms?  Also are you sure you have the potentiometer connected correctly? (e.g. from top to bottom, not from wiper to top/bottom. If it is connected from wiper, is it turned the right way?) Did you test the resistance of both with your multimeter? 

As Leon says it would help if you post a schematic of your setup (whilst doing this you may notice a mistake) A picture would probably be very useful too (or instead)  

Set your multimeter to continuity test mode (or low ohms) and test for shorts between the output and other junctions of your circuit (e.g. output to ground, output to input, output to Vadj etc)
You can also confirm that the circuit is drawing too much current by setting multimter to Amps range and connceting it in series between the battery and circuit (e.g. red lead to battery plus, black lead to Vin of regulator. You will probably need to swap the red lead to the dedicated Amps socket)  

EDIT - From reading the comments you need to sort the resistive divider out to suitable values. 212k ohms and 4.7 ohms could cause strange things to happen. Also it sounds like your battery may be dead if you are measuring such low voltages (it may be due to a short though, so measure the battery voltage when unplugged from circuit) 

EDIT2 - I noticed you have the TS317 attached to a piece of aluminium. Be careful here as the metal tab on many TO-220 regulator packages is electrically connected to the output. Make sure nothing else that shouldn't be is touching the aluminium and causing the short. Easiest way to be sure is remove it from the aluminium, as you don't need a heatsink unless you are dissipating a reasonable amount of power (say > 1W) so with no load on the output it will be fine. You can add it again if necessary when you have figured out the problem.