Is it safe to fly if the flaps won't retract on ground?
The main thing to be concerned with, is that the spring tension is important for keeping the bicycle chain segment secure on the eccentric sprocket (item 17 below) that drives the torque tube.  If there are issues with spring tension in the return spring (item 32), I'd be concerned that slack could result in the chain potentially coming off, but that would require the tension spring on the bottom (item 22) to be not doing its job.
The top return spring should be a fair bit stronger than the tension spring, which is only there to keep the chain from going slack when the cable from the cockpit lever is slack.  There is likely a combination of a weakened return spring and some binding in the bushing blocks for the torque tube.
There isn't any risk of flap asymmetry unless the torque tube itself breaks or a control rod lets go, so a problem with the chain arrangement where it jams up or disconnects will, in a worst case scenario, just leave you stuck with flaps partially extended or at zero.
So what to do about it?  Depends.
I'm assuming they went almost fully up and you had to push them a bit to get them to move the last little bit.  If I knew the plane well, and if the panels came up almost all of the way and just needed a little nudge to overcome some friction, and they stayed up after the nudge, and every thing otherwise felt smooth and solid (when you do a walkaround, you shouldn't just be looking at stuff; you should be gently tugging and moving bits as part of your inspection to detect unusual play), it should be ok and air loads will do the job fine at retracting them as soon as you are moving with some speed, although I would snag it and expect it to be fixed the next time I saw it.
On the other hand, technically it shouldn't be doing that and if there is significant friction, or slop, or you are just uncomfortable with it, you should refuse it and shouldn't be charged for it.  A weak return spring and/or binding in the mechanism that prevents retraction on the ground is still a &quot;defect&quot; and needs to be rectified.
If you have access to the A&amp;P that maintains the airplane it might be a good idea to mention it.  They may not be aware of the problem.