Is it possible to use a capacitor and a resistor to keep voltage above 3v and limit current to a certain value?
Right now, it sounds like the battery voltage and the load voltage are one and the same. You can put a resistor between the battery and the load, and that will help keep the BATTERY voltage higher. But it will make the load voltage lower. Capacitor can't really help.
I also want to point out that there is a fundamental contradiction in what you are trying to do. You are trying to restrict the current to the load AND boost the load voltage. Those two things are opposites. Anything you do to restrict the current will make the load voltage sag even more.
It is like you have an employee who is sad and wants a raise. Then you tell him that you are cutting his pay and that also he must be happy. You can't have it both ways.