Solar powered 5W DC12V pump
A dc to dc converter is going to be a low impedance load on your panel. Check the panel voltage when you connect the DC to DC converter with a load on it and see if it is dropping too low. 

A solar panel cannot supply infinite ammounts of current. When you draw too much current the voltage falls and falls fast. You may need a maximum peak power tracking circuit to regulate the voltage. An MPPT will also usually have an indicator to let you know if it can provide the current to the load.

A DC to DC converter is not going to find the optimum point of voltage and current for efficient power draw.

Lastly, the panel needs to have enough power from the sun. A good minimum figure to use is 300W/m^2 (thats watts per meter squared, at the equator it is 1000W/m^2). You also need to factor panel efficiency and your panel may have some info printed on a label on the back and the testing conditions or in a datasheet. Anyway you need to find the amount of power that your panel is receiving, so calculate the area then multiply it by 300W/m^2. This will give you a minimum ballpark number of how much energy the panel is receiving. The panel can't collect all of that energy, some of it is heat and the panel is not 100% efficient. So you need to take the total energy received and multiply it by the efficiency of the panel (which will probably be around 10 to 15%). I'll recap

300W/m^2*(area of the panel)*(the efficiency of the panel) = power received

That number better be higher than 5W, or your pump wont start. You'd also better try this on a sunny day pointing the panel's face normal to the direction of the sun (perpendicular). You will also lose some power to the inefficiency with any converter you use, but that will be in the DC converters datasheet and it varies with voltage.