How to combine 2 Solar panels for double A
It would be best if you tested the solar cells outside. Testing with a lamp doesn't provide the same intensity of light as the sun in terms of wavelength or lumens. 

Your voltage is low probably because of the same reason. You're right about connecting them in parallel to increase the current, you just need better environmental conditions.

Good luck.