Connecting a supercapacitor to a supply with voltage lesser than capacitor's rated voltage
Make sure you've let the capacitor soak for the time mentioned on the datasheet or application data. For example, AVX specifies the leakage as 10uA after 72 hours.