Capacitor discharging theory vs experiment doesnt match
You should check the datasheet to be sure, but the usual cause would be that ultracaps are often quite limited in terms of current they can provide. You're trying to draw about 100 mA, while most ultracaps are rated at 1-10 mA discharge current.

BTW, I suggest you stop your experiments if you need that cap later on. You can easily damage an ultracap by an out-of-spec charge/discharge current. The cap may fail open or lose some of its nominal capacity (maybe that has already happened, contributing to the short discharge time you're observing).

Update:

According to the datasheet your cap has a maximum Equivalent Series Resistance of 22 Ohm. When you connect a 47 Ohm load to it, it is allowed for the cap to drop up to 1/3 of its voltage internally. That's what you observe: the voltage quicly drops from 5V to about 3.3V and then keeps falling exponentially as you expected. In the long run, the discharge time constant should be around (R+ESR)*C.