Complex structure vs object
The CHI uses many caching backends and you did not said which one you are using. Fastest is perl's internal arrays, then hashes and then any other implementations. CHI may use persistent files to caching, like Berkeley DB backend or one of mmap implementation to reduce memory usage. When you are using such caching to improve perfomance you may place cache files to tmpfs (or some of ramfs) on *nix systems (linux, *bsd).

So, when I'm storing a lot of data in cache and I need to reduce memory usage I prefer use BDB without CHI. Then I need speed and managing few data in cache I prefer perl's internal structures. In last case you need to remember that Perl uses optimistic memory allocation. Thats means that Perl reallocates and allocates new memory segments and never returns consumed memory back to the OS. It is very important when writing daemons and you need to limit somehow memory usage. In other cases you may use any caching system and do not worry about performance or memory usage. Just use Devel::NYTProf to find bottlenecks.