Built in cache system doesn't work in Drupal 6
Drupal generates cache when a node, block, or view is rendered the first time. How much cache is served versus how much is generated on that request can depend on both that visitor's role and specific user account.

If two anonymous users hit the same page, the first anonymous user will trigger the cache to be generated for elements on that page, then the second anonymous user will receive those cached elements and the page will load faster for them.

However, if one authenticated user hits a page, then an anonymous user hits the same page, two different versions of a selection of elements on that page will be cached (for instance, if authenticated users are allowed to see a block, but anonymous users aren't).

Therefore, Drupal's caching mechanism works best when there are lots of visitors with the same permissions hitting the same non-personalized pages. If it's a site with users who sign in, and there are lots of permissions (especially field-level permissions) at play, Drupal's cache will be fairly ineffective for anything but static content nodes and blocks.