Symfony\Component\HttpFoundation\Response is never cached
Return a cacheable response. For example
use Drupal\Core\Cache\CacheableResponse;
use Drupal\user\Entity\User;

  /**
   * A route returning a CacheableResponse object.
   *
   * @return \Drupal\Core\Cache\CacheableResponseInterface
   *   A CacheableResponseInterface object.
   */
  public function cacheableResponse() {
    $user = User::load(1);
    $response = new CacheableResponse($user-&gt;label());
    $response-&gt;addCacheableDependency($user);
    return $response;
  }

From Drupal\dynamic_page_cache_test\DynamicPageCacheTestController.
There are multiple ways to add cache data to the response object additional to the one used in the example.
You can use addCacheTags()/addCacheContexts():
$response-&gt;getCacheableMetadata()-&gt;addCacheTags($tags);
$response-&gt;getCacheableMetadata()-&gt;addCacheContexts(['route']);

The route cache context is by the way not necessary in a controller because it depends already on the route.
Or add cacheable metadata contained in a render array
use Drupal\Core\Cache\CacheableMetadata;

$response-&gt;addCacheableDependency(CacheableMetadata::createFromRenderArray($build));

See also How to handle cache metadata in a cacheable response? and How to get cache metadata from nested render array when returning response?