SQL Server Setup: Report Server and RAM Considerations
Report Server executes the query and stores a copy of it within its cache. This is outside of the SQL Server process and so you will need to account for it when architecting your installation. Generally, the biggest winner I have noticed when I have done installations myself is the number of CPU's. RAM helps of course, but I have found additional CPU's to have the biggest gains - no point having 1TB of RAM if your request is stuck in a queue on a 2 core machine!

There is a very good (but lengthy) white-paper on MS-Docs called Planning for Scalability and Performance with Reporting Services which outlines the difference between different hardware setups, scaling and provides some very good advice on how to architect a SSRS solution. I would highly recommend that you check it out.