How to get a breakdown of CPU usage at each point in time? I have a SQL Server RDS instance, running on a db.m5.xlarge (4 vCPU, 16GB RAM) with Multi AZ enabled.

I am trying to determine the cause of a constant 25%-30% CPU load on the instance at a time when we know that there is very little client activity hitting the database. We've taken into account our most expensive queries and feel that they are not the cause. Regular DB maintenance such as index and statistics rebuilds has been done. 

The image below is a graph on a CloudWatch dashboard, monitoring database connections and CPU usage. Is there a way I can query or visualise the breakdown of CPU usage at each data point in the graph? (I'm happy to dive in and run manual queries if that is what is suggested).
You can turn on Enhanced Monitoring and Performance Insights for more detail by modifying the RDS instance in the portal.

RDS > Databases > databasename > Modify

Also, on an RDS instance you can run Glenn Berry's DMV scripts and most of the queries will work. The stuff that breaks generally centers around drives, the errorlog, agent jobs, and the registry. Just download his scripts for the version of SQL you are running and search on CPU. You can start with the one titled - "Get CPU Utilization History for last 256 minutes (in one minute intervals)". 

https://glennsqlperformance.com/2020/02/04/sql-server-diagnostic-information-queries-for-february-2020/

Depending on the version of SQL your RDS instance is running (must be SQL 2016 or newer), you can turn on Query Store as well.