Restore Verify Only option vs Manual Restore
Is this enough to be confident or should we still be manually restoring these backups to a test server and running tests?


To be 100% confident, a restore of database is required. You need a seperate server to do the restores. Since you are using Ols's backup solution, the entire restore and verify process can be automated using dbatools - Restore-DbaDatabase esp specifying parameter MaintenanceSolutionBackup

e.g. Below powershell script Scans all the backup files in \\server2\backups$ stored in an Ola Hallengreen style folder structure,
filters them and restores the database to the c:\restores folder on server1\instance1


  Restore-DbaDatabase -SqlServer server1\instance1 -Path \server2\backups\$ -MaintenanceSolutionBackup -DestinationDataDirectory c:\restores\


Also refer to my answer : Understanding the impact/risk of turning off “verify backup integrity” on SQL backup