Restoring SQL Server database with CDC
Ok I got the answer.

I used the following:

EXEC sys.sp_cdc_add_job
 @job_type = N'cleanup'
,@start_job = 0
,@retention = 52494800;


This adds the cleanup job but does not start it immediately i.e. start_job set to 0 does the magic. Retention can be set accordingly (100 years in my case as we do not want data to be deleted)