launchctl starts my plist job much later than StartCalendarInterval
Since your per-user launch agent does actually execute, albeit later than scheduled, it is highly likely that your system may be asleep at the scheduled time. I would recommend reading through the man page by running the following command from a terminal session:

$ man 5 launchd.plist


Regarding the StartInterval option specifically, the man page has this to say:


  Unlike cron which skips job invocations when the computer is asleep,
  launchd will start the job the next time the computer wakes up.  If multiple intervals transpire before the computer is woken, those events will
  be coalesced into one event upon wake from sleep.


If you are working remotely, or not physically present at the scheduled time — which is not clear from your question — a good starting point would be to confirm that the system has not entered sleep mode prior to the scheduled time of your launch agent.