One LED has massive delay before turning on
So, I have this working now.

I tried a simple switch and it did not help
I swapped the lights and it didn't help

Finally I took down the two back lights and re-did the wiring.  The non-working light was chained from the other light so I took the caps and re-wired everything I could.  Once I put it back together it worked.  So I can only assume that there was a shaky connection for one the connections.  It is strange that it worked after a time after turning it on, but that does seem to be the issue.
Thanks for everyone's help.