Dependency injection thread safety
Your MyCacheProvider will hold an instance of JilSerializer and since your MyCacheProvider is a singleton - there's going to be an instance of JilSerializer around forever as well.

As it's apparently fine for your ICacheProvider to use the same serializer throughout the lifetime of the app, and the implementation looks completely fine for sharing an instance, I don't see any reason why you can't just make ISerializer a singleton too.

I think that this isn't quite a captive dependency as the default lifetime of ISerializer is transient but it's certainly a related topic that you can read about on Mark Seemann's blog.