Indirect Lighting is very slow proportional to vertex count even if they are not in the view I have this library scene which is full of instanced group of the same model. The vert count says only 58, but when actually rendering it shoots to 1.7M due to all the books. However, the rendering is fast because Blender seems to exclude unseen objects.

However, when I turn on Indirect Lighting because I want it's effect, now the rendering time is exponentially higher. It seems to be longer proportional to vert count, even occluded ones.

Hiding some model is not an option since the camera will move around and eventually will reach the hidden part that I set. (Imagine a zooming all the way on this corridor)

Is there any solution to make indirect lighting calculate only the relevant model that matters in the current frame automatically, like the normal rendering without indirect lighting?

In this scene I even have a mist, so technically I can exclude everything further than the visible range. I thought things like setting camera's far clipping plane can help excluding things for Blender to consider but that is not the case. Rendering time is the same even with far plane as low as 100.
You are making wrong assumptions, something not being directly visible doesn't mean it is not contributing to the rendered pixels, in fact indirect light means exactly the opposite.

The walls out of view are still castings shadows in the visible ones, and the window behind the mist still lets light in even when you look away.

That is true for many direct light situations, even more so  with indirect light, all elements influence scene lighting everywhere, unless behind a watertight boundary.

Also Indirect light already calculates only relevant data as best as possible.

Some optimizations are done internally but it is not trivial to determine what contributes to a scene and what doesn't, more geometry will invariably make a render slower eventually.

If you want faster renders manually optimize your scene, reduce polygon counts, hide unnecessary stuff, simplify lightning, use instanced geometry wherever possible, reduce object count etc.

Also match your expectations to your hardware. If necessary bake lighting into textures to make animations faster.