Physically realistic received irradiance values given some BRDF?
I didn't get this part:


  That way each ray would just multiply its solid angle by its radiance
  value, and you'd get an "Irradiance" value of W/m^2.


I'll try to explain how the whole thing works though. You have a light source, it emits light rays. Those light rays bounce around the scene and eventually end up at your camera film. So at the end of the day you have multiple rays hitting the film surface and the radiance of those gets multiplied with the sensitivity function. For efficiency reasons you usually start backwards - from the camera. But if you work with symmetric BRDFs this should not matter. So what arrives at your film is radiance, you multiply that radiance with the sensitivity function and integrate within a "film pixel" to get the result in terms of your screen pixel.

Edit: To clarify what I meant by integrated out:

If you have radiance which is a measure of W/(sr * m^2), you can integrate out sr or m^2 like so:

$$E(x) = \int_{\Omega}{L(x,\omega)\cos\theta\,d\omega}$$

From where you get irradiance at a point $x$, which gives you the energy arriving from all directions at point $x$ (W/m^2). Now you can go further and find out the energy that arrives at some surface with area A, by integrating over all points on the surface:

$$\Phi = \int_{A}{E(x)\,d\mu(x)} = \int_{A}\int_{\Omega}{L(x,\omega)\cos\theta\,d\omega\,d\mu(x)}$$