How to implement a custom ray tracing algorithm in Blender? I want to implement a custom raytracing algorithm in Blender using nodes or preferably via python scripting. 

Can someone please point me to the right direction? A tutorial will be nice.

Edit: Mainly, I want to have a rendering pipeline like BlenSor but through without relying on the native backend code. OSL does not seem to be the solution for me because I want to simulate the real optics behind it.
A very broad question. I know there is functionality in OSL for this but OSL equals Cycles. So if it should be possible in OSL it is possible with nodes. OSL does not have that much of references. The people who explored this field are Blender Sushi and the blog Small Blenderthings.

You do not exactly state what you are after so I can not help you that much further.