Can there be an algorithm for a relation which is not a function?
Your question is very general, especially it is not clear to me what you mean by "algorithm for a relation". I still have some answers.

The difference between (binary) relations and functions is that the former can have several different outputs for the same input. This is just what non-determinism offers us. Thus the theory of computation offers us many models of computation thatcan be seen as computing relations instead of functions.

Further, the input in the most common models of computations has the form of a string. Thus you could compute a relation in the way that string "(x,y)" is accepted iff (x,y) forms part of the relation. This, of course, generalizes to relations of higher order.