Is it appropriate to ask a job candidate if we can record their interview?
There may be more issues than you raise here. Some people will say yes and feel intimidated. Some people will say no out of general principles. Some people will say yes and regret it later. Some people will say yes initially but decide otherwise in the middle of the interview. 

I suggest that before you implement such a process you game it out thoroughly, developing a lot of what-if scenarios and how you will respond to them. I think an essential element, possibly with legal ramifications (though I don't know), is that you don't disadvantage anyone for giving either answer or for declining to give a reason. 

Another possible issue is that some candidates may not be as candid as they would otherwise if they are being recorded. It isn't a case of being devious or calculating, just being cautious. Can my words come back to haunt me? 

You will also need to decide what to do with the tapes and when to delete them and how to assure the candidate that you will do so, especially if requested. In particular, who will have access to the tapes and for how long?  

Finally, if you develop a policy with a lot of nuances, you should publish it, probably online, and let the candidate have access to it prior to an interview.

But, overall, I'd suggest that in the case you mention of not everyone being available, that you make it possible for a follow up interview rather than taping. There are probably other issues here that I haven't considered.