An R function for checking whether a variable is binary or continuous
Using the return statement is not recommended.
You can get the same effect by rewriting with else if and else, like this:

checkBinaryTrait = function(v, naVal="NA") {
  if (!is.numeric(v)) stop("Only numeric vectors are accepted.")
  vSet = unique(v)
  if (!missing(naVal)) vSet[vSet == naVal] = NA
  vSet = vSet[!is.na(vSet)]

  if (any(as.integer(vSet) != vSet)) "con"
  else if (length(vSet) &gt; 2) "con"
  else "bin"
}


I also removed the unnecessary which calls.
This code still passes all your tests.

Actually the last statement can be further simplified to:

  if (any(as.integer(vSet) != vSet) || length(vSet) &gt; 2) "con"
  else "bin"


You might also want to change the return type to TRUE or FALSE,
in which case the last statement would become simply:

  !(any(as.integer(vSet) != vSet) || length(vSet) &gt; 2)


And then, how about renaming checkBinaryTrait to is.binary?

Finally, the &lt;- operator is more common than =.
For example Google's style guide explicitly forbids using =.