What is reference voltage in a D/A converter
I think the first definition is closer to being correct, but it's not quite right. For an \$N\$-bit D/A converter, the maximum output voltage is produced when the digital input word is equal to \$2^N-1\$, which is the unsigned value with all bits equal to 1. So, the maximum output value is \$V_{max}=((2^N-1)/2^N) * V_{ref}\$. For large values of \$N\$, the maximum output voltage is very close to \$V_{ref}\$.

What your instructor called the "reference voltage" is the voltage equivalent of the LSB: \$V_{LSB}=(1/2^N) * V_{ref}\$.