Creating a common ground
You are seeing exactly the behaviour I would expect.

When turned on, Silicon BJT transistors need a \$V_{be}\$ (that's the voltage of the base relative to the emitter) of around \$\approx0.7\mathrm{V}\$. That means that if your Arduino output is at \$5\mathrm{V}\$, then if the emitter is higher than \$5-0.7=4.3\mathrm{V}\$, the transistor will be turned off.

If you put the motor between emitter and ground, it means you cannot have more than \$4.3\mathrm{V}\$ across the motor, because the transistor would turn off. However you will get enough current flowing to get the motor voltage up to about that. If you don't have a base resistor, the current flowing into the base in this scenario will be quite high, and will far exceed the current sourcing capability of the ATMega IC which will cause the output voltage to drop due to internal resistance, which is why you see closer to \$3\mathrm{V}\$ across your motor.

If instead you connect the emitter to ground, and the motor to the collector (and the other side of the motor to the power supply), then when you apply \$5\mathrm{V}\$ to the base, you would have \$V_{be}=5\mathrm{V}\$ because the emitter is at ground potential. This would turn on the transistor fully and you would have close to \$12\mathrm{V}\$ across the motor. However this is not good either - it would either fry your transistor or fry the control pin (or both).

BJT devices are current controlled current sources - this means that the current flowing from collector to emitter is proportional to the current flowing from base to emitter. That is to say, increasing the base voltage is not how you control the output, but rather you need to change the current. If you increase the base voltage too high on the transistor you end up with very high currents flowing into the base (the transistor it is effectively a diode from base to emitter) which will damage it.

So what you need to do is convert the \$5\mathrm{V}\$ voltage output of the Arduino pin to the \$\approx0.7\mathrm{V}\$ required by the transistor at a safe current level - this is done by simply adding a suitably sized resistor between the Arduino pin and the base of the transistor like so:


Image Source