How to isolate two ground signals?
First, the arduino's I2C bus is 5V. You have no guarantees that it'll work with a 3.3V device. So you were already living on the edge before adding the motor. Best case scenario, the MPU6050's 3.3V are not considered high by the arduino. Worst case, the arduino's 5V fries the MPU6050. You should use some kind of logic shifter or, at the very least, add a ~330 ohms resistor in series with the I2C bus to guarantee that when the arduino outputs 5V, the MPU6050's internal protection diodes survive (don't quote me on that though, the MPU6050's datasheet says nothing about this. For all I know, it could go singularity when you show VDD+0.5V to its I2C pins)

Now, assuming you don't care about that, you just want the motor to not influence the rest of the circuit: the reason you need to connect the grounds is because when the arduino sends a 5V signal, the driver has to understand that it's a 5V signal. It does so by comparing the signal's voltage to its own reference (GND pin). If the grounds are disconnected, the voltage the driver "sees" will be random, since the driver's ground could be, say, 2V above the arduino's ground, completely messing up your logic. The problem is that your motor is probably making a lot of current flow through this ground path, making some voltage drops appear on it.

Voltage drops in the ground path happen in every circuit. The problem is when two components are attached to the ground path in different points, causing them to "see" different grounds, and so they disagree on the logic levels. This is probably your problem: the inertial unit and the arduino see very different grounds when the motor produces a current pulse, and that's enough to make the logic mess up.

You could: (in order from less work to more work)


Add a big capacitor between the driver's +12V and GND terminals to supress pulses.
Assuming Add the 330R resistors I mentioned earlier: they should make the '1' logic level signal more robust (because of the arduino's I2C pull-up. See this). Also, safer. I'm assuming a pull-up resistor on the MPU6050's board.
Add a resistor between grounds so that the rise in one of them does not affect the other so much. Problem is, the driver uses the other ground, so it could stop understanding the arduino's control signals.
Connect the battery's GND to the USB's GND, so both the arduino and the MPU are not so far apart in the ground path. Bad dangerous idea, you'll fry your USB, don't do it at all.
Use the damn logic shift circuit.
Use an optoisolated driver. This way, you don't need to connect the grounds.