How to disable scrolling while clicking with the Magic Mouse?
Since we've established that your Magic Mouse doesn't have this behaviour with another computer, I'd reset your NVRAM and also do a boot into Safe Mode.

Resetting the NVRAM

Older Macs had what's called Parameter RAM (PRAM), newer Macs use Non-Volatile Random-Access Memory (NVRAM). 

Since you haven't specified what model of Mac you're using, below are some generic steps that will work with all models (including all Late-2016 MacBook Pro models):


Shut down your machine. Yes, a full shut down, not just logging out.
Press the power button and then immediately press the commandoptionPR keys. 
Keep these keys pressed down for at least 20 seconds!
Let go of the keys and let your Mac reboot normally.


Now test your Magic Mouse to determine if the issue is still present. If this doesn't resolve it for you, proceed to booting into Safe Mode.

Note 1: When you log back in you may need to readjust some of your system preferences (e.g. speaker volume, screen resolution, startup disk selection, time zone information, etc).

Boot into Safe Mode

Follow these steps to boot your Mac into Safe Mode:


Fully shut down your Mac
Restart your Mac
Immediately press the Shift key and keep it down 
Let go of the Shift key when you see the login window (NOTE: If you have FileVault enabled you may need to log in twice).
Take a note of what happens (i.e. test your Magic Mouse)
Now exit Safe Mode by restarting your Mac as normal
Again, take a note of what happens (i.e. test your Magic Mouse)


Once you've booted into Safe Mode, let me know how you went and we'll go from there.

Trash Magic Mouse .plist files

Since your issue persists both in and out of Safe Mode, I would trash the relevant Magic Mouse preference files.

As you don't specify what version of macOS you're using, I'll assume you're running a version from Mountain Lion and above. Follow these steps:


Go to Finder
Press and hold the option key
While keeping the option key down, click on the Go menu and select Library from the list
This should open up your user's Library folder in a new Finder window
Locate and open the Preferences folder (it's within the Library folder)
Within this folder find the following two .plist files:


com.apple.AppleMultitouchMouse.plist
com.apple.driver.AppleBluetoothMultitouch.mouse.plist

Now Trash these files
Restart your Mac


Your Mac will now create a new set of default preference files for your Magic Mouse. 

Note: You will need to go back to System Preferences and reconfigure your Magic Mouse settings. However, before doing that I would test the Magic Mouse first to see if the problem behaviour still persists.

Reset Bluetooth Device List and Bluetooth Controller

This is something I've had a number of users do to resolve issues with 3rd party Bluetooth devices (usually a keyboard and/or mouse, but also headphones etc).

Since you're using a genuine Apple Magic Mouse you my decide not to bother, but I think it's worth trying.

In a nutshell you need to reset both the macOS Bluetooth Device List and Bluetooth Controller. However, this will reset all connected Bluetooth devices, including your keyboard and mouse, so each Bluetooth device will need to be paired again.

I have some steps you can follow, but I strongly suggest you read all of the steps first before commencing (or perhaps print out the instructions / view them on another device).

Before starting, ensure you have the Bluetooth icon showing in the menu bar. If it is not there, open System Preferences > Bluetooth and enable the option Show Bluetooth in Menu Bar.

Also, since you're trying to resolve an issue with your Magic Mouse,  you will need to use another mouse to follow these steps:


Hold down both the Shift and Option keys and at the same time click on the Bluetooth icon in the menu bar. Once the menu is showing, release the keys.    
From the Bluetooth menu, choose Debug > Remove All Devices
Hold down both the Shift and Option keys again and at the same time click on the Bluetooth icon in the menu bar. 
From the Bluetooth menu,  choose Debug > Reset the Bluetooth Module
Shutdown your Mac
Ensure your Magic Mouse is fully powered off
Restart your Mac
On your Mac, click on the Bluetooth icon in the menu bar 
Select the Set Up Bluetooth Device option
On the mouse, turn the power on
Once the Setup Assistant finds the mouse, select it from the list and click on “Continue”. 
Now pair your mouse
If you have any other Bluetooth devices, you can go ahead and pair them now (or you can do it later). 


Let me know how you go.