How to identify if noise is coming from circuit or scope?
Excessive ground lead inductance usually shows up as ringing and overshoot in square(ish) waveforms. Here's an example:



This is from an article by Teledyne LeCroy: Passive Probe Ground Lead Effects. Here, the upper yellow trace shows the output of a signal generator as measured by the scope where the probe has been stuck into a coaxial connector, thus keeping the inductance of the ground lead to a minimum. You can see there is minimal overshoot and ringing. The corners are a bit rounded due to the limited frequency response of the probe (500 MHz in their example) or the signal generator.

The red trace shows the result of measuring the same signal with the typical alligator clip lead. Note the excessive overshoot at the edges and the ringing.

Another issue is with an excessively long ground lead is that this stray inductance can mutually couple with nearby magnetic fields, or act as an antenna for RFI. This shows up as noise depending on with what it is coupling. Usually it's some high current, high di/dt thing elsewhere in the circuit, like motor drivers and switching converters tend to have.

To reduce these issues, we must reduce the inductance of the ground lead. Since inductance is proportional to the area encircled by a circuit, the shorter and closer to the probe tip we can make the ground connection, the better.

Some probes have a ground spring. Again, from Teledyne LeCroy:



If your probes have no such accessory, then you can make your own with a bit of solid wire.