Visualize the Arduino PWM's output by means an oscilloscope
Yes, this is exactly how the equipment are supposed to be connected.
Earthed/grounded PC, earthed/grounded scope.
Safest for you and the probed equipment, there is no danger of damage.
Even better if both the PC and the scope are connected to same dual wall socket or same power strip, so it can be sure that they are grounded at the strip or wall socket, instead of long wires within the wall if they are on separate wall sockets. It also guarantees that they are powered from the same mains fuse and mains phase.