How to measure Oscilloscope's self random jitter
Using a square wave signal will not get your desired results unless you can control the square wave very precisely, which is not easy. An easy way to measure noise in such a system is to use a sine source, with as little noise as possible (but not overly so), use a power splitter and connect it to two inputs of the scope. Then sample the inputs and store the raw samples. Using cross correlation, you can eliminate the noise of the signal source you have used and get the noise of each channel. But be aware that there are certain noise sources within the scope that are equal on all channels and thus will cancel out when you do cross correlation. An important one of these noise sources is the noise of the sampling clock, which is correlated between channels.

If you want to measure the sampling clock jitter, you have to start with a known good signal, which has a noise that is lower than the jitter you expect from the scope. As I guess you want to measure in the GHz range, this will be most likely be a special DRO or if you cannot find a DRO with low enough noise a CSO. Unfortunately, a CSO is prohibitively expensive to buy, unless you are at a university that owns one. Alternatively you can ask your national metrology institute whether you can stop buy for some measurements.