Dialog I cannot get rid of
Search for OS X identify process by window I found What process owns a certain window (Mac OS X) which told me this after moving the Window:

RetinaMBPro1TB:bin jeroenp$ ./show-recently-moved-windows.py 
Move target window

List of windows that moved:
{(
        {
        kCGWindowAlpha = 1;
        kCGWindowBounds =         {
            Height = 1094;
            Width = 1130;
            X = 43;
            Y = 45;
        };
        kCGWindowIsOnscreen = 1;
        kCGWindowLayer = 0;
        kCGWindowMemoryUsage = 5246548;
        kCGWindowName = "bin \U2014 Python \U2014 160\U00d776";
        kCGWindowNumber = 23921;
        kCGWindowOwnerName = Terminal;
        kCGWindowOwnerPID = 357;
        kCGWindowSharingState = 1;
        kCGWindowStoreType = 2;
    },
        {
        kCGWindowAlpha = 1;
        kCGWindowBounds =         {
            Height = 153;
            Width = 420;
            X = 1359;
            Y = 182;
        };
        kCGWindowIsOnscreen = 1;
        kCGWindowLayer = 8;
        kCGWindowMemoryUsage = 390292;
        kCGWindowName = "";
        kCGWindowNumber = 23246;
        kCGWindowOwnerName = CoreServicesUIAgent;
        kCGWindowOwnerPID = 87976;
        kCGWindowSharingState = 1;
        kCGWindowStoreType = 2;
    }
)}


Since CoreServicesUIAgent is the owner, I performed this:

killall -KILL CoreServicesUIAgent


That worked. Now I can open the PDF again.