   [.net] NativeWindow weirdness
  Earlier today I was fighting with .NET once again, wondering why creating an OpenFileDialog whose parent was from another process wasn't actually working as expected.After checking out the situation in WinSpy++, it turns out that instead of the expected window being the owner some random hidden WinForm in my own process was being made parent. I asked myself.. "Why?"After hours of pillaging the OpenFileDialog source in Reflector I came to the conclusion that the problem must lie somewhere else. And lie it did.I was actually using NativeWindow.FromHandle() to get an IWin32Window to pass in the owner parameter of OpenFileDialog.ShowDialog(). What I didn't know was that this is where everything went haywire. Reflector to the rescue:private static int handleCount; public static NativeWindow FromHandle(IntPtr handle){    if ((handle != IntPtr.Zero) && (handleCount > 0))    {   return GetWindowFromTable(handle);    }    return null;} private static NativeWindow GetWindowFromTable(IntPtr handle){    uint num;    uint num2;    HandleBucket bucket;    HandleBucket[] hashBuckets = NativeWindow.hashBuckets;    int num3 = 0;    uint num4 = InitHash(handle, hashBuckets.Length, out num, out num2);    do    {   int index = (int) (num % hashBuckets.Length);   bucket = hashBuckets[index];   if (bucket.handle == IntPtr.Zero)   {  return null;   }   if ((((bucket.hash_coll & 0x7fffffff) == num4) && (handle == bucket.handle)) && bucket.window.IsAllocated)   {  return (NativeWindow) bucket.window.Target;   }   num += num2;    }    while ((bucket.hash_coll < 0) && (++num3 < hashBuckets.Length));    return null;}That's right. If there's more than one NativeWindow in a process it tries to look up the handle in a table of cached NativeWindow objects. If that fails it helpfully returns null. Because, you know, you couldn't conceivably want to create a NativeWindow from a handle that wasn't already created as a NativeWindow elsewhere in the same process.Yes, it's in the docs. Had I read them I probably would have noticed that, but unfortunately it was too late by the time I realized where the problem came from. Let this be a warning for anyone else who sees NativeWindow.FromHandle() in IntelliSense and thinks "ooh! I don't have to write my own IWin32Window helper!"   
  Here's one from OpenFileDialog.ShowDialog() that really screwed me today:public DialogResult ShowDialog(IWin32Window owner){    IntSecurity.SafeSubWindows.Demand();    if (!SystemInformation.UserInteractive)    {   throw new InvalidOperationException(SR.GetString("CantShowModalOnNonInteractive"));    }    NativeWindow window = null;    IntPtr zero = IntPtr.Zero;    DialogResult cancel = DialogResult.Cancel;    try    {   if (owner != null)   {  zero = Control.GetSafeHandle(owner);   }   if (zero == IntPtr.Zero)   {  zero = UnsafeNativeMethods.GetActiveWindow();   }   if (zero == IntPtr.Zero)   {  window = new NativeWindow();  window.CreateHandle(new CreateParams());  zero = window.Handle;   }   if (helpMsg == 0)   {  helpMsg = SafeNativeMethods.RegisterWindowMessage("commdlg_help");   }   NativeMethods.WndProc d = new NativeMethods.WndProc(this.OwnerWndProc);   this.hookedWndProc = Marshal.GetFunctionPointerForDelegate(d);   IntPtr userCookie = IntPtr.Zero;   try   {  this.defOwnerWndProc = UnsafeNativeMethods.SetWindowLong(new HandleRef(this, zero), -4, d);  if (Application.UseVisualStyles)  { userCookie = UnsafeNativeMethods.ThemingScope.Activate();  }  Application.BeginModalMessageLoop();  try  { cancel = this.RunDialog(zero) ? DialogResult.OK : DialogResult.Cancel;  }  finally  { Application.EndModalMessageLoop();  }  return cancel;   }   finally   {  IntPtr windowLong = UnsafeNativeMethods.GetWindowLong(new HandleRef(this, zero), -4);  if ((IntPtr.Zero != this.defOwnerWndProc) || (windowLong != this.hookedWndProc))  { UnsafeNativeMethods.SetWindowLong(new HandleRef(this, zero), -4, new HandleRef(this, this.defOwnerWndProc));  }  UnsafeNativeMethods.ThemingScope.Deactivate(userCookie);  this.defOwnerWndProc = IntPtr.Zero;  this.hookedWndProc = IntPtr.Zero;  GC.KeepAlive(d);   }    }    finally    {   if (window != null)   {  window.DestroyHandle();   }    }    return cancel;}It's subclassing the parent window's WndProc for its own nefarious purposes. In my case that didn't work so well, because the parent window I was passing in lived in another process. Normally creating dialogs with parents in other processes is all well and good, but subclassing them is not. Windows will try to call the address you're passing in in that process's address space, so you can imagine the results.The WndProc it installs:[SecurityPermission(SecurityAction.LinkDemand, Flags=SecurityPermissionFlag.UnmanagedCode), SecurityPermission(SecurityAction.InheritanceDemand, Flags=SecurityPermissionFlag.UnmanagedCode)]protected virtual IntPtr OwnerWndProc(IntPtr hWnd, int msg, IntPtr wparam, IntPtr lparam){    if (msg != helpMsg)    {   return UnsafeNativeMethods.CallWindowProc(this.defOwnerWndProc, hWnd, msg, wparam, lparam);    }    if (NativeWindow.WndProcShouldBeDebuggable)    {   this.OnHelpRequest(EventArgs.Empty);    }    else    {   try   {  this.OnHelpRequest(EventArgs.Empty);   }   catch (Exception exception)   {  Application.OnThreadException(exception);   }    }    return IntPtr.Zero;}Looks like I'm going to be rewriting some more Framework classes myself.   ;  Just a naive question, would it work to create your own window whose parent is the out-of-process window, and then have OpenFileDialog use your window as the parent?And yes, I agree, you really do have to fight .NET Winforms sometimes.   ;  Probably. In fact, is looks as if Form.ShowDialog() doesn't try to subclass its parent window when shown modally, so that should actually work fine. On the other hand it requires extra overhead that's probably pointless when I can just go and P/Invoke GetOpenFileName().   