   Problem in VMR9 Renderless Mode loading mixer control
  Hi, I've a strange problem...I have an application that simply takes video input from a DV Camcorder, put it on a surface via custom allocator/presenter, and then present it on a textured object. All works well, until I've tried to set deinterlacing...As I read on SDK help and googlin', I've noticed that Renderless mode doesen't load the mixer component necessary to do deinterlacing (in fact "GetNumberOfDeinterlaceModes" returns an error that tells me so). So I've tried the "SetNumberOfStreams(1)" over the IVMRFilterConfig9 interface, to force the VMR to load the mixer, but doing this the allocator/presenter ceased to work. It goes about 50 times through "InitializeDevice" method, and then it never calls the "PresentImage".... If I remove the call to "SetNumeberOfStreams" all returns to work as aspected...In the end I need to know a way to deinterlace the video...Many thanks in advance,Juri Capovilla   
  I've not had much experience with VMR, but if you don't find a proper solution you could work around the problem by doing the deinterlace yourself. A vertical blur should provide reasonably good results - try weights of (0.25, 0.5, 0.25). Even better only do that blur where the difference between the two fields is significant. That blur should be fairly simple to do with a pixel shader.   ;Quote:Original post by Adam_42I've not had much experience with VMR, but if you don't find a proper solution you could work around the problem by doing the deinterlace yourself. A vertical blur should provide reasonably good results - try weights of (0.25, 0.5, 0.25). Even better only do that blur where the difference between the two fields is significant. That blur should be fairly simple to do with a pixel shader.Unfortunatly I don't think this could be a solution for my problem, because I need to record high  speed videos (eg. a golf swing), and then see them frame by frame... If I do the way you said, I think I'll see the club "melted" in two different positions...;  Here is a sample of what I mean: "Blurring" the image would produce figure 1 to melt with 2, isn't right?   ;  I'm afraid that image you posted is almost impossible to restore to a static golf club, even if an expert processed the frame manually. I'll try and explain why. I had a quick go at editing it myself but I realized that image isn't at the original resolution, which makes it much more difficult to do anything with it.Firstly cameras will always tend to introduce an element of motion blur into shots with fast motion. As the frames taken by the camera are blurred, it's almost impossible to unblur them completely. That motion blur can however be reduced with bright lighting and a wider apature as that will improve the shutter speed. Increasing the films light sensitivity will also help for the same reason.Some video cameras will deliberately choose to allow significant motion blur in the image in order to give a better sense of motion at low frame rates.Let me try to explain interlacing, and why deinterlacing is probably not what you want here. The way interlacing works is that the camera takes 60 images a second, and combines them to 30 interlaced frames per second. The way it combines them is that one image will be the even numbered lines of the frame, and the other the odd numbered lines. You can see that in your sample image. The golf club appears to be in two places on the one frame, because it's actually a combination of two frames.A standard TV will update the even and odd lines of it's display alternately to give a good reproduction of the original. It's basically a reduction of vertical resolution to gain frame rate.Deinterlacing attempts to hide the problems you get with fast moving objects by blurring the image to produce a half speed version of the original. It works well when the movement is fairly small, but it just looks blurred when the movement is quick. That looks decent when you want to create an impression of fast motion, but it'll just look blurred when viewing a static frame of something that's moving quickly.Assuming you want static frames rather than smooth motion, your best option is to separate those two original images back out again which will double the frame rate. All you need to know is which order the even and odd lines are in (sometimes it's odd first, sometimes it's even). You will then end up with a half height but double frame rate movie that should be exactly what the camera saw.   ;Quote:[...]Assuming you want static frames rather than smooth motion, your best option is to separate those two original images back out again which will double the frame rate. All you need to know is which order the even and odd lines are in (sometimes it's odd first, sometimes it's even). You will then end up with a half height but double frame rate movie that should be exactly what the camera saw.Many thanks Adam!For separating the two images you mean that I must do it manually on the surface?Or could I tell the allocator to take only the odd/even lines?;  I've no idea of what's the best way to take only the even/odd lines... Anyone has any clue?   ;  I don't think it's the best way to go about it. Even if you did take all even/odd frames you will still get vertical lines as they are supposed to interleve each other to create a full frame anyway.Here are some things to check..- Do you use GetMixingPrefs() then turn off RenderTargetMask and RenderTargetYUV on just after you SetNumberOfStreams()?Looks like this// Get the current mixing preferences.DWORD dwPrefs;pMixControl->GetMixingPrefs(&dwPrefs);  // Remove the current render target flag.dwPrefs &= ~MixerPref_RenderTargetMask; // Add the render target flag that we want.dwPrefs |= MixerPref_RenderTargetYUV;// Set the new flags.pMixControl->SetMixingPrefs(dwPrefs);Are you dealing with YUV color space in your allocator/presenter (usually requires an offscreen surface to convert pixel format in PresentImage from YUV to RGB using StretchRectangle)? Does your video card even support YUV color space and VMR9 hardware accelerated video? You need mixer mode to deinterlace and maybe your video card doesn't support it.   ;Quote:Original post by HeadkazeI don't think it's the best way to go about it. Even if you did take all even/odd frames you will still get vertical lines as they are supposed to interleve each other to create a full frame anyway.Here are some things to check..- Do you use GetMixingPrefs() then turn off RenderTargetMask and RenderTargetYUV on just after you SetNumberOfStreams()?Looks like this// Get the current mixing preferences.DWORD dwPrefs;pMixControl->GetMixingPrefs(&dwPrefs);  // Remove the current render target flag.dwPrefs &= ~MixerPref_RenderTargetMask; // Add the render target flag that we want.dwPrefs |= MixerPref_RenderTargetYUV;// Set the new flags.pMixControl->SetMixingPrefs(dwPrefs);Are you dealing with YUV color space in your allocator/presenter (usually requires an offscreen surface to convert pixel format in PresentImage from YUV to RGB using StretchRectangle)? Does your video card even support YUV color space and VMR9 hardware accelerated video? You need mixer mode to deinterlace and maybe your video card doesn't support it.I've tried the code above just after the call to SetNumberOfStreams, but nothing changed. Regarding the Video Card, it's pretty new so I don't think is this the problem (nVidia 7600GT)Here's my PresentImage:STDMETHODIMP cAllocator::PresentImage(    DWORD_PTR dwUserID,VMR9PresentationInfo* pPresInfo){    CLock lock(&m_CritSec);// Find the VideoSource by ID.    VideoSource* pSurf = GetVideoSource(dwUserID);    if (pSurf)    {   CComPtr<IDirect3DSurface9> pTexSurf;   pSurf->GetPrivateTexture()->GetSurfaceLevel(0, &pTexSurf);   m_pDevice->StretchRect(pPresInfo->lpSurf,    NULL, pTexSurf, NULL, D3DTEXF_NONE );    }    return S_OK;}And here is where I create the surfaces...HRESULT VideoSource::CreateBuffers(    VMR9AllocationInfo* pAllocInfo, DWORD dwNumBuffers){// Dimensiono ed alloco un array per i puntatori alle superfici    ReleaseSurfaces();    m_pSurf = new IDirect3DSurface9* [dwNumBuffers];    if (m_pSurf == NULL)    {   return E_OUTOFMEMORY;    }    HRESULT hr;    // Create offscreen surfaces for the VMR.    pAllocInfo->dwFlags = VMR9AllocFlag_OffscreenSurface;//pAllocInfo->Format = D3DFMT_UNKNOWN;    hr = m_pNotify->AllocateSurfaceHelper(pAllocInfo, &dwNumBuffers, m_pSurf);    if (FAILED(hr))    {   OutputDebugString("Could not allocate the private surfaces\n");OutputDebugAllocInfo(*pAllocInfo);   return hr;    }    m_dwNumSurfaces = dwNumBuffers;// Creo la texture da usare nell'applicazione. DEVE essere come RendeTarget    AdjustTextureSize(pAllocInfo);    pAllocInfo->dwFlags = VMR9AllocFlag_TextureSurface |  VMR9AllocFlag_3DRenderTarget;    dwNumBuffers = 1;    CComPtr<IDirect3DSurface9> pTexSurf;    hr = m_pNotify->AllocateSurfaceHelper(pAllocInfo, &dwNumBuffers, &pTexSurf);    if (FAILED(hr))    {   OutputDebugString("Could not allocate the texture\n");OutputDebugAllocInfo(*pAllocInfo);   return hr;    }    m_pTexture.Release();    pTexSurf->GetContainer(IID_IDirect3DTexture9, (void**)&m_pTexture);OutputDebugString("Init Texture/Surface VMR OK!\n");OutputDebugAllocInfo(*pAllocInfo);    return hr;}and here are the debug results of the function above:Could not allocate the texturedwflags: 5dwWidth: 720dwHeight: 576Format: 844715353Pool: 0MinBuffers: 1szAspectRatio: 4szNativeSize: 3Could not allocate the texturedwflags: 5dwWidth: 720dwHeight: 576Format: 1498831189Pool: 0MinBuffers: 1szAspectRatio: 4szNativeSize: 3Could not allocate the private surfacesdwflags: 8dwWidth: 720dwHeight: 576Format: 20Pool: 0MinBuffers: 1szAspectRatio: 4szNativeSize: 3Could not allocate the private surfacesdwflags: 8dwWidth: 720dwHeight: 576Format: 20Pool: 0MinBuffers: 1szAspectRatio: 4szNativeSize: 3Init Texture/Surface VMR OK!dwflags: 5dwWidth: 720dwHeight: 576Format: 22Pool: 0MinBuffers: 1szAspectRatio: 4szNativeSize: 3[Edited by - kiske1 on September 14, 2007 8:07:40 AM];  Sorry about the C# code, but you should be able to follow it.public int InitializeDevice(IntPtr dwUserID, ref VMR9AllocationInfo lpAllocInfo, ref int lpNumBuffers){int hr = 0;Debug.WriteLine(string.Format("{0}x{1} : {2} / {3} / 0x{4:x}", lpAllocInfo.dwWidth, lpAllocInfo.dwHeight, FourCCToStr(lpAllocInfo.Format), lpAllocInfo.Pool, lpAllocInfo.dwFlags));// if format is YUV ? (note : 0x30303030 = "    ")if (lpAllocInfo.Format > 0x30303030){// Check if the hardware support format conversion from this YUV format to the RGB desktop formatif (!Manager.CheckDeviceFormatConversion(m_creationParam.AdapterOrdinal, m_creationParam.DeviceType, (Format)lpAllocInfo.Format, m_adapterInfo.CurrentDisplayMode.Format)){// If not, refuse this format!// The VMR9 will propose other formats supported by the downstream filter output pin.return D3DERR_INVALIDCALL;}}try{IntPtr unmanagedDevice = m_device.GetObjectByValue(DxMagicNumber);IntPtr hMonitor = Manager.GetAdapterMonitor(m_adapterInfo.Adapter);// Give our Direct3D device to the VMR9 filterhr = m_vmrSurfaceAllocatorNotify.SetD3DDevice(unmanagedDevice, hMonitor);DsError.ThrowExceptionForHR(hr);m_videoSize = new Size(lpAllocInfo.dwWidth, lpAllocInfo.dwHeight);int width = 1;int height = 1;// If hardware require textures to power of two sizedif (m_device.DeviceCaps.TextureCaps.SupportsPower2){// Compute the ideal sizewhile (width < lpAllocInfo.dwWidth)width = width << 1;while (height < lpAllocInfo.dwHeight)height = height << 1;m_tu = (float)(lpAllocInfo.dwWidth) / (float)(width);m_tv = (float)(lpAllocInfo.dwHeight) / (float)(height);// notify this change to the VMR9 filterlpAllocInfo.dwWidth = width;lpAllocInfo.dwHeight = height;}m_textureSize = new Size(lpAllocInfo.dwWidth, lpAllocInfo.dwHeight);// Just in caseDeleteSurfaces();// if format is YUV ?if (lpAllocInfo.Format > 0x30303030){// An offscreen surface must be createdlpAllocInfo.dwFlags |= VMR9SurfaceAllocationFlags.OffscreenSurface;// Create itm_videoSurface = m_device.CreateOffscreenPlainSurface(lpAllocInfo.dwWidth, lpAllocInfo.dwHeight, (Format)lpAllocInfo.Format, (Pool)lpAllocInfo.Pool);// And get it unmanaged pointerm_unmanagedSurface = m_videoSurface.GetObjectByValue(DxMagicNumber);// Then create a private texture for the client applicationm_privateTexture = new Texture(  m_device,  lpAllocInfo.dwWidth,  lpAllocInfo.dwHeight,  1,  Usage.RenderTarget,  m_adapterInfo.CurrentDisplayMode.Format,  Pool.Default  );// Get the MipMap surface 0 for the copy (see PresentImage)m_privateSurface = m_privateTexture.GetSurfaceLevel(0);// This code path need a surface copym_needCopy = true;}else{// in RGB pixel formatlpAllocInfo.dwFlags |= VMR9SurfaceAllocationFlags.TextureSurface;// Simply create a texturem_privateTexture = new Texture(  m_device,  lpAllocInfo.dwWidth,  lpAllocInfo.dwHeight,  1,  Usage.RenderTarget,  m_adapterInfo.CurrentDisplayMode.Format,  Pool.Default  );// And get the MipMap surface 0 for the VMR9 filterm_privateSurface = m_privateTexture.GetSurfaceLevel(0);m_unmanagedSurface = m_privateSurface.GetObjectByValue(DxMagicNumber);// This code path don't need a surface copy.// The client appllication use the same texture the VMR9 filter use.m_needCopy = false;}// This allocator only support 1 buffer.// Notify the VMR9 filterlpNumBuffers = 1;}catch (DirectXException e){// A Direct3D error can occure : Notify it to the VMR9 filterreturn e.ErrorCode;}catch{// Or else, notify a more general errorreturn E_FAIL;}// This allocation is a successreturn 0;}public int PresentImage(IntPtr dwUserID, ref VMR9PresentationInfo lpPresInfo){lock (this){try{// If YUV mixing is activated, a surface copy is neededif (m_needCopy){// Use StretchRectangle to do the Pixel Format conversionm_device.StretchRectangle(  m_videoSurface,  new Rectangle(Point.Empty, m_videoSize),  m_privateSurface,  new Rectangle(Point.Empty, m_videoSize),  TextureFilter.None  );}}catch (DirectXException e){// A Direct3D error can occure : Notify it to the VMR9 filterreturn e.ErrorCode;}catch{// Or else, notify a more general errorreturn E_FAIL;}// This presentation is a successreturn 0;}}