   Anti-aliasing problems DirectX
  Hi,My game is nearly finished but I''m having some problems with my anti-aliasing of the tiles, unfortunatly.I''ve been rendering my objects in 3DSMax, and just found out a week ago about TGA files and an Alpha Channel. I''ve used Colorkeys before, but it made the objects look aliased around it (the black border), like this:With Alpha Channels it got a lot better, but unfortunatly I''m stuck with an annoying problem. There is some grey pixels coming around my mirrors, caused by the anti-aliasing. Although barely visibile in PSP, it is clearly visible in-game. It produces results like:Now, simply turning off anti-aliasing works quite well, but then I have to filter the texture, which causes some grey pixels again. Also, when I rotate a mirror on 45 degrees or something, it will look terribly aliased, like you can see here ( in the full game it is more visible, but if you look closely, you will see what I mean ):I''ve spent like four full evenings trying to get this solved, without having to edit ALL my tiles by hand, to remove this grey stuff. I''ve also tried rendering it 64x64 and scaling it down to 32x32. didn''t exactly work well either. Searching all the Gamedev forums which are somehow related to this issue, didn''t give much of a help either . I''ve been googling around as well, for quite some keywords, but still no solution. FSAA and stuff is not an option Can someone maybe shed some light which anti-aliasing method to use in 3DSMax, or some tips how to get rid of these grey surroundings? (Without having to manually fix all my tiles...) 
  Max has a preference setting, "Don''t antialias against background". Maybe that is just the thing you need -Nik ;  Been there . It results in choppy mirrors, well, just the same happens as mentioned above. I''m pretty clueless. But some people in commerical games obviously get it right, and I''m sure someone could explain how exactly . Well I hope ;  A suggestion:Use a colorkey as usual to make the background color completely transparent (i.e. 0 alpha). Then, for every pixel in your tile, calculate the distance between it and the nearest background pixel, and use this as an estimator for determining the pixel''s alpha value.Let''s call the distance from the current pixel to the nearest background pixel "dist". Let''s say we''re going to blend pixels that are less than 6 pixels away from background pixels.pixel.alpha = min( 1, dist/6 )That way, after you enable alpha blending, pixels that are away from background pixels by less than 6 pixels are "blended" with the background, increasingly as you go near the background (i.e. near the edges of the tile).This can reduce the aliasing you''re seeing, although I can''t gurantee it''ll look right. Anyway, you can easily try this on a couple of tiles, and if the result is satisfactory, apply it to all others. Make sure you play with the "6" value until you find what''s best for your tiles.   Muhammad Haggag;  Hmm, that''s an interesting theory. I miht try that. Still, I think there should be a more simple solution somehow ;  Render your disc against a black background with antialiasing enabled and whatever t takes to generate alpha in the image.  Then set the texture stage state to use a color op of D3DTOP_BLENDTEXTUREALPHAPM.;  Hm, then it got all white, but with the mirror shape. Then I tried without the "M" and got the same situation  The right images get''s all weird grey stuff. This is just the rotated version (rotating the vertex quad) of the left one..;  You''ve got a problem with whatever tool you''ve used to generate the alpha values for the disc.  If you ask the tool to antialias against a background and it is generating alpha values, it should scale the alpha values near the edges by the same amount that it has scaled the color values to perform the blending.This evening I generated a similar disc using Rhino3D.  Rhino won''t generate alpha values when rendering so I had to improvise.I rendered two images of the disc with antialiasing enabled; one over a black background and then another over a white background.  Neither of these images contain any alpha information. I then wrote a small app that constructed an alpha map using the two images comparing white and black pixels and pixels that were identical between the two images.  I used the resulting alpha map with the image that I rendered over the black background in the MFC Texture DXSDK sample, using the premultiplied blending mode I mentioned before, and it worked perfectly. I don''t know how to post images here otherwise I''d show you the series of 64x64 bitmaps I generated.There''s got to be a better way to do this than having to write code to perform 2D post-processing of rendered images.  You might want to ask in the Visual Arts forum.;quote:I don''t know how to post images here otherwise I''d show you the series of 64x64 bitmaps I generated.You can go here: http://www.freepgs.com/cgi-bin/easyhost.cgiLogin:User: MHaggagpassword: gduploadUpload whatever you wish, and directly link to it from here (img src="bla bla bla" stuff).And yes, anyone wishing to upload can use this. It''s a 20 MB free space, with a limit of 20 MB daily transfer, 2 GB monthly (I *think*).   Muhammad Haggag;  Here''s an more detailed explanation on how I used a post-process technique to generate the appropriate alpha data.First I generated two 64x64 pixel images of a disc using Rhino3D. Both images were rendered with antialiasing enabled. The only difference between the two images is the color of the background (black vs. white):Next I wrote a small app that parsed the pixel data of each of these two bitmaps to generate an alpha mask. White pixels in the image with a white background, that were also black in the image with a black background are transparent. Pixels that are the same color between the two images are opaque. The remaining pixels have some degree of translucency. Here is an image showing the output of the first stage of this process. Black pixels are transparent, white pixels are opaque, and green pixels are translucent:In order to get the various values of the translucent pixels you can use the typical alpha blending equation and substitute portions of the equation with the color values of pixels in the images with the white and black background:color = (src_color * alpha) + dest_color * (1.0 - alpha)In the image with the black background, the dest_color is 0.0 (black). This simplifies the equation to:color = (src_color * alpha)We know the value of ''color'' in the equation above already since it''s the pixel color in the image with the black background. We need to find the other two unknowns, src_color and alpha. (Actually if we''re eventually going to be doing premultiplied blending, we only need the alpha value). To find the alpha value, we need to bring in the pixel from the image with the white background where dest_color in the alpha equation is 1.0 (white).pixel_from_white_bkgnd_image = (src_color * alpha) + 1.0 - alphaSubstituting in the pixel value from the image with the black background for (src_color * alpha) and a little shuffling we get:alpha = 1.0 - (pixel_from_black_bkgnd_image - pixel_from_white_bkgnd_image)The following image is the completed alpha map with the green pixels replaced by the appropriate alpha values:Once you have the alpha value you can solve for the src_color value in the alpha blending equation and produce a texture map that contains the proper color and alpha values. I used the premultiplied alpha blending mode I mentioned in previous posts (this is the image with the black background with the computed alpha map added) to generate the following image of a disc rendered on a gouraud shaded quad. As you can see, there are no gray pixels or odd halos around the edges of the disc and the image is antialiased quite nicely:I hope that explains the process and you can gather something from it. It really sounds like whatever tool you''re using to generate your alpha mask isn''t generating the correct values. If you get really stuck you can use the technique I''ve described above to generate the correct values. Good luck.