Removing Noise from Dental Radiography I'm working on a project of applying Active Shape Model  to locate tooth in dental radiograph. For those familiar with the technique, I'm currently trying to sample along normal vectors for each landmark. The paper recommends to take derivatives of sampled pixels: "To reduce effects of global intensity changes we sample the derivative along the profile, rather than the absolute grey-level values."

So my problem is how to filter dental radiographs in the best way to prepare them for applying derivative operator. I'm currently using combination of median filter to remove most of what I think is quantum noise (mottle). It is followed by bilateral filter. Then I apply Scharr operator to compute the actual gradient which should be sampled.

The results are presented below:


The first image shows original data. In the second and third image, filtered data are presented, first as a magnitude of spectrum after FFT and then as a filtered image data. Fourth image show result of applying Scharr operator to third image.

My questions are:


Are there a well known approaches for reducing noise in dental radiograph that would differ from my approach?
What is causing the "smoky" appearance of the edges and "flat"(non-edge) areas? Is it some kind of leftover noise in the filtered image or is it inherent to the gradient operator? If it is indeed a noise, which filter would be most suitable to use? Median filter was good at removing small noisy blobs but large kernel causes the edges to blur out too much. So bilateral filter is used to filter out larger blobs and equalize color over area without harming edges, but it is not able to filter this smoky structure.
Is there a better option than Scharr operator to create gradient in this case?
Bonus: Would this be considered a good input for Active Shape Model? I'm not yet aware how robust they are.
As far as I understood, by image derivation you mean extracting edges. I would recommend to filter the image by a relatively large Gaussian filter. If computational cost of image derivation is uncritical to your work, I would recommend using canny edge detector. It is less sensitive to noise and does not fool by noise, and finds weak edges along with strong edges. Matlab instruction for that:

   [MinThresh MaxThresh]=[-0.3 0.5];
   EDGE_No_SMOKE=edge(im,'canny',[MinThresh MaxThresh]);


and the result is (I know it might not be the results you're seeking, however playing with Threshold variables and filter size will bring you desirable results):

 

Note that you do not see smoky effect anymore. Also about those wrong edges, you can remove them using image opening and closing techniques.