This is what I have to do: 

To build a CAD-like application that loads a point cloud (i.e. thousands of 3D points representing a 3D object) from file, allows the users to manipulate the points (i.e. change the shape by moving the points), do a lot of calculations the points on the points (e.g. finding the intersection points between lines and surfaces, detect a point is above or under a surface etc., measure the distances between points, or points to surface etc.), and then save the modified points to file.

It also provides basic CAD-like UI features such as zoom in/out, pan the view, rotation the camera etc.

Speed is the major concern.

Instead of writing my own functions for matrix operation and defining my own point/line/surface classes, I would like to use existing libraries/APIs to do the job.

I know WPF, XNA and SlimDX provides the API to do 3D geometric calculations and all of them are finally calling DirectX, but I'm just newbie to all of them. I'm wondering:

1. Which one (or some other suggestion) could give better performance in speed.
2. My understanding about DirectX's 3D functions is that it mainly deals with gaming graphics / screen outputs, is it also suitable for data-level calculations(i.e. use the 3D functions to manipulate the point data, calculate the distances etc., but not outputting it on the screen)? By suitable, I mean if I create thousands of DirectX vertexes and mainpulate them, would it be much slower than using my own data types and structures?
Pls correct me if my understanding is wrong.
3. If I use WPF, do I need to use XNA as well? I'm kind of mixing up these two things.
4. The application is supposed to run in research lab's PC which doesn't have powerful gaming display card, so does it mean XNA is not preferred?
5. An suggestion about the technologies should be used for this application?

Thanks!!

========update

To make it clearer, the app will load ~108,000 points in 3D, and every points will form surfaces with other adjacent points, so roughly the the same number of 3D surfaces are involved (I'm not generating them at the same time). I will do a lot of 3D geometric and matrix calculations with the point and surfaces, such as intersection, interpolation, transformation etc. , so the speed of the "calculations" is my major concern. Most of the time I will only draw the final result to the screen and the drawing is mainly lines and points, the speed of "drawing" is not a big concern . so it is not really a graphic-intensive app, but a geometric-calculation-intensive app. 

After reading the answer and comments, I think of two options: 

1. store & calculate the data with primitive data-types, and convert data to the WPF/XNA/SlimDX data structures when drawing them on screen, or
2. use these API's data structures to store, calculation and drawing all those points.

which one is better?