Is the Arduino library so slow? I have a plan to operate 5MP camera module with the arduino Due. But I wonder if I can use the arduino libraries that include digitalRead, digitalWrite and so on. But some people said that it is a bad idea. Especially I found that people wrote values on registers instead of using the library, when they control ov7670 with Uno. 

Contrary to this, ArduCam uses the libraries, even for OV5642!. Here is the link 
https://github.com/ArduCAM/Arduino/blob/master/ArduCAM/examples/mini/ArduCAM_Mini_OV5642_Video_Streaming/ArduCAM_Mini_OV5642_Video_Streaming.ino

So could anyone please let me know, if it is okay to use such library and never to use register directly? With 'register' I mean arduino's register, not camera's. 

I guess the system clock and existence of FIFO would affect the answer. But still I don't know how to evaluate their effect analytically.
Unless you have a really large number of read/write operations and very tight time constraints, I doubt you are going to have any significant advantage in terms of speed.

Accessing the bits in a port together, rather than individually is certainly more efficient, both in terms of time and space, but it might cost in readability, depending on how comfortable you are with leaving the Arduino libraries behind.

You can have also a look at this thread.

Notice that the maximum gain happens only when all the pins you have to read or write belong to the same register.