Is the Arduino library so slow?
Unless you have a really large number of read/write operations and very tight time constraints, I doubt you are going to have any significant advantage in terms of speed.

Accessing the bits in a port together, rather than individually is certainly more efficient, both in terms of time and space, but it might cost in readability, depending on how comfortable you are with leaving the Arduino libraries behind.

You can have also a look at this thread.

Notice that the maximum gain happens only when all the pins you have to read or write belong to the same register.