Bit operations; Treating a number like an array
The answer depends on the programming language and, to some extent, on the CPU. For any practical purpose, though, extracting the $k$th bit is an $O(1)$ operation even on a "big integer" stored on several machine words, since in order to extract the $k$th bit, you can use the following algorithm:


Determine which word the $k$th bit is in, and which bit in the word.
Extract the appropriate bit from the appropriate word.


In practice it is a reasonable assumption to make that indices of bits fit into a machine word, and so all of these operations are $O(1)$.