How long does it take a quantum computer to brute force AES? I understand that using Grover's algorithm it only requires $2^{64}$ lookups for a 128 bit AES encryption, leading people to say we need to increase to 256 bit keys. But how long would it actually take a quantum computer to do $2^{64}$ lookups? 

I can't say I fully understand quantum computers, but aren't people making the assumption that 128 bit is only unsafe to quantum computers because 64 bit (which 128 bit becomes using a quantum computer) can be broken through brute force on our current computers...? Even if a quantum computer only needs to do $2^{64}$ lookups, presumably it doesn't do them at the exact speed of current computers. 

When people have asked in the past how long a quantum computer would take to break AES 128 bit, people always answer that it would take $2^{64}$ lookups (which some people take to mean the amount of time we currently take to break 64 bit), but there's never any indication of an actual time.

I understand that quantum computers are highly theoretical at this stage in terms of large scale implementation, but can anyone offer any ideas?
I'm not going to exactly answer your question, because I have no idea. I simply do not know how fast the quantum computer is that NSA is building in secret.

However I could explain why people recommend 256-bit security in the face of quantum computing using some numbers. If you feel that $2^{128}$ is a comfortable security against bruteforcing, remember that a $2^{64}$ security level is $18446744073709551616$ times faster to bruteforce. 

That means a ~0.000000000162630 GHz quantum processor bruteforces a 128 bit key faster than a 3GHz regular processor, assuming one lookup per cycle.