Testing query speed in two different RDBMS I'm planning to compare the speed of queries in PostgreSQL and SQL server (on the same datasets). I was planning to run the same queries multiple times in both databases and use average response time in miliseconds as a rough estimate. 

This seems a little too basic. Is there a better way to compare a speed of a query in two different database systems running on the same hardware? Any tools that I can use?
You could use HammerDB. With HammerDB you perform a TPC benchmark test. TPC benchmark testing can be used by vendors for official certification, but you can also use it as 'non-offical' for your personal benchmarking. The TPC-E (OLTP) and TPC-H (OLAP) are the more modern version of the TPC benchmark tests. With HammerDB you can emulate an OLAP or OLTP workload with the same characteristics (you can configure this) and run it on different RDBMSes. 

You have 2 types of measurements; TPM and NOPM. TPM cannot be compared between different database types, but NOPM can. This tool is fairly easy to use, although you can make things a little more complex if you want to. :-) You can get HammerDB simpy from hammerdb.com.

Although this is not intended to test with a specific query, it could be usefull depending on your scenario.