How can I setup and use a PostgreSQL database without installing a driver?
postgres can be "installed" from a ZIP file by simply extracting it. 

psql is part of the Postgres "installation" can be used without any further steps. 

Any JDBC based query tool can also be used without "installing" a driver, as you can put the JDBC driver's .jar file anywhere you want.

It seems you can download the ODBC driver binaries directly without an installer. But I don't know if it's possible to properly register that with any tool that needs it.