How to set up signed SSL Root and Leaf certificates on a PostgreSQL 11 database on Ubuntu and Windows 10 Power BI plus PostgreSQL?
I believe that PowerBI uses npgsql to connect, which defaults to the behavior that libpq/psql gives you under 'PGSSLMODE=verify-full'.  Can you connect to your server from Windows psql if you use 'PGSSLMODE=verify-full'?

The tutorial you linked assumes that you use the same machine for the certificate authority and for the database server, which will probably work for you.  But it also assumes the client machine is debian/ubuntu, not Windows.  So you will have do everything before "Configure the client" on your ubuntu machine.  But you probably already did that, based on the error message you are getting.

And you will have to do everything after "Configure the client" on your Windows machine, except you have to do it using Windows mechanisms, not linux mechanisms.  How to do that is a question of how your infrastructure is set up--you need some way to copy the file ca-cert.pem from the cert authority to the Windows machine, but that mechanism could be scp (if you have that installed on Windows, perhaps via cygwin) or a CIFS mount, or just copy the text of the .pem files out of putty and paste it into notepad, or use a thumbdrive.

Note that the tutorial is for using certs in both directions--for the client to verify the identity of the server, and for the server to verify the client too.  But it is rare for people to actually use client certs--they usually use server certs only.  If you are not using client certs, then you only need to install one cert on the client, which is the "ca-cert.pem" file (which should get named "root.crt" when installed on the client).  The other two are only needed for client certs.  And on Windows, the path it needs to be installed in is %APPDATA%\postgresql\root.crt, rather than ~/.postgresql/root.crt.