Using a real certificate with rethinkdb cert chain bundle
Silly me. There was --driver-tls-ca option that I was missing. I found it in the rethinkdb manual (man rethinkdb).

TLS options:

  --http-tls-key key_filename                 private key to use for web
                                              administration console TLS
  --http-tls-cert cert_filename               certificate to use for web
                                              administration console TLS
  --driver-tls-key key_filename               private key to use for client driver
                                              connection TLS
  --driver-tls-cert cert_filename             certificate to use for client driver
                                              connection TLS
  --driver-tls-ca ca_filename                 CA certificate bundle used to verify
                                              client certificates; TLS client
                                              authentication disabled if omitted
  --cluster-tls-key key_filename              private key to use for intra-cluster
                                              connection TLS
  --cluster-tls-cert cert_filename            certificate to use for intra-cluster
                                              connection TLS
  --cluster-tls-ca ca_filename                CA certificate bundle used to verify
                                              cluster peer certificates


I set it in my rethinkdb instance conf file:

# TLS stuff
driver-tls-key=/etc/ssl/star.cert.key
driver-tls-cert=/etc/ssl/star.cert.crt
driver-tls-ca=/etc/ssl/star.cert.ca-bundle


And everything works as expected. openssl s_client returns the proper 0 (ok) code.

Edit note:
Although with the rethinkdb dump utility it looks like there is no ca option so I can't use a real cert anyway.