How to install cstore_fdw inside RDS postgres 9.6
No. It's not supported on RDS at this point. See https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.Extensions.

If you really need cstore_fdw, you'll have to manage your own PG instance.