Exporting certificate without private key
Then I export the certificate and import it into the machine where the SQL Server is running. 


That defeats the purpose of using Always Encrypted if the server that is running SQL Server also has the certificate to decrypt all the data.

Please, please, please don't do this! Put it on the client machine or a "tools" or "intermediate loading" machine to get the data in and out. Then remove the key and keep it only on the client machines that are locked down.

I'd also keep, in the back of your mind, a solution for when a client machine is lost/stolen/breached (laptop), or a server is compromised. How do you find out, how long do you have to rotate CMKs, do you know how to rotate CMKs, etc.


  I am wondering is there any cases where I may need to export only the certificate without the privet key in the context of Always Encrypted feature (as the work of this certificate it to protect the column encryption keys only) ?


The private key is what keeps everything safe, so no private key then no decryption. I can't think of any times when this would be useful.