How do I programatically add address to commerce order
To record the address on the order, you must create a customer profile of the relevant type (billing information or, if you have Commerce Shipping on the site, shipping information), populate its addressfield value, and reference it from the order you've created. My recommendation would be to use the entity_metadata_wrapper() from the Entity API to manipulate those field values.

The addressfield on the customer profile you need to populate is commerce_customer_address, and the reference field on the order is commerce_customer_billing (or _shipping respectively).