Can we add the destination query parameter to a link from yml configuration?
Yes, you can add the destination query directly in the yml file. 
@see : \Drupal\Core\Menu\LocalActionDefault::getOptions

ie: mymodule.links.action.yml

my_module.my_entity.add:
  route_name: node.add
  title: 'Add My Entity'
  route_parameters:
        node_type: 'my_entity'
  options:
    query:
      destination: '/path/to/redirect'
  appears_on:
    - my_module.my_entity.admin_content