title: Using SageMaker with Hydra
body: <p>I have a question about SageMaker and Hydra.</p>
  <p><strong>TL;DR</strong>
  Is there a way to pass arguments from SageMaker estimator to a Hydra script? Currently it passes parameters in a very strict way.</p>
  <p><strong>Full Question</strong>
  I use Hydra in order to pass configs to my training script. I have many configs and it works good for my. For example, if I want to use a specific optimizer, I do:</p>
  <pre><code>python train.py optimizer=adam
  </code></pre>
  <p>This is my training script, for instance:</p>
  <pre><code>@hydra.main(version_base=None, config_path=&quot;configs/&quot;, config_name=&quot;config&quot;)
  def train(config: DictConfig):
      logging.info(f&quot;Instantiating dataset &lt;{config.dataset._target_}&gt;&quot;)
      train_ds, val_ds = hydra.utils.call(config.dataset)
  
      logging.info(f&quot;Instantiating model &lt;{config.model._target_}&gt;&quot;)
      model = hydra.utils.call(config.model)
  
      logging.info(f&quot;Instantiating optimizer &lt;{config.optimizer._target_}&gt;&quot;)
      optimizer = hydra.utils.instantiate(config.optimizer)
  
      logging.info(f&quot;Instantiating loss &lt;{config.loss._target_}&gt;&quot;)
      loss = hydra.utils.instantiate(config.loss)
  
      callbacks = []
      if &quot;callbacks&quot; in config:
          for _, cb_conf in config.callbacks.items():
              if &quot;_target_&quot; in cb_conf:
                  logging.info(f&quot;Instantiating callback &lt;{cb_conf._target_}&gt;&quot;)
                  callbacks.append(hydra.utils.instantiate(cb_conf))
  
      metrics = []
      if &quot;metrics&quot; in config:
          for _, metric_conf in config.metrics.items():
              if &quot;_target_&quot; in metric_conf:
                  logging.info(f&quot;Instantiating metric &lt;{metric_conf._target_}&gt;&quot;)
                  metrics.append(hydra.utils.instantiate(metric_conf))
  
      model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
  
      model.fit(
          train_ds,
          validation_data=val_ds,
          epochs=config.epochs,
          callbacks=callbacks,
      )
  
  
  if __name__ == &quot;__main__&quot;:
      train()
  </code></pre>
  <p>And I have a relevant <code>optimizer/adam.yaml</code> file.</p>
  <p>Now, I started using SageMaker to run my experiments in the cloud and I noticed a problem.
  It doesn't support the hydra syntax (<code>+optimizer=sgd</code>), stuff like that.</p>
  <p>Is there a way to make it play nicely with Hydra syntax? If not, do you have a suggestion for refactoring my training code so that it would work nicely with Hydra/OmegaConf?</p>
  <p>I saw there is a similar question in SageMaker issues page, but it doesn't have any replies:
  <a href="https://github.com/aws/sagemaker-python-sdk/issues/1837" rel="nofollow noreferrer">https://github.com/aws/sagemaker-python-sdk/issues/1837</a></p>