docstr: MXNet Estimator provides an automatic setup for synchronous distributed MXNet training.
  
  :param config: A dictionary for training configurations. Keys must include the following:
  optimizer, optimizer_params, log_interval.
  optimizer should be an MXNet optimizer or its string representation.
  optimizer_params should be a dict in companion with the optimizer. It can contain
  learning_rate and other optimization configurations.
  log_interval should be an integer, specifying the interval for logging throughput and
  metrics information (if any) during the training process.
  You can call create_config to directly create it.
  You can specify "seed" in config to set random seed for weight initialization.
  You can specify "init" in extra_config to set model initializer for gluon models.
  :param model_creator: A function that takes config as argument and returns an MXNet model.
  The model can be defined either using MXNet symbolic API or imperative(gluon) API.
  :param loss_creator: A function that takes config as argument and returns an MXNet loss.
  This is not needed for symbolic API where loss is already defined as model output.
  :param eval_metrics_creator: A function that takes config as argument and returns one or
  a list of MXNet metrics or corresponding string representations of metrics, for example,
  'accuracy'. This is not needed if you don't need evaluation on the training data set.
  :param validation_metrics_creator: A function that takes config as argument and returns one
  or a list of MXNet metrics or corresponding string representations of metrics, for example,
  'accuracy'. This is not needed if you don't have validation data throughout the training.
  :param num_workers: The number of workers for distributed training. Default to be the number
  of nodes in the cluster.
  :param num_servers: The number of servers for distributed training. Default is None and in
  this case it would be equal to the number of workers.
  :param runner_cores: The number of CPU cores allocated for each MXNet worker and server.
  Default is None. You may need to specify this for better performance when you run in
  cluster.
  :return: MXNet Estimator object.
docstr_tok: ["MXNet","Estimator","provides","an","automatic","setup","for","synchronous","distributed","MXNet","training",".",":","param","config",":","A","dictionary","for","training","configurations",".","Keys","must","include","the","following",":","optimizer",",","optimizer_params",",","log_interval",".","optimizer","should","be","an","MXNet","optimizer","or","its","string","representation",".","optimizer_params","should","be","a","dict","in","companion","with","the","optimizer",".","It","can","contain","learning_rate","and","other","optimization","configurations",".","log_interval","should","be","an","integer",",","specifying","the","interval","for","logging","throughput","and","metrics","information","(","if","any",")","during","the","training","process",".","You","can","call","create_config","to","directly","create","it",".","You","can","specify","``","seed","''","in","config","to","set","random","seed","for","weight","initialization",".","You","can","specify","``","init","''","in","extra_config","to","set","model","initializer","for","gluon","models",".",":","param","model_creator",":","A","function","that","takes","config","as","argument","and","returns","an","MXNet","model",".","The","model","can","be","defined","either","using","MXNet","symbolic","API","or","imperative","(","gluon",")","API",".",":","param","loss_creator",":","A","function","that","takes","config","as","argument","and","returns","an","MXNet","loss",".","This","is","not","needed","for","symbolic","API","where","loss","is","already","defined","as","model","output",".",":","param","eval_metrics_creator",":","A","function","that","takes","config","as","argument","and","returns","one","or","a","list","of","MXNet","metrics","or","corresponding","string","representations","of","metrics",",","for","example",",","'accuracy","'",".","This","is","not","needed","if","you","do","n't","need","evaluation","on","the","training","data","set",".",":","param","validation_metrics_creator",":","A","function","that","takes","config","as","argument","and","returns","one","or","a","list","of","MXNet","metrics","or","corresponding","string","representations","of","metrics",",","for","example",",","'accuracy","'",".","This","is","not","needed","if","you","do","n't","have","validation","data","throughout","the","training",".",":","param","num_workers",":","The","number","of","workers","for","distributed","training",".","Default","to","be","the","number","of","nodes","in","the","cluster",".",":","param","num_servers",":","The","number","of","servers","for","distributed","training",".","Default","is","None","and","in","this","case","it","would","be","equal","to","the","number","of","workers",".",":","param","runner_cores",":","The","number","of","CPU","cores","allocated","for","each","MXNet","worker","and","server",".","Default","is","None",".","You","may","need","to","specify","this","for","better","performance","when","you","run","in","cluster",".",":","return",":","MXNet","Estimator","object","."]
code: def from_mxnet(
      *,
      config,
      model_creator,
      loss_creator=None,
      eval_metrics_creator=None,
      validation_metrics_creator=None,
      num_workers=None,
      num_servers=None,
      runner_cores=None,
  ):
      """
      MXNet Estimator provides an automatic setup for synchronous distributed MXNet training.
  
      :param config: A dictionary for training configurations. Keys must include the following:
      optimizer, optimizer_params, log_interval.
      optimizer should be an MXNet optimizer or its string representation.
      optimizer_params should be a dict in companion with the optimizer. It can contain
      learning_rate and other optimization configurations.
      log_interval should be an integer, specifying the interval for logging throughput and
      metrics information (if any) during the training process.
      You can call create_config to directly create it.
      You can specify "seed" in config to set random seed for weight initialization.
      You can specify "init" in extra_config to set model initializer for gluon models.
      :param model_creator: A function that takes config as argument and returns an MXNet model.
      The model can be defined either using MXNet symbolic API or imperative(gluon) API.
      :param loss_creator: A function that takes config as argument and returns an MXNet loss.
      This is not needed for symbolic API where loss is already defined as model output.
      :param eval_metrics_creator: A function that takes config as argument and returns one or
      a list of MXNet metrics or corresponding string representations of metrics, for example,
      'accuracy'. This is not needed if you don't need evaluation on the training data set.
      :param validation_metrics_creator: A function that takes config as argument and returns one
      or a list of MXNet metrics or corresponding string representations of metrics, for example,
      'accuracy'. This is not needed if you don't have validation data throughout the training.
      :param num_workers: The number of workers for distributed training. Default to be the number
      of nodes in the cluster.
      :param num_servers: The number of servers for distributed training. Default is None and in
      this case it would be equal to the number of workers.
      :param runner_cores: The number of CPU cores allocated for each MXNet worker and server.
      Default is None. You may need to specify this for better performance when you run in
      cluster.
      :return: MXNet Estimator object.
      """
      return MXNetEstimator(
          config=config,
          model_creator=model_creator,
          loss_creator=loss_creator,
          eval_metrics_creator=eval_metrics_creator,
          validation_metrics_creator=validation_metrics_creator,
          num_workers=num_workers,
          num_servers=num_servers,
          runner_cores=runner_cores,
      )
  
imports: import os
  import types
  import itertools
  import subprocess
  import ray
  from dmlc_tracker.tracker import get_host_ip
  from bigdl.orca.data.utils import partitions_get_data_label, process_spark_xshards
  from bigdl.orca.ray import OrcaRayContext
  from bigdl.orca.learn.mxnet.mxnet_runner import MXNetRunner
  from bigdl.orca.learn.mxnet.utils import find_free_port
  from bigdl.orca.learn.ray_estimator import Estimator
  from bigdl.dllib.utils.log4Error import *
label_desc: Use image node_id 1 for calling the Estimator obj's underlying member method code with example usage: obj.from_mxnet() and returns: MXNetEstimator