class_docstr: Custom dropout layer which accepts dropout mask as an input (drop_prob is only used for scaling input activations).
  Can be used with RemoteExpert/ModuleBackend to ensure that dropout mask is the same at forward and backward steps
class_docstr_tok: ["Custom","dropout","layer","which","accepts","dropout","mask","as","an","input","(","drop_prob","is","only","used","for","scaling","input","activations",")",".","Can","be","used","with","RemoteExpert\/ModuleBackend","to","ensure","that","dropout","mask","is","the","same","at","forward","and","backward","steps"]
code: def __init__(self, drop_prob):
      super().__init__()
      self.keep_prob = 1 - drop_prob
  
file_path: hivemind/hivemind/moe/server/layers/dropout.py
imports: import torch.autograd
  from torch import nn
  from hivemind.moe.server.layers.custom_experts import register_expert_class
label_desc: Use image node_id 1 to create a new DeterministicDropout object from inherited base classes: nn with example: obj = DeterministicDropout(drop_prob)