class_docstr: The config for sharding the huggingface model
  
  Args:
      tensor_parallel_process_group (Optional[ProcessGroup]): The process group of tensor parallelism, it's necessary when using tensor parallel. Defaults to None, which is the global process group.
      pipeline_stage_manager (Optional[PipelineStageManager]): If using pipeline parallelism, it's necessary to specify a pipeline stage manager for inter-process communication in pipeline parallelism. Defaults to None, which means not using pipeline parallelism.
      enable_tensor_parallelism (bool): Whether to use tensor parallelism. Defaults to True.
      enable_fused_normalization (bool): Whether to use fused layernorm. Defaults to False.
      enable_flash_attention (bool, optional): Whether to switch on flash attention. Defaults to False.
      enable_jit_fused (bool, optional): Whether to switch on JIT fused operators. Defaults to False.
      enable_sequence_parallelism (bool): Whether to turn on sequence parallelism, which partitions non-tensor-parallel regions along the sequence dimension. Defaults to False.
      enable_sequence_overlap (bool): Whether to turn on sequence overlap, which overlap the computation and communication in sequence parallelism. It can only be used when enable_sequence_parallelism is True. Defaults to False.
      enable_all_optimization (bool): Whether to turn on all optimization tools including 'fused normalization', 'flash attention', 'JIT fused operators', 'sequence parallelism' and 'sequence overlap'. Defaults to False.
docstr: Turn on all optimization.
code: def _turn_on_all_optimization(self):
      """
      Turn on all optimization.
      """
      # you can add all the optimization flag here
      self.enable_fused_normalization = True
      self.enable_flash_attention = True
      self.enable_jit_fused = True
      self.enable_sequence_parallelism = True
      self.enable_sequence_overlap = True
  
file_path: colossalai/colossalai/shardformer/shard/shard_config.py
imports: from dataclasses import dataclass, field
  from typing import Any, Dict, Optional
  import torch.distributed
  from torch.distributed import ProcessGroup
  from colossalai.pipeline.stage_manager import PipelineStageManager
label_desc: Use image node_id 3 for calling the ShardConfig obj's underlying member method code with example usage: obj._turn_on_all_optimization() without return types