Let’s walk through more in-detail design decisions for each class.  Pipelines Pipelines are designed to be easy to use (therefore do not follow Simple over easy 100%), are not feature complete, and should loosely be seen as examples of how to use models and schedulers for inference. The following design principles are followed: Pipelines follow the single-file policy. All pipelines can be found in individual directories under src/diffusers/pipelines. One pipeline folder corresponds to one diffusion paper/project/release. Multiple pipeline files can be gathered in one pipeline folder, as it’s done for src/diffusers/pipelines/stable-diffusion. If pipelines share similar functionality, one can make use of the #Copied from mechanism. Pipelines all inherit from DiffusionPipeline. Every pipeline consists of different model and scheduler components, that are documented in the model_index.json file, are accessible under the same name as attributes of the pipeline and can be shared between pipelines with DiffusionPipeline.components function. Every pipeline should be loadable via the DiffusionPipeline.from_pretrained function. Pipelines should be used only for inference. Pipelines should be very readable, self-explanatory, and easy to tweak. Pipelines should be designed to build on top of each other and be easy to integrate into higher-level APIs. Pipelines are not intended to be feature-complete user interfaces. For future complete user interfaces one should rather have a look at InvokeAI, Diffuzers, and lama-cleaner. Every pipeline should have one and only one way to run it via a __call__ method. The naming of the __call__ arguments should be shared across all pipelines. Pipelines should be named after the task they are intended to solve. In almost all cases, novel diffusion pipelines shall be implemented in a new pipeline folder/file.  Models Models are designed as configurable toolboxes that are natural extensions of PyTorch’s Module class. They only partly follow the single-file policy. The following design principles are followed: Models correspond to a type of model architecture. E.g. the UNet2DConditionModel class is used for all UNet variations that expect 2D image inputs and are conditioned on some context. All models can be found in src/diffusers/models and every model architecture shall be defined in its file, e.g. unet_2d_condition.py, transformer_2d.py, etc… Models do not follow the single-file policy and should make use of smaller model building blocks, such as attention.py, resnet.py, embeddings.py, etc… Note: This is in stark contrast to Transformers’ modeling files and shows that models do not really follow the single-file policy. Models intend to expose complexity, just like PyTorch’s Module class, and give clear error messages. Models all inherit from ModelMixin and ConfigMixin. Models can be optimized for performance when it doesn’t demand major code changes, keeps backward compatibility, and gives significant memory or compute gain. Models should by default have the highest precision and lowest performance setting. To integrate new model checkpoints whose general architecture can be classified as an architecture that already exists in Diffusers, the existing model architecture shall be adapted to make it work with the new checkpoint. One should only create a new file if the model architecture is fundamentally different. Models should be designed to be easily extendable to future changes. This can be achieved by limiting public function arguments, configuration arguments, and “foreseeing” future changes, e.g. it is usually better to add string “…type” arguments that can easily be extended to new future types instead of boolean is_..._type arguments. Only the minimum amount of changes shall be made to existing architectures to make a new model checkpoint work. The model design is a difficult trade-off between keeping code readable and concise and supporting many model checkpoints. For most parts of the modeling code, classes shall be adapted for new model checkpoints, while there are some exceptions where it is preferred to add new classes to make sure the code is kept concise and