controlnet-RiddleHe/SD14_pathology_controlnet
These are controlnet weights trained on CompVis/stable-diffusion-v1-4 with new type of conditioning. You can find some example images below.
prompt: A histopathology image of breast cancer tissue.
prompt: An image of breast cancer histopathology with detailed cellular structures.
Intended uses & limitations
How to use
controlnet = ControlNetModel.from_pretrained("RiddleHe/SD14_pathology_controlnet", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
"RiddleHe/SD14_pathology_base", controlnet=controlnet, torch_dtype=torch.float16
)
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
pipe.to('cuda')
prompt = "A histopathology image of breast cancer tissue."
mask = mask.convert("RGB") # Provide a mask
generator = torch.Generator(device='cuda').manual_seed(42)
with torch.no_grad():
out = pipe(prompt, image=mask, num_inference_steps=70, num_images_per_prompt=3, generator=generator).images
Limitations and bias
[TODO: provide examples of latent issues and potential remediations]
Training details
The model is trained on 28216 image-mask pairs from the BRCA breast cancer dataset. Input is mask and output is image.
Mask is a single channel image with integer values from 0 to 21 representing 22 classes, eg. 1 representing tumor, 2 representing stroma.
- Downloads last month
- 21
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
Model tree for RiddleHe/SD14_pathology_controlnet
Base model
CompVis/stable-diffusion-v1-4