Boreal-HL

Boring Reality Hunyuan LoRA

This LoRA is an attempt at improving on the overall detail of generations from Hunyuan. It focus in particular on things such as improved depth of field, realistic skin texture, and better lighting. It works at generating both realistic short video clips and single frames for images.

At the moment, the candidate LoRA used here is overtrained. It is recommended to use a strength of around 0.6. You will also need to experiment a lot with the seeds, guidance, steps, and resolution. Try to keep steps over 35 and minimum resolution above 512x512. Guidance seems to work to varying degrees between 3.5-12.5. Higher guidance and strength may lead to more similarities in things such as scene and characters. You will want to experiment with lowering the strength while raising the resolution and guidance when you run into lots of distortion. Also swap seeds as results vary a lot.

IMPORTANT NOTES:

This LoRA is still very experimental and difficult to control. Even with the settings recommended above it may still be difficult to get usable results. I am continuing doing new training runs to see what will help improve it. Also trying to see if there are any new improvements with an updated workflow.

If you want an easier way of using this LoRA than with Comfyui, you might be able to try running it via Fal's Hunyuan video LoRA. Use this LoRA URL and set steps to 55 (pro mode) with 720p resolution. Expect to wait at least five minutes for it to run. Replicate should also have an option though I have not tested it out yet to verify the results.

Video Examples

Image Examples

The LoRA can also sometimes perform decently on images when you use the initial frame.

image/png

Training Details

I used around 150 images only for this initial LoRA. I focused on public domain photos from around the early 2010s.

  • epochs = 600
  • gradient_accumulation_steps = 4
  • warmup_steps = 100

Adapter

  • type = "lora"
  • rank = 32
  • dtype = "bfloat16"
  • only_double_blocks = true

Optimizer

  • type = "adamw_optimi"
  • lr = 0.0002
  • betas = [ 0.9, 0.99,]
  • weight_decay = 0.01
  • eps = 1e-8

Additional Info

Full Video Demonstration

Diffusion Pipe for training

Gradio UI Diffusion Pipe option for training

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.

Model tree for kudzueye/boreal-hl-v1

Finetuned
(14)
this model