NYU VisionX

university
Activity Feed

AI & ML interests

None defined yet.

Recent Activity

nyu-visionx's activity

sayakpaulย 
posted an update 27 days ago
view post
Post
3136
Inference-time scaling meets Flux.1-Dev (and others) ๐Ÿ”ฅ

Presenting a simple re-implementation of "Inference-time scaling diffusion models beyond denoising steps" by Ma et al.

I did the simplest random search strategy, but results can potentially be improved with better-guided search methods.

Supports Gemini 2 Flash & Qwen2.5 as verifiers for "LLMGrading" ๐Ÿค—

The steps are simple:

For each round:

1> Starting by sampling 2 starting noises with different seeds.
2> Score the generations w.r.t a metric.
3> Obtain the best generation from the current round.

If you have more compute budget, go to the next search round. Scale the noise pool (2 ** search_round) and repeat 1 - 3.

This constitutes the random search method as done in the paper by Google DeepMind.

Code, more results, and a bunch of other stuff are in the repository. Check it out here: https://github.com/sayakpaul/tt-scale-flux/ ๐Ÿค—
sayakpaulย 
posted an update about 2 months ago
view post
Post
2007
We have been cooking a couple of fine-tuning runs on CogVideoX with finetrainers, smol datasets, and LoRA to generate cool video effects like crushing, dissolving, etc.

We are also releasing a LoRA extraction utility from a fully fine-tuned checkpoint. I know that kind of stuff has existed since eternity, but the quality on video models was nothing short of spectacular. Below are some links:

* Models and datasets: https://huggingface.co/finetrainers
* finetrainers: https://github.com/a-r-r-o-w/finetrainers
* LoRA extraction: https://github.com/huggingface/diffusers/blob/main/scripts/extract_lora_from_model.py
  • 1 reply
ยท
sayakpaulย 
posted an update about 2 months ago
view post
Post
1972
We have authored a post to go over the state of video generation in the Diffusers ecosystem ๐Ÿงจ

We cover the models supported, the knobs of optims our users can fire, fine-tuning, and more ๐Ÿ”ฅ

5-6GBs for HunyuanVideo, sky is the limit ๐ŸŒŒ ๐Ÿค—
https://huggingface.co/blog/video_gen
sayakpaulย 
posted an update 3 months ago
view post
Post
4378
Commits speak louder than words ๐Ÿคช

* 4 new video models
* Multiple image models, including SANA & Flux Control
* New quantizers -> GGUF & TorchAO
* New training scripts

Enjoy this holiday-special Diffusers release ๐Ÿค—
Notes: https://github.com/huggingface/diffusers/releases/tag/v0.32.0