YINYANG-ALIGN: Benchmarking Contradictory Objectives and Proposing Multi-Objective Optimization based DPO for Text-to-Image Alignment

YinYang-Align: Benchmarking Contradictory Objectives and Proposing Multi-Objective Optimization Based DPO for Text-to-Image Alignment

• Authors: Amitava Das, Yaswanth Narsupalli, Gurpreet Singh, Vinija Jain, Vasu Sharma, Suranjana Trivedy, Aishwarya Naresh Reganti, Aman Chadha, Amit Sheth

The paper introduces YinYangAlign, a benchmarking framework for evaluating alignment in Text-to-Image (T2I) systems across six contradictory objectives and proposes Contradictory Alignment Optimization (CAO), a multi-objective extension of Direct Preference Optimization (DPO), to balance competing constraints effectively.

Key Contributions & Technical Novelty

1. Introduction of the YinYangAlign Benchmark

• Establishes a systematic framework for evaluating alignment fidelity in T2I systems.
• Defines six inherently contradictory design objectives for image generation:
  • Faithfulness to Prompt vs. Artistic Freedom
  • Emotional Impact vs. Neutrality
  • Visual Realism vs. Artistic Freedom
  • Originality vs. Referentiality
  • Verifiability vs. Artistic Freedom
  • Cultural Sensitivity vs. Artistic Freedom
• Provides an axiom dataset with human prompts, aligned AI outputs, rejected outputs, and explanations of contradictions.

2. Multi-Objective Optimization for Alignment: Contradictory Alignment Optimization (CAO)

• Extends DPO by explicitly modeling and optimizing competing objectives using a multi-objective optimization approach.
• Incorporates:
  • Per-axiom loss functions that separately model each trade-off.
  • Global synergy function to balance overall alignment across axioms.
  • Synergy Jacobian (JS) to manage conflicting gradient updates dynamically.
  • Axiom-specific regularization using Sinkhorn-regularized Wasserstein Distance for scalable optimization.

3. YinYangAlign Dataset & Annotation Pipeline

• Leverages state-of-the-art T2I models (Stable Diffusion XL, Midjourney 6) to generate image samples.
• Uses a hybrid annotation strategy:
  • Automated annotation via GPT-4o and LLaVA for scalable alignment scoring.
  • Human verification (50,000+ samples) to ensure dataset reliability (Kappa score 0.83).
• Implements a multi-step verification process using:
  • Pairwise ranking via fine-tuned vision-language models.
  • Manual annotators for validation.
  • Consensus filtering with secondary validation from models like Prometheus-Vision.

4. Axiom-Specific Loss Formulations & Multi-Objective Aggregation

• Defines custom loss functions for each trade-off:

  • Faithfulness Loss: Measures semantic alignment using Sinkhorn-Wasserstein Distance.
  • Artistic Freedom Loss: Quantifies stylistic deviation via VGG-based Gram features and CLIP embeddings.
  • Emotional Impact vs. Neutrality Loss: Evaluates emotion intensity using DeepEmotion and neutrality scores.
  • Originality vs. Referentiality Loss: Uses CLIP Retrieval against WikiArt/BAM datasets.
  • Verifiability Loss: Measures real-world alignment via Google Image Search and DINO ViT embeddings.
  • Cultural Sensitivity Loss: Introduces Simulated Cultural Context Matching (SCCM) via dynamic LLM-generated sub-prompts.

• Implements Pareto-aware multi-objective control:

  • Balances trade-offs between contradictory alignment constraints.
  • Uses Sinkhorn regularization to smooth the optimization landscape.
  • Leverages Bradley-Terry preference models to incorporate user-driven trade-offs dynamically.

5. Empirical Evaluation: Comparison of DPO vs. CAO

• Demonstrates significant performance improvement of CAO over standard DPO:

  • Faithfulness to Prompt improved by 39% while maintaining artistic flexibility.
  • Cultural Sensitivity and Verifiability increased by 44% and 45%, mitigating misinformation risks.
  • Reduces the performance trade-offs observed in single-axiom DPO training.

• Weighted Alpha Generalization Analysis:

  • Evaluates whether alignment reduces generalizability.
  • Finds marginal overfitting (≤10%) but superior balance across alignment dimensions.

Key Takeaways & Future Directions

• CAO achieves state-of-the-art multi-objective alignment for T2I models, setting new benchmarks for ethical and aesthetic trade-offs.
• YinYangAlign dataset serves as a crucial benchmark for evaluating AI image alignment, addressing gaps in multimodal alignment research.
• Future work includes dynamic weight tuning for user-adaptive alignment and expanding CAO to handle emerging alignment challenges in AI-generated media.

This paper introduces a groundbreaking approach to alignment in generative AI, ensuring that T2I systems balance creativity, accuracy, and ethical considerations through multi-objective optimization. 🚀