Spaces:
Running
title: README
emoji: 📉
colorFrom: gray
colorTo: blue
sdk: static
pinned: false
All Things ViTs: Understanding and Interpreting Attention in Vision (CVPR'23 tutorial)
By: Hila Chefer and Sayak Paul
Website: all-things-vits.github.io/atv/
Abstract: The attention mechanism has revolutionized deep learning research across many disciplines, starting from NLP and expanding to vision, speech, and more. Different from other mechanisms, the elegant and general attention mechanism is easily adaptable and eliminates modality-specific inductive biases. As attention become increasingly popular, it is crucial to develop tools that allow researchers to understand and explain the inner workings of the mechanism to facilitate better and more responsible use of it. This tutorial focuses on understanding and interpreting attention in the vision, and in the multi-modal settings combining text and vision. We present state-of-the-art research on representation probing, interpretability, and attention-based semantic guidance, alongside hands-on demos to facilitate interactivity. Additionally, we discuss open questions arising from recent works and future research directions.
This organization hosts all the interactive demos to be presented at the tutorial. Below, you can find some of them.