Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:1905.10650

Papers - Attention - Induction Heads

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3
Are Sixteen Heads Really Better than One?

Paper • 1905.10650 • Published May 25, 2019 • 2

Papers - XAI - Attention - Induction Heads

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3
Are Sixteen Heads Really Better than One?

Paper • 1905.10650 • Published May 25, 2019 • 2

Papers - Ablation - Attention - Head Pruning

Causal ablations taking into account LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3
Are Sixteen Heads Really Better than One?

Paper • 1905.10650 • Published May 25, 2019 • 2

Papers - Attention - Multi-Head Attention (MHA)

Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings

Paper • 2305.13571 • Published May 23, 2023 • 2
Transformers Can Represent n-gram Language Models

Paper • 2404.14994 • Published Apr 23, 2024 • 19
Are Sixteen Heads Really Better than One?

Paper • 1905.10650 • Published May 25, 2019 • 2
Reasoning in Large Language Models: A Geometric Perspective

Paper • 2407.02678 • Published Jul 2, 2024 • 1

Papers - Interpretability - Attention

A Multiscale Visualization of Attention in the Transformer Model

Paper • 1906.05714 • Published Jun 12, 2019 • 2
Are Sixteen Heads Really Better than One?

Paper • 1905.10650 • Published May 25, 2019 • 2

Papers - University - Carnegie Mellon University

Can large language models explore in-context?

Paper • 2403.15371 • Published Mar 22, 2024 • 32
Long-context LLMs Struggle with Long In-context Learning

Paper • 2404.02060 • Published Apr 2, 2024 • 36
PIQA: Reasoning about Physical Commonsense in Natural Language

Paper • 1911.11641 • Published Nov 26, 2019 • 2
AQuA: A Benchmarking Tool for Label Quality Assessment

Paper • 2306.09467 • Published Jun 15, 2023 • 1

LIMA: Less Is More for Alignment

Paper • 2305.11206 • Published May 18, 2023 • 23
Garment3DGen: 3D Garment Stylization and Texture Generation

Paper • 2403.18816 • Published Mar 27, 2024 • 23
EgoLifter: Open-world 3D Segmentation for Egocentric Perception

Paper • 2403.18118 • Published Mar 26, 2024 • 12
The Unreasonable Ineffectiveness of the Deeper Layers

Paper • 2403.17887 • Published Mar 26, 2024 • 79

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

Paper • 2310.17157 • Published Oct 26, 2023 • 13
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

Paper • 2305.15805 • Published May 25, 2023 • 1
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt

Paper • 2305.11186 • Published May 17, 2023 • 1
Composable Sparse Fine-Tuning for Cross-Lingual Transfer

Paper • 2110.07560 • Published Oct 14, 2021 • 1

Efficient Memory Management for Large Language Model Serving with PagedAttention

Paper • 2309.06180 • Published Sep 12, 2023 • 25
LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models

Paper • 2308.16137 • Published Aug 30, 2023 • 40
Scaling Transformer to 1M tokens and beyond with RMT

Paper • 2304.11062 • Published Apr 19, 2023 • 2
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

Paper • 2309.14509 • Published Sep 25, 2023 • 18

Company

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs