Name: Lightning Talk: Sparsifying Vision Transformers with Minimal Accuracy Loss - Jesse Cai, Meta
Start: 2024-09-18T14:55:00-0700
End: 2024-09-18T15:05:00-0700

September 18-19, 2024
San Francisco, California
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for PyTorch Conference 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Daylight Time (UTC-7). To see the schedule in your preferred timezone, please select from the drop-down located at the bottom of the menu to the right.

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

Wednesday September 18, 2024 2:55pm - 3:05pm PDT

Gateway Pavilion - Cowell Theater

Sparsity, like quantization, is an approximate model optimization technique, where we trade some model accuracy for increased performance.

In this talk we'll explore how to minimize the accuracy degradation of sparsifying Vision Transformer (ViT) based models to GPU accelerable sparsity patterns like block sparsity and semi-structured sparsity.

We'll cover the best techniques to ensure a < 5% loss in accuracy when:
- training a sparse model from scratch
- pruning and retraining an existing dense model
- zero-shot/one-shot pruning a dense model

We've collected these techniques into a single repository, torchao, so that model optimization enthusiasts like you can sparsify your models with just a few lines of code.

Speakers

Jesse Cai

Software Engineer, Meta

Jesse is a software engineer on the PyTorch Core Performance team, where he works on accelerating models with sparsity. Before joining Meta, he worked at several startups, focusing on natural language processing.

Sparsifying ViT lightning talk slides pdf

Wednesday September 18, 2024 2:55pm - 3:05pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

PyTorch Conference 2024

Jesse Cai

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!