Name: Slaying OOMs - Mark Saroufim & Jane Xu, Meta
Start: 2024-09-19T15:00:00-0700
End: 2024-09-19T15:25:00-0700

September 18-19, 2024
San Francisco, California
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for PyTorch Conference 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Daylight Time (UTC-7). To see the schedule in your preferred timezone, please select from the drop-down located at the bottom of the menu to the right.

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

Thursday September 19, 2024 3:00pm - 3:25pm PDT

Room B

Have you ever hit an OOM (and wished you had more VRAM)? Who hasn't! Hop on the bus with us and feel the road become smoother as we talk about stacking together techniques like FSDP2 + QLoRa + CPU Offloading + Fused ADAM (thanks Intel) + more in PyTorch native. We will give an overview of these techniques as well as the hard edges we solved in their composition. Curious for more? Or...still OOMing? We also plan on discussing our more researchy work on offloading, pagedness, and low precision optimizers.

Speakers

Jane Xu

PyTorch SWE, Meta

I'm Jane and I focus on making our optimizers more...optimal :) in terms of stability, consistency, and performance. My favorite part of PyTorch is the people--they're all smart and cool and fun to learn from! I also like potatoes quite a lot.

Mark Saroufim

Software Engineer, Meta

Mark Saroufim is a PyTorch Engineer at Meta working on inference, compilers and community.

Thursday September 19, 2024 3:00pm - 3:25pm PDT
Room B

Breakout Sessions

Audience Intermediate

PyTorch Conference 2024

Jane Xu

Mark Saroufim

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!