Loading…
Attending this event?
September 18-19, 2024
San Francisco, California
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for PyTorch Conference 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Daylight Time (UTC-7). To see the schedule in your preferred timezone, please select from the drop-down located at the bottom of the menu to the right.

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

← Back to schedule
Panel Discussion - Tim Dettmers, AI2/Carnegie Melon; Hailey Schoelkopf, EleutherAI; Aakanksha Chowdhery, Meta; Alexis Conneau, OpenAI; Moderated by Kartikay Khandelwal, Meta
Wednesday September 18, 2024 4:45pm - 5:30pm PDT
Festival Pavilion - Breakout Room A
As open-source LLMs have become more capable, a substantial ecosystem has developed around the fine-tuning of these models. A thriving community of researchers, developers, practitioners and hobbyists has emerged which focuses on topics ranging from memory efficiency, parameter-efficient fine-tuning and quantization to performance at scale and reproducible evaluations. The goal of this mini-summit is to bring this community together to discuss ideas, share knowledge and build connections.

The agenda features a keynote from Joe Spisak on the state of the Llama ecosystem followed by invited talks from the founders of Axolotl, Unsloth and torchtune. We conclude the summit with a riveting discussion on what’s next for LLMs, fine-tuning and the PyTorch ecosystem with a fabulous panel of experts - Tim Dettmers (author of bitsandbytes and QLoRA), Hailey Schoelkopf (maintainer of LM Eval Harness at EleutherAI), Aakanksha Chowdhery (Lead author on PaLM and Gemini) and Alexis Conneau (Research Lead at OpenAI)
Moderators
avatar for Kartikay Khandelwal

Kartikay Khandelwal

Software Engineer, PyTorch, Meta
Kartikay Khandelwal is a software engineer in the PyTorch and AI Infra team at Meta where he leads the development of the PyTorch ecosystem for Generative AI, including open-source libraries like torchtune for LLM fine-tuning and torchchat for LLM inference. Prior to PyTorch, he worked... Read More →
Speakers
TD

Tim Dettmers

Research Scientist & Assistant Professor, Ai2 & Carnegie Mellon University
Tim Dettmers’s is a research scientist at AI2 and an incoming assistant professor at CMU. His research focuses on making foundation models, such as ChatGPT, accessible to researchers and practitioners by reducing their resource requirements. This involves developing novel compression... Read More →
HS

Hailey Schoelkopf

EleutherAI
Hailey Schoelkopf is a Research Scientist at EleutherAI, a non-profit research lab focused on enabling open science on large-scale AI models. Her research has focused on building reproducible infrastructure for empowering open science on large-scale models, with core interests in... Read More →
AC

Aakanksha Chowdhery

Research Scientist, Meta
Aakanksha has been a lead researcher in pre-training large language models, such as PaLM and Gemini. She led the 540B PaLM model at Google and was a core member of the Gemini, Pathways, PaLM-E and MedPaLM projects. Before Google, she led interdisciplinary teams at Microsoft Research... Read More →
AC

Alexis Conneau

Member of Technical Staff, OpenAI
Alexis Conneau is a Member of Technical Staff at OpenAI in the Multimodal Frontiers team, where he has led the research for Audio+Text language modeling and the next-generation GPT4-o VoiceMode ("Her"). Prior to that, he was a research scientist at Facebook AI Research since 2015... Read More →
Wednesday September 18, 2024 4:45pm - 5:30pm PDT
Festival Pavilion - Breakout Room A
  Fine Tuning Mini-Summit
Log in to leave feedback.

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Share Modal

Share this link via

Or copy link