PyTorch Conference 2024: Full Schedule

September 18-19, 2024
San Francisco, California
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for PyTorch Conference 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Daylight Time (UTC-7). To see the schedule in your preferred timezone, please select from the drop-down located at the bottom of the menu to the right.

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

9:00am PDT

Keynote: Welcome & Opening Remarks - Matt White, Executive Director, PyTorch Foundation

Wednesday September 18, 2024 9:00am - 9:10am PDT

Festival Pavilion - Keynote Room

Over the past few years, and especially since the deployment of ChatGPT in November 2022, neural language models with billions of parameters and trained on trillions of words are powering the fastest-growing computing applications in history and generating discussion and debate across society. However, AI scientists cannot study or improve those state-of-the-art models because the models' parameters, training data, code, and even documentation are not openly available. In this talk, I present our OLMo project toward building strong language models and making them fully open to researchers along with open-source code for data management, training, inference, and interaction. In particular, I describe DOLMa, a 3T token open dataset curated for training language models, Tulu, our instruction-tuned language model, and OLMo v1, a fully-open 7B parameter language model trained from scratch.

Speakers

Matt White

Executive Director, PyTorch Foundation. GM of AI., Linux Foundation

Matt White is the Executive Director of the PyTorch Foundation and GM of AI at the Linux Foundation. He is also the Director of the Generative AI Commons, an open community initiative focused on advancing responsible generative AI under the LF AI & Data Foundation. Matt has nearly... Read More →

Wednesday September 18, 2024 9:00am - 9:10am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:12am PDT

Keynote: PyTorch Technical Deep Dive - Piotr Bialecki, NVIDIA; Peng Wu, Will Constable, Kartikay Khandelwal & Mengtao (Martin) Yuan, Meta

Wednesday September 18, 2024 9:12am - 10:12am PDT

Festival Pavilion - Keynote Room

This Deep Dive provides an update on PyTorch development since last conference and dives into the key new features coming in PyTorch 2.5 and beyond. We will explore how advancements across a number of PyTorch features combine to better support the full model development lifecycle across training, fine-tuning, and deployment.

Speakers

Kartikay Khandelwal

Software Engineer, PyTorch, Meta

Kartikay Khandelwal is a software engineer in the PyTorch and AI Infra team at Meta where he leads the development of the PyTorch ecosystem for Generative AI, including open-source libraries like torchtune for LLM fine-tuning and torchchat for LLM inference. Prior to PyTorch, he worked... Read More →

Peng Wu

Engineering Manager, Meta

Dr. Peng Wu is the engineering manager of the PyTorch Compiler team at Meta. Dr. Wu spent over a decade at IBM research, working on many aspects of programming systems. She then founded the Programming Technologies Lab at Huawei and led its growth for six years. At Meta, she... Read More →

Will Constable

engineer, meta

Will Constable works on PyTorch Distributed Algorithms and Infrastructure at Meta as an IC and Tech Lead. Previously, he worked at Intel and Nervana Systems on different parts of the Deep Learning SW stack including Compiler Frontends, Integrations to TensorFlow and PyTorch, Distributed... Read More →

Mengtao (Martin) Yuan

Tech Lead Manager, Meta

Mengtao (Martin) Yuan is a Tech Lead Manager in Meta’s PyTorch Edge team. With multiple years of experience in the AI industry, Mengtao is focused at building software systems to help AI researchers and engineers to deploy their models on edge devices such as mobile phones, AR/VR... Read More →

Piotr Bialecki

Director of Engineering, Deep Learning Frameworks, NVIDIA

Piotr joined PyTorch team at NVIDIA in 2019 and currently manages the team. He drives NVIDIA's effort in maintaining and advancing PyTorch's CUDA backend and received the PyTorch SUPERHERO award in 2023 for his community contributions especially in the PyTorch discussion board... Read More →

PTC 2024 PyTorch Technical Keynote Slides pdf

Wednesday September 18, 2024 9:12am - 10:12am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

Slides Attached Yes

10:14am PDT

Keynote: Open Language Models (OLMo): Accelerating the Science of Language Modeling - Hanna Hajishirzi, Senior Director NLP Research, Allen Institute for AI

Wednesday September 18, 2024 10:14am - 10:29am PDT

Festival Pavilion - Keynote Room

Speakers

Hanna Hajishirzi

Associate Professor/Senior Director of NLP, UW/AI2

Hanna Hajishirzi is the Torode Family Associate Professor in the Allen School of Computer Science and Engineering at the University of Washington and a Senior Director of NLP at AI2. She received her Ph.D in Computer Science from University of Illinois at Urbana-Champaign, and spent... Read More →

Wednesday September 18, 2024 10:14am - 10:29am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

10:30am PDT

Keynote: Enabling Generative AI on the Edge - Cormac Brick, Principal Engineer, Google

Wednesday September 18, 2024 10:30am - 10:45am PDT

Festival Pavilion - Keynote Room

Generative AI is no longer just in the cloud - recently it's also getting deployed on edge devices. A disruptive goal of this work is AI-powered applications that respond instantly, work offline, and protect user privacy by processing data locally. In this talk, we'll explore the cutting edge of edge-based generative AI, showcasing open models that are pushing the boundaries of what's possible today on the edge. We'll dive deep into the PyTorch ecosystem, looking at projects that are making it easier than ever to author, optimize, and deploy these models across a wide range of devices.

Speakers

Cormac Brick

Principal Engineer, Core Machine Learning Software, Google

Cormac Brick is a principal Engineer at Google working on frameworks and on device machine learning. He has over 10 years experience in AI software, silicon and systems, with work spanning AI frameworks and ecosystems and compilers down to silicon microarchitecture. Over that... Read More →

Wednesday September 18, 2024 10:30am - 10:45am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:15pm PDT

Sponsored Keynote: The Lightning AI OSS Stack for Accelerating the AI Lifecycle - Luca Antiga, CTO, Lightning AI

Wednesday September 18, 2024 1:15pm - 1:20pm PDT

Festival Pavilion - Keynote Room

We introduce the Lightning AI open source stack, a high-performance stack for training, fine-tuning, and deploying AI systems that augments the PyTorch ecosystem.

Today PyTorch Lightning powers training workloads across the industry, from small-scale research to large-scale training endeavors. The package has reached 130M total downloads in June 2024, 2x since early 2023. PyTorch Lightning 2.4 features support for 2D parallelism via DTensors, first introduced in PyTorch 2.3.

The open source stack is completed by Fabric (lightweight building blocks for scaling training workloads), LitGPT (library for pre-training, fine-tuning, serving LLMs), LitData (parallel data processing and streaming data loading), LitServe (lightweight, high-performance serving framework), TorchMetrics (de-facto standard in deep learning metrics), and the recently released Thunder compiler. Together, these packages provide a low-friction, high-performance stack to democratize and accelerate the AI lifecycle.

The stack is optimized to run on Lightning Studios, a PyTorch native, fully integrated AI development environment on the cloud.

Speakers

Luca Antiga

CTO, Lightning AI

CTO @ Lightning AI, Founder (Orobix, Tensorwerk), early PyTorch core contributor, Manning Author (Deep Learning with PyTorch). PhD in Bioengineering.

Wednesday September 18, 2024 1:15pm - 1:20pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:20pm PDT

Sponsored Keynote: Enabling AI Everywhere with PyTorch and Intel - Kismat Singh,VP of Engineering for AI Frameworks, Intel

Wednesday September 18, 2024 1:20pm - 1:25pm PDT

Festival Pavilion - Keynote Room

Unlocking the availability of and access to generative AI technologies has great societal value. In this keynote, Kismat Singh will present how open software built on industry-standard frameworks such as PyTorch, and ubiquitous hardware from Intel that forms a large part of the current installed base across edge, PC and cloud are keys to democratizing AI and allowing new solutions to be implemented across industries ranging from healthcare, telecommunication, industrial and more. Kismat will share his thoughts on how software acceleration, flexibility and security are important factors in deploying AI applications in production and what he sees as challenges with those projects. He will also discuss Open Platform for Enterprise AI (OPEA), a new Linux Foundation AI and Data project that gives developers access to open source, standardized, modular, and heterogenous retrieval-augmented generation (RAG) pipelines that they can use for their enterprise-grade Generative AI deployments. Lastly, he will share some exciting Intel contributed features recently upstreamed into PyTorch. He will end the keynote by stating what he believes to be the future of AI and the part each of us will play in it!

Speakers

Kismat Singh

VP, Software, Intel Corporation

Kismat Singh is the VP of Engineering for AI Frameworks at Intel. He brings over two decades of AI experience and has also worked at companies such as Nvidia, AMD, HP and Stream Processors Inc. Kismat has made significant contributions to industry leading deep learning libraries... Read More →

Wednesday September 18, 2024 1:20pm - 1:25pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:30pm PDT

Sponsored Keynote: From Containers to Cognition: Conducting the AI Orchestra - Taylor Dolezal, Head of Ecosystem, Cloud Native Computing Foundation

Wednesday September 18, 2024 1:30pm - 1:35pm PDT

Festival Pavilion - Keynote Room

Let's explore the powerful harmony created when the CNCF and PyTorch communities join forces. This keynote highlights how the collaboration between cloud native experts and AI innovators is orchestrating a new era of technological symphonies. We'll touch on critical initiatives and shared victories that demonstrate the strength of this partnership. To illustrate the creative potential of this alliance, we'll briefly showcase a demo of how containerized workloads can produce unexpected melodies. Join us for this exploration of community-driven innovation, where containers and cognition come together to compose the future of technology.

Speakers

Taylor Dolezal

Head of Ecosystem, CNCF

Taylor Dolezal, Head of Ecosystem at CNCF, is an experienced technologist with a passion for cloud native technologies. He has a rich background in software development, infrastructure management, and open source and is deeply committed to community-building and knowledge sharing... Read More →

Wednesday September 18, 2024 1:30pm - 1:35pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:35pm PDT

Keynote Panel Discussion: Responsible AI - Kate Rooney, CNBC; Kush Varshney, IBM T. J. Watson Research Center; Sara Hooker, C4AI; Aleksander Madry, OpenAI; Rishi Bommasani, Stanford University

Wednesday September 18, 2024 1:35pm - 2:05pm PDT

Festival Pavilion - Keynote Room

Moderators

Kate Rooney

Technology Reporter, CNBC

Kate Rooney is a technology reporter based out of CNBC’s San Francisco bureau, covering Amazon, financial technology, payments and venture capital for the network. She also writes for CNBC’s digital platforms.Rooney won a National Headliner Award for her Celsius coverage in 2023... Read More →

Speakers

Sara Hooker

Head of Cohere For AI, Cohere For AI

Sara Hooker leads Cohere For AI, the dedicated research arm of Cohere. Cohere For AI seeks to solve complex machine learning problems and supports fundamental research that explores the unknown. With a long track-record of impactful research at Google Brain, Sara brings a wealth of... Read More →

Kush Varshney

IBM Fellow, IBM Research

Kush R. Varshney is an IBM Fellow based at the IBM T. J. Watson Research Center where he is responsible for leading innovations in AI governance. He and his team developed the well-known open-source toolkits AI Fairness 360, AI Explainability 360, and Uncertainty Quantification 360... Read More →

Aleksander Mądry

Member of Technical Staff, OpenAI

Aleksander Mądry is a Member of Technical Staff at OpenAI. Aleksander is also a Professor of Computing at MIT (currently on leave), where he has been serving as the Director of the MIT Center for Deployable Machine Learning and a Faculty Co-Lead of the MIT AI Policy Forum.

Rishi Bommasani

Society Lead, Stanford Center for Research on Foundation Models

I am the Society Lead at the Stanford Center for Research on Foundation Models (CRFM). I am completing my PhD at Stanford Computer Science, advised by Percy Liang and Dan Jurafsky. Funding: Lieberman Fellowship (active), NSF Graduate Research Fellowship (completed).Prior to St... Read More →

Wednesday September 18, 2024 1:35pm - 2:05pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:00am PDT

Keynote: Welcome Back & Opening Remarks

Thursday September 19, 2024 9:00am - 9:05am PDT

Festival Pavilion - Keynote Room

Thursday September 19, 2024 9:00am - 9:05am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:07am PDT

Keynote: Why You Should Think Twice Before Paying for an Evaluation Tool - Chip Huyen, VP of AI & OSS, Voltron Data

Thursday September 19, 2024 9:07am - 9:22am PDT

Festival Pavilion - Keynote Room

Open-ended evaluation is hard, and the number of evaluation tools has exploded in response to this challenge. However, if tools could solve evaluation, evaluation would have been solved by now. While the right tools can make your life easier, this talk discusses why you should think twice before outsourcing your evaluation to an external tool.

Speakers

Chip Huyen

VP of AI & OSS, Voltron Data

Chip Huyen works to accelerate data analytics on GPUs at Voltron Data. She also advises companies on building AI platforms. Previously, she was with Snorkel AI and NVIDIA, founded an AI infrastructure startup (acquired), and taught Machine Learning Systems Design at Stanford. She’s... Read More →

Thursday September 19, 2024 9:07am - 9:22am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:24am PDT

Keynote: Navigating the Architectural Timeline of LLMs - Sebastian Raschka, Staff Research Engineer, Lightning AI

Thursday September 19, 2024 9:24am - 9:39am PDT

Festival Pavilion - Keynote Room

The evolution of large language models (LLMs) from the original Generative Pre-trained Transformer (GPT) series to the recent advancements seen in models like Llama 3 has been accompanied by several architectural and methodological innovations. This talk aims to catch attendees up on the latest AI and LLM development trends, highlighting the key changes and motivations that led to the development of recent state-of-the-art LLMs, such as Llama 3.1.

Specifically, this presentation explores key developments in attention mechanisms, such as sliding window attention, group query, multi-query attention, and FlashAttention, and explains their key motivations and advantages. In addition to exploring the structural changes, this presentation also reviews the recent "tricks of the trade" that have improved the training processes and performance of the latest LLMs. This includes the recent two-step pretraining approach in Llama 3.1 and applying knowledge distillation techniques using real datasets like Gemma 2 and synthetic data, as seen in Llama 3.1.

Moreover, we will also examine the integration of system-level optimizations, such as the Mixture of the Expert method and the hybrid model Samba, which combines Mamba techniques with attention mechanisms and illustrates a broader trend toward more specialized and efficient architectures.

This talk will provide attendees with an understanding of the most notable transformations that have defined the architectural timeline of LLMs.

Speakers

Sebastian Raschka

Staff Research Engineer, Lightning AI

PhD

Thursday September 19, 2024 9:24am - 9:39am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:41am PDT

Keynote: Building an Advanced Knowledge Assistant - Jerry Liu, Co-Founder & CEO, LlamaIndex

Thursday September 19, 2024 9:41am - 9:56am PDT

Festival Pavilion - Keynote Room

A huge promise for LLMs is being able to answer questions and solve tasks of arbitrary complexity over an arbitrary number of data sources. The world has started to shift from simple RAG stacks, which are mostly good for answering pointed questions, to agents that can more autonomously reason over a diverse set of inputs, and interleave retrieval and tool use to produce sophisticated outputs.

Building a reliable multi-agent system is challenging. There's a core question of developer ergonomics and production deployment - what makes sense outside a notebook setting. In this talk we outline some core building blocks for building advanced research assistants, including advanced RAG modules, event-driven workflow orchestration, and more.

Speakers

Jerry Liu

CEO, LlamaIndex

Jerry is the co-founder/CEO of LlamaIndex, the data framework for building LLM applications. Before this, he has spent his career at the intersection of ML, research, and startups. He led the ML monitoring team at Robust Intelligence, did self-driving AI research at Uber ATG and worked... Read More →

Thursday September 19, 2024 9:41am - 9:56am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

9:58am PDT

Keynote: Ray: A Distributed Framework for Heterogeneous Computing - Ion Stoica, Professor, UC Berkeley

Thursday September 19, 2024 9:58am - 10:13am PDT

Festival Pavilion - Keynote Room

Ray has recently become the framework of choice for scaling machine learning workloads—from data preprocessing, to training, fine-tuning, and serving. This talk will highlight Ray’s key features responsible for its flexibility and generality, as well as its recent support for GPUs.

Speakers

Ion Stoica

Professor, UC Berkeley

Ion Stoica is a Professor in the EECS Department at the University of California at Berkeley, and the Director of Sky Computing Lab (https://sky.cs.berkeley.edu/). He is currently doing research on cloud computing and AI systems. Past work includes Ray, Apache Spark, Apache Mesos, Tachyon, Chord DHT, and Dynamic Packet State (DPS). He is an Honorary Member of the Romanian Academy, an ACM Fellow and has received numerous awards, including the Mark Weiser Award (2019... Read More →

Thursday September 19, 2024 9:58am - 10:13am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

10:15am PDT

Keynote: Contributor Awards

Thursday September 19, 2024 10:15am - 10:25am PDT

Festival Pavilion - Keynote Room

Thursday September 19, 2024 10:15am - 10:25am PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:25pm PDT

Sponsored Keynote: Accelerating AI: How AMD and PyTorch Drive Innovation with Seamless Day-0 Support and High Performance - Anush Elangovan, CVP Software Development, AMD

Thursday September 19, 2024 1:25pm - 1:30pm PDT

Festival Pavilion - Keynote Room

In this keynote presentation, we explore the robust collaboration between AMD and PyTorch that is propelling advancements in artificial intelligence and machine learning. Discover how AMD's commitment to Day-0 PyTorch support ensures that PyTorch users benefit from cutting-edge performance enhancements and out-of-the-box compatibility. We delve into the technical synergies that make AMD hardware an ideal choice for PyTorch frameworks, showcasing real-world examples of accelerated workflows and breakthrough AI applications. Join us to learn how this dynamic partnership is enabling researchers, developers, and data scientists to push the boundaries of innovation and achieve unprecedented results in their AI projects.

Speakers

Anush Elangovan

Vice President - AI Software, AMD

Thursday September 19, 2024 1:25pm - 1:30pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:32pm PDT

Sponsored Keynote: Optimizing AI Inference for Large Language Models - Mudhakar Srivatsa, Distinguished Engineer, IBM

Thursday September 19, 2024 1:32pm - 1:37pm PDT

Festival Pavilion - Keynote Room

This talk will cover two new ways IBM has optimized generative AI inferencing with PyTorch: speculative decoding and Triton kernel development. Speculative decoding leverages predictive modeling to reduce latency by anticipating potential outputs, streamlining the inference process without sacrificing accuracy. IBM Research's team developed new speculative architectures and open sourced speculators for LLama3 models. It will also discuss various Triton kernels to accelerate inference, one of which was contributed to vLLM for accelerating MoE models. Finally, it will share a glimpse of IBM's AI hardware work, including how the IBM Artificial Intelligence Unit (AIU) could integrate into the PyTorch stack.

Speakers

Mudhakar Srivatsa

Distinguished Engineer, IBM Research

Mudhakar Srivatsa is a distinguished research staff member at the Distributed Cloud department in IBM T. J. Watson Research Center. His work is focussed on heterogeneous spatiotemporal data with applications to edge computing, AIOps and Hybrid AI Scaling. He is an IBM master inv... Read More →

Thursday September 19, 2024 1:32pm - 1:37pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions

1:40pm PDT

Keynote Panel Discussion: Scaling & Benchmarking - Anastasios Nikolas Angelopoulos, UC Berkeley/LMSYS; Lisa Dunlap, UC Berkeley; James Bradbury, Anthropic; Tri Dao, together.ai; Aparna Ramani & Soumith Chintala, Meta

Thursday September 19, 2024 1:40pm - 2:10pm PDT

Festival Pavilion - Keynote Room

Moderators

Soumith Chintala

VP/Fellow of Meta & Co-Creator of PyTorch

I am an Artificial Intelligence researcher, engineer and community builder.I am currently at Meta, jumping between Engineering, Research and Leadership as I find convenient. I also visit NYU as a part-time researcher.My career interests have been defined by two sets of work: AI Platforms/Ecosystems... Read More →

Speakers

James Bradbury

Software Engineer, Anthropic

James is Head of Compute at Anthropic, where he is focused on ensuring that the company has the accelerator resources it needs to pursue its mission, and that the resources can be used effectively and efficiently across the organization. He joined in 2023 from Google DeepMind, where... Read More →

Lisa Dunlap

Student, UC Berkeley

PhD student at UC Berkeley working on (1) interpreting and evaluating generative models and (2) automating data science on unstructured data using large multimodal modelsAlso an underwhelming nail enthusiast and reader of old psychiatry books.

Tri Dao

Assistant Professor at Princeton University, Chief Scientist of Together AI, Princeton University, Together AI

Tri Dao is an Assistant Professor at Princeton University and chief scientist of Together AI. He completed his PhD in Computer Science at Stanford, co-advised by Christopher Ré and Stefano Ermon. He works at the intersection of machine learning and systems, and his research highlights... Read More →

Aparna Ramani

VP Engineering, Meta

Aparna is VP Engineering at Meta, responsible for AI Infrastructure, Data Infrastructure and Developer Infrastructure. Over the last eight years at Meta, Aparna has built a world-class team that is responsible for some of the largest scale systems on the planet - to process exabyte-scale... Read More →

Anastasios Nikolas Angelopoulos

Researcher, UC Berkeley/LMSYS

Thursday September 19, 2024 1:40pm - 2:10pm PDT
Festival Pavilion - Keynote Room

Keynote Sessions