PyTorch Conference 2024: Full Schedule

September 18-19, 2024
San Francisco, California
View More Details & Registration
Note: The schedule is subject to change.

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for PyTorch Conference 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

This schedule is automatically displayed in Pacific Daylight Time (UTC-7). To see the schedule in your preferred timezone, please select from the drop-down located at the bottom of the menu to the right.

IMPORTANT NOTE: Timing of sessions and room locations are subject to change.

11:10am PDT

Meta Llama 3 and the Future of Responsible AI Development - Spencer Whitman & Vincent Gonguet, Meta

Wednesday September 18, 2024 11:10am - 11:35am PDT

Gateway Pavilion - Cowell Theater

As AI models become increasingly powerful and pervasive, trust and safety have become top priorities. Join us for a timely talk on Llama 3, our latest foundation model, and the cutting-edge trust and safety models and tools we've developed to ensure responsible AI development. In this talk, we'll dive into: •The advancements of Llama 3 and its applications •Our innovative trust and safety approaches, including toxicity detection and mitigation •The open-source tools and resources we're sharing to empower the community Discover how Meta is pushing the boundaries of trust and safety and learn how you can integrate these solutions into your own projects. Let's build a safer, more responsible AI future together!

Speakers

Spencer Whitman

Product Manager (AI Security), Meta

Vincent Gonguet

Director, GenAI Trust & Safety, Meta

Meta Trust and safety pytorch conf24 pdf

Wednesday September 18, 2024 11:10am - 11:35am PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Any
Slides Attached Yes

11:40am PDT

Building Scientific Computing Infrastructure Software with the PyTorch Ecosystem - Bharath Ramsundar, Deep Forest Sciences

Wednesday September 18, 2024 11:40am - 12:05pm PDT

Gateway Pavilion - Cowell Theater

The DeepChem library is a scientific computing library that implements deep learning infrastructure for drug discovery, materials discovery, and biology. The DeepChem community is one of the largest scientific open source projects built in PyTorch, with over 5K stars on Github and thousands of citations. The DeepChem community has learned a number of useful lessons for building and maintaining high quality scientific code built on top of PyTorch. In this talk, I will share our learnings with the PyTorch community and also highlight opportunities for improving scientific support in the ecosystem.

Speakers

Bharath Ramsundar

CEO, Deep Forest Sciences

Bharath received a BA and BS from UC Berkeley in EECS and Mathematics and was valedictorian of his class in mathematics. He received his PhD in computer science from Stanford where he founded the DeepChem project. Bharath is founder and CEO of Deep Forest Sciences, a startup building... Read More →

2024 9 16 Linux Foundation PyTorch pdf

Wednesday September 18, 2024 11:40am - 12:05pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Any

2:10pm PDT

The Impact and Challenges of Open Source Generative Datasets and Models - Aaron Gokaslan, Cornell University

Wednesday September 18, 2024 2:10pm - 2:35pm PDT

Gateway Pavilion - Cowell Theater

Open source generative models like OpenGPT2, BLOOM, and others have been pivotal in advancing AI technology. These models leverage extensive text data to achieve advanced linguistic capabilities. However, the trend towards proprietary tools and closed large language models is growing, posing unique challenges in open-source AI development. This discussion will explore the intricacies of training such models, the hurdles in dataset management, and the regulation of open-source contributions. We'll explore how to effectively iterate on collected data, prepare for extensive training sessions, and coordinate research across large open-source organizations. We will discuss the challenges of generative models in three different modalities: text, image, and genomics. The talk will draw from the speaker’s personal experience on working on OpenWebText, OpenGPT2, BLOOM, CommonCanvas, Caduceus, and other generative models. We will also cover the changing AI environment and how the future of open souce is threatened by onerous regulation, ever increasing compute costs, and the commoditization of previously open data.

Speakers

Aaron Gokaslan

PhD Student, Cornell University

Aaron Gokaslan has worked on many popular generative models and datasets such as OpenWebText, CommonCanvas, BLOOM, DBRX, and Caduceus, collectively downloaded millions of times. His work on open source has earned him a Community Contributor Award at PyTorch Con and recognition from... Read More →

PyTorchCon 2024 pdf

Wednesday September 18, 2024 2:10pm - 2:35pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Intermediate

2:40pm PDT

Lightning Talk: Beyond Zero: Eliminating Vulnerabilities in PyTorch Container Images - Patrick Smyth, Dan Fernandez & Srishti Hegde, Chainguard

Wednesday September 18, 2024 2:40pm - 2:50pm PDT

Gateway Pavilion - Cowell Theater

Container images are increasingly the future of production applications at scale, providing reproducibility, robustness, and transparency. As PyTorch images get deployed to production, however, security becomes a major concern. PyTorch has a large attack surface, and building secure PyTorch images can be a challenge. Currently, the official PyTorch runtime container image has 30 CVEs (known vulnerabilities) rated critical and 256 CVE rated high. Improving this situation could secure many deployments that incorporate PyTorch for cloud-based inference or training. In this fast-paced session, we'll take a deep dive on the official PyTorch image from a vulnerability mitigation perspective, looking hard at included packages, executables, and active CVE. We'll identify low-hanging fruit for increasing security, including stripping bloat and building fresh. We'll also talk about the next level of security practiced in Chainguard's PyTorch image builds, such as including SBOMs and going distroless. Finally, we'll consider emerging tools and approaches for analyzing AI artifacts such as models and how these systems can benefit PyTorch in production.

Speakers

Dan Fernandez

Staff Product Manager, Chainguard

Dan is a Management Information Systems graduate from Florida's FIU and recently completed his Master of Cybersecurity at the Georgia Institute of Technology. He is currently focusing on securing the software supply chain at Chainguard. In his free time, he enjoys writing about analytics... Read More →

Patrick Smyth

Staff Developer Relations Engineer, Chainguard

Dr. Patrick Smyth is Staff Developer Relations Engineer at Chainguard, where he shows developers how to deploy AI and other applications with 0 CVEs using Chainguard Images. Patrick has a PhD in the digital humanities and in a previous life led technical bootcamps for researchers... Read More →

Srishti Hegde

Software Engineer, Chainguard

Wednesday September 18, 2024 2:40pm - 2:50pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

2:55pm PDT

Lightning Talk: Sparsifying Vision Transformers with Minimal Accuracy Loss - Jesse Cai, Meta

Wednesday September 18, 2024 2:55pm - 3:05pm PDT

Gateway Pavilion - Cowell Theater

Sparsity, like quantization, is an approximate model optimization technique, where we trade some model accuracy for increased performance.

In this talk we'll explore how to minimize the accuracy degradation of sparsifying Vision Transformer (ViT) based models to GPU accelerable sparsity patterns like block sparsity and semi-structured sparsity.

We'll cover the best techniques to ensure a < 5% loss in accuracy when:
- training a sparse model from scratch
- pruning and retraining an existing dense model
- zero-shot/one-shot pruning a dense model

We've collected these techniques into a single repository, torchao, so that model optimization enthusiasts like you can sparsify your models with just a few lines of code.

Speakers

Jesse Cai

Software Engineer, Meta

Jesse is a software engineer on the PyTorch Core Performance team, where he works on accelerating models with sparsity. Before joining Meta, he worked at several startups, focusing on natural language processing.

Sparsifying ViT lightning talk slides pdf

Wednesday September 18, 2024 2:55pm - 3:05pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

3:10pm PDT

Lightning Talk: PyTorch/XLA Auto-Sharding - Yeounoh Chung, Google

Wednesday September 18, 2024 3:10pm - 3:20pm PDT

Gateway Pavilion - Cowell Theater

PyTorch/XLA recently launched the new PyTorch/XLA SPMD feature as a first-step to automate ML workloads parallelization using GSPMD. It turns out that the performance largely depends on the quality of sharding hints provided by the user – and it requires a correct and deep understanding of model architectures and much expertise to come up with optimal sharding hints. To address this problem, we propose to integrate PyTorch/XLA SPMD with XLA's auto sharding service that allows the XLA compiler to shard and optimize the whole model without any user input.

Speakers

Yeounoh Chung

Software Engineer, Google

SystemsResearch@Google

pytorchconf24 autosharding pdf

Wednesday September 18, 2024 3:10pm - 3:20pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

3:25pm PDT

Lightning Talk: Introduction to Torch.Distributed.Pipelining - Howard Huang & Ke Wen, Meta

Wednesday September 18, 2024 3:25pm - 3:35pm PDT

Gateway Pavilion - Cowell Theater

Pipeline parallelism is a technique employed in distributed deep learning that enhances model execution by dividing the model into distinct segments, or "stages." As large language models and other memory-intensive models become more common, pipeline parallelism has grown increasingly important for several key areas: - Executing large-scale training jobs. - Enhancing performance in bandwidth-limited clusters. - Supporting large model inference. In this talk, we will introduce the `torch.distributed.pipelining` package which provides users a seamless way of applying pipeline parallelism. We will demonstrate the following features: - Splitting of model code based on simple specification. - Support for pipeline schedules, including GPipe, 1F1B, Interleaved 1F1B and Looped BFS, and providing the infrastructure for writing customized schedules. - Composability with other PyTorch parallel techniques such as data parallel (DDP, FSDP) or tensor parallel. - Out of the box integration with Hugging Face models for efficient inference.

Speakers

Howard Huang

Software Engineer, Meta

Howard Huang is a software engineer at Meta. He has been working on PyTorch and the PyTorch distributed team for the past 4 years.

Ke Wen

Software Engineer, Meta

Ke Wen is a software engineering at Meta. He works on PyTorch Distributed features, including pipeline parallelism, distributed inference, and graph-based analysis.

Introduction to torch.distributed.pipelining pdf

Introduction to torch.distributed.pipelining pptx

Wednesday September 18, 2024 3:25pm - 3:35pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate
Slides Attached Yes

4:00pm PDT

Lightning Talk: On-Device Profiling and Debugging with ExecuTorch - Olivia Liu & Vaun Puri, Meta

Wednesday September 18, 2024 4:00pm - 4:10pm PDT

Gateway Pavilion - Cowell Theater

High developer velocity is crucial to shipping new ML-enabled experiences from a server-trained model to a customers’ device. ExecuTorch is an on-device runtime that seamlessly integrates with the PyTorch stack with a focus on developer productivity. We present the ExecuTorch Dev Tools and highlight key features that tighten the iteration loop when optimizing models for deployment and execution on edge devices. We demonstrate how ExecuTorch’s built-in profiler and bundled tools tackle key pain-points, such as: 1. Examining the memory footprint of an ExecuTorch program ahead-of-time; 2. Collecting runtime performance metrics and intermediate outputs for accuracy analysis; 3. Correlating runtime data with the underlying graph of an exported model.

Speakers

Olivia Liu

Software Engineer, Meta

Olivia has been worked on PyTorch at Meta for over 2 years, focusing on on-device inference and building out profiling and debugging tools for model developers.

Varun Puri

On Device Profiling and Debugging with ExecuTorch PyTorch Conference 2024 Olivia Liu, Varun Puri pdf

Wednesday September 18, 2024 4:00pm - 4:10pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

4:15pm PDT

Lightning Talk: In-Transit Machine Learning Using PyTorch on Frontier Exascale System - Vineeth Gutta, University of Delaware

Wednesday September 18, 2024 4:15pm - 4:25pm PDT

Gateway Pavilion - Cowell Theater

Traditional ML workflows use offline training where the data is stored on disk and is subsequently loaded into accelerator (CPU,GPU, etc) memory during training or inference. We recently devised a novel and scalable in-transit ML workflow for a plasma-physics application (chosen as 1 out of 8 compelling codes in the country) for the world’s fastest supercomputer, Frontier) with an aim to build a high-energy laser particle accelerator. Data generated in distributed HPC systems like Frontier create volumes of data that is infeasible to store on HPC file systems. A mismatch between modern memory hierarchies occurs due to high volume and rate of data generation. Our novel ML workflow utilizes continuous learning where the data is consumed in batches as the simulation produces the data and then discards after each batch is trained. This in-transit workflow integrates particle-in-cell simulations with distributed ML training on PyTorch using DDP allows for an application coupling enabling the model to learn correlations between emitted radiation and particle dynamics within simulation in an unsupervised method. This workflow is demonstrated at scale on Frontier using 400 AMD MI250X GPUs

Speakers

Vineeth Gutta

PhD Student, University of Delaware

Vineeth is a fifth-year PhD student in the department of Computer and Information Sciences at the University of Delaware. He is a member of the Computational Research and Programming Lab (CRPL) and is advised by Dr. Sunita Chandrasekaran. His research interests lie at the intersection of High Performance Computing and Machine Learning. He works with the National Cancer Institute (NCI/NIH) on improving drug response and drug discovery models. Currently working on improving the AMPL model that predicts binding free energy of ligand-protein dock... Read More →

pytorch conf24 ppt final pdf

Wednesday September 18, 2024 4:15pm - 4:25pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Advanced

4:30pm PDT

A Distributed Stateful Dataloader for Large-Scale Pretraining - Davis Wertheimer, IBM & Linsong Chu, IBM Research

Wednesday September 18, 2024 4:30pm - 4:55pm PDT

Gateway Pavilion - Cowell Theater

Large-scale model pretraining crucially relies on specialized and dedicated dataloaders that can, for example, partition and stream data asynchronously across multiple processes and physical nodes. In this talk we discuss one of the torch-native dataloaders we built and use at IBM Research for addressing these needs. Intended for use in large-scale model pretraining, particularly in research settings where rapid iteration between datasets may be required, our dataloader is distributed, stateful, checkpointable, composable and rescalable – while remaining a simple extension of the existing PyTorch dataloading framework. It automatically and invisibly handles data sharding, shuffling, subdataset weighting, checkpoint saving and loading, and custom user-defined preprocessing functions, with minimal overhead and high throughput. We discuss these properties and how we achieved them, such as reducing overhead by implementing a custom LCG random number generator, and demonstrate proof of concept on production-scale training of a 7B parameter Llama model over 4 trillion tokens.

Speakers

Davis Wertheimer

Staff Research Scientist, IBM

Davis Wertheimer earned his Ph.D. in Computer Science at Cornell University in 2022, conducting research under Bharath Hariharan on few-shot learning and machine learning under constraints. He now researches and develops AI models for IBM, training and accelerating large language... Read More →

LINSONG CHU

Senior Technical Staff Member, IBM Research

Linsong is a STSM at IBM Research, focusing on FSDP, torch compile and FP8 in the area of pre-training.

pytorch conf 2024 data loading pdf

Wednesday September 18, 2024 4:30pm - 4:55pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Any

5:00pm PDT

Pushing the Performance Envelope: An Optimization Study for 3D Generative Modelling with PyTorch - Suvaditya Mukherjee & Shireen Chand, University of Southern California

Wednesday September 18, 2024 5:00pm - 5:25pm PDT

Gateway Pavilion - Cowell Theater

This work explores performance optimization strategies for training 3D generative models using PyTorch. We focus on training Variational Autoencoders (VAEs) on the ShapeNet dataset, a popular benchmark for this task. Our objective is to achieve high-fidelity reconstructions while minimizing the computational footprint and training time. We focus on: 1) Large-scale 3D dataset loading strategies using PyTorch & Google Cloud Storage Buckets 2) Implementation details and insights for 3D VAEs using PyTorch 2.x 3) Training using Automatic Mixed-precision regimes 4) Optimized training using torch.compile and different quantization techniques (as supported) - Dynamic Quantization - Static Quantization - Static Quantization-aware Training 5) Comparative Benchmark over several experiments performed with a focus on execution time and memory footprint Through this comprehensive study, we present a comparative analysis of the performance gains achieved by our optimized models. Our findings present empirical insights into the trade-offs between model accuracy, computational complexity, and hardware resource utilization.

Speakers

Shireen Chand

Student, University of Southern California

Shireen is a Masters student at the University of Southern California. She is majoring in Artificial Intelligence. She is also a Machine Learning Developer, a Google Summer of Code Contributor, and a Technical Writer for Medium.

Suvaditya Mukherjee

MS AI @ USC | ML GDE, University of Southern California

Suvaditya is a Masters student at the University of Southern California, majoring in Artificial Intelligence. He is also a Google Developer Expert for Machine Learning, and an external author at PyImageSearch. He likes to work on problems related to Computer Vision, VLMs, 3D Reconstruction... Read More →

3D Generative Modelling Optimization Study pdf

Wednesday September 18, 2024 5:00pm - 5:25pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Any

10:50am PDT

Sponsored Session: Democratizing AI: Powering the Future with Arm’s Global Compute Ecosystem - Gian Marco Iodice, Arm

Thursday September 19, 2024 10:50am - 11:15am PDT

Gateway Pavilion - Cowell Theater

Arm is excited to be at the center of the world's largest compute ecosystem at the dawn of the AI era. A key tenant of our mission is to democratize AI capabilities, empowering millions of developers to put advanced AI features into the hands of billions of users.

In this presentation, we'll explore how Arm is enabling the world’s leading open-source AI frameworks to leverage power-efficient Arm-based computing platforms and Arm architecture features, as a tool for enabling fast and secure AI workloads. The session focuses on how our strategic partnership with the Pytorch and Executorch community is enabling a seamless and transparent developer experience, to run workloads everywhere from cloud to edge. This session will highlight some of our optimized libraries, upstreamed contributions and a wealth of AI-related developer material to build the future of AI on Arm.

Speakers

Gian-Marco Iodice

GenAI Engineering Lead, Arm

Gian Marco Iodice is an experienced edge and mobile computing specialist at Arm for machine learning (ML) and leads engineering development for on-device GenAI. He received the MSc with honors in electronic engineering from the University of Pisa (Italy), where he specialized in HW/SW... Read More →

Thursday September 19, 2024 10:50am - 11:15am PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

11:20am PDT

Lightning Talk: Building and Supporting the Chinese PyTorch Community: Resources, Tutorials, and Engagement - Zong Zesheng, Huawei

Thursday September 19, 2024 11:20am - 11:30am PDT

Gateway Pavilion - Cowell Theater

Description: This proposal aims to provide a comprehensive introduction to the Chinese PyTorch community, we hope to inspire more users to join and contribute, fostering a vibrant and inclusive environment for PyTorch enthusiasts in China. Chinese PyTorch Homepage Introduction to the official Chinese version of the PyTorch website, highlighting its features. Navigation tips and key sections, such as documentation, tutorials, and community events. Improve the connection of users from China with PyTorch Community. Localized Tutorials and Documentation The 2.x version not have Translated version, it hard to catch up with latest features of PyTorch if the beginner not good at English. We translated official documents and tutorials, covering everything from basic PyTorch concepts to advanced applications. Interactive tutorials No interactive tutorials(Like Google Colab) for Chinese students or beginners before, they have to setup environment before start with PyTorch, which might be hard for beginners. And now, an online notebook & tutorials are available to practice or tuning steps for beginners.

Speakers

zong zesheng

Software Engineer, Huawei

Currently, trying to let Chinese users to have easier access to PyTorch resources and make a friendly user experiences for beginners.

lightningtalk zezheng zong pdf

Thursday September 19, 2024 11:20am - 11:30am PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Beginner
Slides Attached Yes

11:35am PDT

Lightning Talk: Distributing a Million Open Models in the Wild: Lessons Learned from the Hugging Face Hub - Omar Sanseviero, Hugging Face

Thursday September 19, 2024 11:35am - 11:45am PDT

Gateway Pavilion - Cowell Theater

The Hugging Face Hub has over 300,000 PyTorch models. Distributing such number of models poses challenges. In this talk, Omar will share how the community has tackled these challenges, including techniques to ensure torch model security and tooling for researchers to share their models. He'll also take attendees on a journey through the evolution of torch models distributed by the community, highlighting new trends and directions. Attending this talk will give attendees practical insights into the latest developments in model distribution and ecosystem trends.

Speakers

Omar Sanseviero

Chief Llama Officer - Head of Platform and Community, Hugging Face

Omar Sanseviero is the Chief Llama Officer and Head of Platform and Community at Hugging Face, where he works at the intersection of open source, community, and product. Omar leads multiple ML teams that work on topics such as Mobile ML, ML for art, and ML Partnerships. Previously... Read More →

Distributing a Million Open Models in the Wild Lessons Learned from the Hugging Face Hub pdf

Thursday September 19, 2024 11:35am - 11:45am PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Beginner

11:50am PDT

Lightning Talk: Understanding and Optimizing PyTorch Models with Thunder - Luca Antiga, Lightning AI

Thursday September 19, 2024 11:50am - 12:00pm PDT

Gateway Pavilion - Cowell Theater

A hallmark feature of PyTorch is the natural expression of computation. This enables practitioners to implement AI models with ease. However, it prompts the question how to optimize the workload for a given hardware setup because those optimizations clutter our code and are tricky to combine. Lightning Thunder provides a Python-to-Python compiler to scale and optimize PyTorch programs that focuses on usability, understandability, and extensibility. A key tool in delivering on these goals is the composability of transformations: without changing the user code, we can stack quantization, distributing the computation across multiple GPUs, dispatching to optimized kernels, offloading, and other pluggable optimizations. Lightning Thunder flourishes in the PyTorch ecosystem: with PyTorch eager and with executors like torch.compile and nvFuser. It also dispatches to libraries like cuDNN, TransformerEngine, Apex, OpenAI Triton. The ability to apply multiple optimizations just-in-time leads to significant compounded speed-ups over unoptimized code out of the box. Luca will discuss the design of Thunder and demonstrate applications on training and inference for large language and multimodal models.

Speakers

Luca Antiga

CTO, Lightning AI

CTO @ Lightning AI, Founder (Orobix, Tensorwerk), early PyTorch core contributor, Manning Author (Deep Learning with PyTorch). PhD in Bioengineering.

Thursday September 19, 2024 11:50am - 12:00pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Any

12:00pm PDT

Lightning Talk: Fast, Scalable Distributed Training with StreamingDataset - Saaketh Narayan, Databricks

Thursday September 19, 2024 12:00pm - 12:10pm PDT

Gateway Pavilion - Cowell Theater

StreamingDataset makes training on large datasets from cloud storage as fast, cheap, and scalable as possible. It’s specially designed for multi-node, distributed training for large models — maximizing correctness guarantees, performance, and ease of use. Key features include elastically deterministic training, instant mid-epoch resumption, effective shuffling, high training throughput, and flexible data mixing, among other features. When training with StreamingDataset, the data shards are written to cloud storage in MDS, our file format that allows for low-latency random access to samples. By being as efficient as possible with shard downloads and shuffling, StreamingDataset minimizes egress costs while ensuring that dataloading never bottlenecks model training. StreamingDataset powers training for LLMs with over 100 billion parameters like DBRX, to advanced diffusion models, to two-tower recommendation models, and more, scaling to training jobs on thousands of GPUs with ease. Join us to learn how StreamingDataset can elevate your distributed model training experience.

Speakers

Saaketh Narayan

Machine Learning Engineer, Databricks

Saaketh Narayan is a machine learning engineer at Databricks. As part of the Mosaic AI Runtime team, he works on the GenAI training stack, including dataloading, training frameworks, and performance across the Mosaic Streaming, Composer, and LLM Foundry libraries.

StreamingDataset PyTorchCon Presentation (1) pdf

Thursday September 19, 2024 12:00pm - 12:10pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate

12:10pm PDT

Lightning Talk: Implementing and Using Iterable Datasets: What Could Go Wrong? - Nicolas Hug, Meta

Thursday September 19, 2024 12:10pm - 12:20pm PDT

Gateway Pavilion - Cowell Theater

PyTorch supports two kinds of datasets: Iterable datasets and indexable "map-style" datasets. Iterable datasets can be more flexible and potentially faster than their indexable cousins. They are also much harder to use correctly, and can easily lead to silently wrong results. This talk is a quick and fun intro to some of the traps that Iterable datasets lay out for you, with some tips to help you avoid them.

Speakers

Nicolas Hug

Research Engineer, Meta

Nicolas is a software engineer in the PyTorch team at Meta, where he mainly contributes to the torchvision library. Prior to that, Nicolas was a research scientist at Columbia University, where he became part of the scikit-learn core development team. Nicolas holds a PhD in machine... Read More →

What could go wrong pdf

Thursday September 19, 2024 12:10pm - 12:20pm PDT
Gateway Pavilion - Cowell Theater

Lightning Talks

Audience Intermediate
Slides Attached Yes

2:15pm PDT

Building PyTorch Computer Vision Algorithms for 100 Skin Shades - Emmanuel Acheampong, roboMUA

Thursday September 19, 2024 2:15pm - 2:40pm PDT

Gateway Pavilion - Cowell Theater

At roboMUA we're leading the charge in building predictive AI models for diverse skin shades with the use of Convolutional Neural Networks (CNNs), and harnessing the power of Generative Adversarial Networks (GANs) specifically for generating realistic images of black hairstyles. Our session showcases PyTorch's versatility in both predictive and generative tasks, offering a comprehensive approach to inclusive AI. For predictive AI models, we leverage PyTorch's flexible framework to develop CNNs. Through innovative techniques in feature engineering and model architecture design, we demonstrate how PyTorch enables accurate prediction across 100 skin shades. Simultaneously, we showcase the transformative potential of GANs in the realm of black hairstyles. By training GANs on a curated dataset of diverse hair textures and styles, we illustrate how PyTorch facilitates the generation of lifelike images that celebrate the beauty and diversity of black hair. Attendees will gain insights into the data preprocessing, model training, and evaluation processes and and learn how PyTorch empowers developers to build inclusive solutions.

Speakers

Emmanuel Acheampong

CEO / Head of AI, yShade.ai (formerly roboMUA)

Emmanuel Acheampong is a co-founder and CEO of roboMUA - an innovative AI solutions company with a visionary focus on catering to all skin shades and types. He graduated from Notre Dame’s ESTEEM program with a Masters thesis on the intersection of Artificial Intelligence and directed... Read More →

Thursday September 19, 2024 2:15pm - 2:40pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Beginner

2:45pm PDT

Blobs to Clips: Efficient End-to-End Video Data Loading - Andrew Ho & Ahmad Sharif, Meta

Thursday September 19, 2024 2:45pm - 3:10pm PDT

Gateway Pavilion - Cowell Theater

The PyTorch team has improved training speed by an order of magnitude for teams at Meta working on Small-to-Large-Scale MultiModal Video models. In this talk we’ll share our learnings on reducing GPU starvation by overcoming data loading challenges such as dealing with large distributed datasets, worker imbalance, compute-bottlenecks due to parallel video decoding and sampling, checkpointing, and debuggability. As part of our commitment to open-source, we are releasing a new decoding library and updating existing PyTorch libraries on GitHub, and invite feedback and contributions from the community.

Speakers

Ahmad Sharif

Software Engineer, Meta

SWE in Pytorch Content Domains Past: SWE at Google in Search, Privacy, ChromeOS

Andrew Ho

Machine Learning Engineer, Meta Platforms

We are ML Engineers at Meta on PyTorch working on multi-modal LLM dataloading

Blobs to Clips Efficient End to End Video Data Loading pdf

Thursday September 19, 2024 2:45pm - 3:10pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

Audience Intermediate

3:15pm PDT

Sponsored Session: PyTorch Support by Google Enabling Performance from Cloud to Edge - Mark Sherwood & Shauheen Zahirazami, Google

Thursday September 19, 2024 3:15pm - 3:40pm PDT

Gateway Pavilion - Cowell Theater

In this session we will cover various ways teams at google are working to help the Pytorch community achieve performance and scale from cloud to edge. We will cover how Google Cloud customers can use PyTorch and OpenXLA to get competitive performance for their ML workloads. We’ll also cover how Google AI Edge Torch works with Pytorch to help developers integrate LLMs, vision models and more to easily create new edge applications that can run on a wide set of devices.

Speakers

Mark Sherwood

Senior Product Manager, Google AI Edge, Google

Mark is a Senior Product Manager on the Google AI Edge team, responsible for LiteRT (formerly known as TensorFlow Lite) and MediaPipe. He specializes in shipping ML powered features on Android, iOS, and Web using the very smallest to the very largest on-device models.

Shauheen Zahirazami

Senior Staff Engineering Manager, Cloud Machine Learning Compute Services, Google

Shauheen has a PhD in control engineering with a BSc in applied mathematics. He is currently leading Cloud TPU Machine Learning teams at Google who are responsible for ML Frameworks and 3P ecosystem including the PyTorch teams that develop PyTorch/XLA.

PyTorch Conf 2024 PyTorch Support by Google Enabling Performance from Cloud to Edge pdf

Thursday September 19, 2024 3:15pm - 3:40pm PDT
Gateway Pavilion - Cowell Theater

Breakout Sessions

4:05pm PDT

Startup Showcase

Thursday September 19, 2024 4:05pm - 5:30pm PDT

Gateway Pavilion - Cowell Theater

The PyTorch Conference Startup Showcase is giving emerging companies the chance to pitch to a panel of VCs looking to support AI/ML startups with high growth potential, and meet some of the best AI focused Engineers in the Industry. This is an exciting and unique opportunity for early-stage founders to showcase their ideas and breakthroughs, connect with leading VCs, and increase visibility in the generative AI and machine learning industry.

The winning startup will be announced at the Flare Party taking place after the Startup Showcase.

Finalists

Moderators

Chappy Asel

Co-founder, GenAI Collective

Successful entrepreneur with an expansive technical and operational background built across 10+ years of experience. Co-founder of the GenAI Collective: a community of founders, funders, and thought leaders built around our shared curiosity for generative AI. Ex-Apple AR/VR. Ex-Apple... Read More →

Judges

Astasia Myers

General Partner, Felicis

Astasia Myers is a General Partner at Felicis. Before joining Felicis, she was an enterprise partner at Quiet Capital and an investor at Redpoint Ventures. Astasia focuses on early-stage investing across AI, data, open source, developer tools, and security. She has invested in LaunchDarkly... Read More →

Kevin Crosby

Sr. Director, Open Source Funding, GitHub

Kevin Crosby is Senior Director leading Open Source Funding at Microsoft’s M12 Github fund. Prior to GitHub, Kevin led business development for VC and Accelerators at Carta and spent 8 years at Amazon in corporate venture and leading product, engineer, and business teams. He is... Read More →

Rajko Radovanovic

Investor, Andreessen Horowitz

Rajko Radovanovic is an investing partner on the infrastructure team at Andreessen Horowitz.

Simon Tiu

VC Investor, Vertex Ventures

Simon Tiu joined Vertex Ventures US in 2024, focusing on enterprise software and cybersecurity investments. Prior to joining Vertex Ventures, Simon worked at Qatalyst Partners, where he was a core member of the Enterprise Software team. During his tenure, he provided strategic and... Read More →

Vig Sachidananda

Investor, Gradient Ventures

Vig is an investor at Gradient Ventures.Vig received his M.S., Ph.D in Electrical Engineering from Stanford University and his B.S. in Mechanical Engineering from the University of Maryland, College Park. During his Ph.D, he worked as a seed stage software engineer at Clockwork.io... Read More →

Vijay Reddy

Partner, Mayfield Fund

Vijay Reddy brings over a decade of inception and early-stage investing experience in AI and Enterprise infrastructure. He had a front-row seat to the rise of AI and has invested across the AI stack from silicon, infrastructure, data, middleware and AI-first applications. Vijay is... Read More →

Thursday September 19, 2024 4:05pm - 5:30pm PDT
Gateway Pavilion - Cowell Theater

Startup Showcase