PyTorch Conference 2024

Open-ended evaluation is hard, and the number of evaluation tools has exploded in response to this challenge. However, if tools could solve evaluation, evaluation would have been solved by now. While the right tools can make your life easier, this talk discusses why you should think twice before outsourcing your evaluation to an external tool.

The evolution of large language models (LLMs) from the original Generative Pre-trained Transformer (GPT) series to the recent advancements seen in models like Llama 3 has been accompanied by several architectural and methodological innovations. This talk aims to catch attendees up on the latest AI and LLM development trends, highlighting the key changes and motivations that led to the development of recent state-of-the-art LLMs, such as Llama 3.1.

Specifically, this presentation explores key developments in attention mechanisms, such as sliding window attention, group query, multi-query attention, and FlashAttention, and explains their key motivations and advantages. In addition to exploring the structural changes, this presentation also reviews the recent "tricks of the trade" that have improved the training processes and performance of the latest LLMs. This includes the recent two-step pretraining approach in Llama 3.1 and applying knowledge distillation techniques using real datasets like Gemma 2 and synthetic data, as seen in Llama 3.1.

Moreover, we will also examine the integration of system-level optimizations, such as the Mixture of the Expert method and the hybrid model Samba, which combines Mamba techniques with attention mechanisms and illustrates a broader trend toward more specialized and efficient architectures.

This talk will provide attendees with an understanding of the most notable transformations that have defined the architectural timeline of LLMs.

A huge promise for LLMs is being able to answer questions and solve tasks of arbitrary complexity over an arbitrary number of data sources. The world has started to shift from simple RAG stacks, which are mostly good for answering pointed questions, to agents that can more autonomously reason over a diverse set of inputs, and interleave retrieval and tool use to produce sophisticated outputs.

Building a reliable multi-agent system is challenging. There's a core question of developer ergonomics and production deployment - what makes sense outside a notebook setting. In this talk we outline some core building blocks for building advanced research assistants, including advanced RAG modules, event-driven workflow orchestration, and more.

Ray has recently become the framework of choice for scaling machine learning workloads—from data preprocessing, to training, fine-tuning, and serving. This talk will highlight Ray’s key features responsible for its flexibility and generality, as well as its recent support for GPUs.

In this keynote presentation, we explore the robust collaboration between AMD and PyTorch that is propelling advancements in artificial intelligence and machine learning. Discover how AMD's commitment to Day-0 PyTorch support ensures that PyTorch users benefit from cutting-edge performance enhancements and out-of-the-box compatibility. We delve into the technical synergies that make AMD hardware an ideal choice for PyTorch frameworks, showcasing real-world examples of accelerated workflows and breakthrough AI applications. Join us to learn how this dynamic partnership is enabling researchers, developers, and data scientists to push the boundaries of innovation and achieve unprecedented results in their AI projects.

This talk will cover two new ways IBM has optimized generative AI inferencing with PyTorch: speculative decoding and Triton kernel development. Speculative decoding leverages predictive modeling to reduce latency by anticipating potential outputs, streamlining the inference process without sacrificing accuracy. IBM Research's team developed new speculative architectures and open sourced speculators for LLama3 models. It will also discuss various Triton kernels to accelerate inference, one of which was contributed to vLLM for accelerating MoE models. Finally, it will share a glimpse of IBM's AI hardware work, including how the IBM Artificial Intelligence Unit (AIU) could integrate into the PyTorch stack.