關於使用條款隱私政策聯絡
 
更新中
Technically Speaking with Chris Wright

Technically Speaking with Chris Wright

發行日期:2025-08-06
Technically Speaking with Chris Wright - QR碼
4 Episodes
音訊
立即收聽,就在 Apple Podcast
4 Episodes
音訊
立即收聽,就在 Apple Podcast
發行日期:2025-08-06
最新的單集
Inside distributed inference with llm-d ft. Carlos Costa

Inside distributed inference with llm-d ft. Carlos Costa

Scaling LLM inference for production isn't just about adding more machines, it demands new intelligence in the infrastructure itself. In this episode, we're joined by Carlos Costa, Distinguished Engineer at IBM Research, a leader in large-scale compute
時間長度:26:23
Scaling LLM inference for production isn't just about adding more machines, it demands new intelligence in the infrastructure itself. In this episode, we're joined by Carlos Costa, Distinguished Engineer at IBM Research, a leader in large-scale compute and a key figure in the llm-d project. We discuss how to move beyond single-server deployments and build the intelligent, AI-aware infrastructure needed to manage complex workloads efficiently.
Carlos Costa shares insights from his deep background in HPC and distributed systems, including:
• The evolution from traditional HPC and large-scale training to the unique challenges of distributed inference for massive models.
• The origin story of the llm-d project, a collaborative, open-source effort to create a much-needed ""common AI stack"" and control plane for the entire community.
• How llm-d extends Kubernetes with the specialization required for AI, enabling state-aware scheduling that standard Kubernetes wasn't designed for.
• Key architectural innovations like the disaggregation of prefill and decode stages and support for wide parallelism to efficiently run complex Mixture of Experts (MOE) models.
Tune in to discover how this collaborative, open-source approach is building the standardized, AI-aware infrastructure necessary to make massive AI models practical, efficient, and accessible for everyone.
單集識別碼:1000720834457
GUID:c7ce4f9e-8f07-406d-883c-04436cd6fcfa
發佈日期:2025/8/6 下午12:00:00

說明

Struggling to keep pace with the ever-changing world of technology? For experienced tech professionals, making sense of this complexity to find real strategic advantages is key. This series offers a clear path, featuring insightful, casual conversations with leading global experts, innovators, and key voices from Red Hat, all cutting through the hype.
Drawing from Red Hat's deep expertise in open source and enterprise innovation, each discussion delves into new and emerging technologies-- from artificial intelligence and the future of cloud computing to cybersecurity, data management, and beyond. The focus is on understanding not just the 'what,' but the important 'why' and 'how': exploring how these advancements can shape long-term strategic developments for your organization and your career. Gain an insider’s perspective that humanizes complex topics, helping you anticipate what’s next and make informed decisions. Equip yourself with the knowledge to turn today's emerging tech into valuable, practical strategies and apply innovative thinking in your work.
Tune in for forward-looking discussions that connect the dots between cutting-edge technology and real-world application, leveraging a rich understanding of the enterprise landscape. Learn to navigate the future of tech with confidence.

Apple Podcasts:評論

無條目