{"article":{"annotation":{"article_id":"ibm-distributed-inference-llmd-explainer-2026","available_at":"2026-10-15T22:00:57.281968Z","geography_revision":"ef6ee0010dda5fe7b240fd514749b43e214c57da73222d2e40ec83b276dd0977","id":"67848197fb1258a0f587d74151c2a00384b6c158169e7c7132f93b7b7c58e50b","input_sha256":"5b428fff99f7ede3fc56352c75a9718f0792e4061386c18d61cbdf673758860b","limitations":["Rule-derived subjects are candidates, not verified editorial labels or calibrated model probabilities.","Coverage is distinct from event location. Unknown country/city/coordinates remain absent; no automatic geocoding.","Article geometries keep their stated precision and methodology; reproducing them is not independent coordinate verification."],"method":"controlled-rules-v1","places":[],"rules_sha256":"fb5ad3c27c91cb840f7a0a05f7f64a5cb26ad0374409ff9457f277c7092e22b4","schema_version":1,"subjects":[{"concept_id":"medtop:20000260","confidence":null,"confidence_method":null,"evidence":[{"field":"en.title+en.summary","language":"en","note":"Subject suggested from the title or summary: KV","source_urls":["https://research.ibm.com/blog/distributed-inference"]}],"method":"controlled-rules-v1","rule_id":"electricity","status":"candidate"},{"concept_id":"medtop:20001298","confidence":null,"confidence_method":null,"evidence":[{"field":"en.title+en.summary","language":"en","note":"Subject suggested from the title or summary: large language models","source_urls":["https://research.ibm.com/blog/distributed-inference"]}],"method":"controlled-rules-v1","rule_id":"artificial-intelligence","status":"candidate"}],"taxonomy_revision":"1f2db31d811fca07e99b16d402c10e2cc2965c9df1c7b41d9be348b40a24e7fb","taxonomy_version":"IPTC Media Topics 2026-Q2"},"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=en","category":"technology","content":{"key_points":[],"paragraphs":["IBM Research described distributed inference through llm-d, an open-source, Kubernetes-native framework that coordinates groups of large-language-model servers rather than balancing each server in isolation. The explainer covers routing requests to servers that already hold shared prefixes, moving KV cache to CPU memory or storage, prioritizing request flow, pacing batch jobs, separating prefill from decode, and distributing mixture-of-experts components across nodes.","IBM reports that one llm-d demonstration captured 97% of available prefix reuse in its tested workload; it also cites wide-expert-parallelism demonstrations with lower expert-weight memory per GPU and three to nine times the KV-cache capacity, depending on topology. These figures belong to demonstrations cited by the company; the post is not an independent benchmark study."],"sections":[]},"corrections":[],"country":null,"data":{"blocks":[{"description":{"ar":"تلخص مادة توضيحية من IBM Research بتاريخ 12 أكتوبر أساليب في إطار llm-d، منها التوجيه الواعي بالذاكرة المؤقتة وإخراج ذاكرة KV والتحكم في التدفق وفصل مراحل الاستدلال. وتعود أرقام الأداء إلى عروض llm-d التي يذكرها المنشور.","de":"Ein IBM-Research-Erklärbeitrag vom 12. Oktober fasst Methoden des llm-d-Frameworks zusammen, darunter cachebewusstes Routing, KV-Cache-Auslagerung, Flusssteuerung und Phasentrennung. Leistungswerte stammen aus den im Beitrag genannten llm-d-Demonstrationen.","en":"An IBM Research explainer dated October 12 outlines methods in the llm-d framework, including cache-aware routing, KV-cache offloading, flow control and phase disaggregation. Performance figures are attributed to llm-d demonstrations cited in the post.","es":"Un explicador de IBM Research del 12 de octubre resume métodos del marco llm-d, como enrutamiento consciente de la caché, descarga de la caché KV, control de flujo y separación de fases. Las cifras de rendimiento corresponden a demostraciones de llm-d citadas en la publicación.","fr":"Un explicatif d’IBM Research du 12 octobre présente des méthodes du cadre llm-d, dont le routage tenant compte du cache, le déchargement du cache KV, le contrôle des flux et la séparation des phases. Les chiffres de performance proviennent des démonstrations llm-d citées dans l’article.","pt":"Um explicador da IBM Research de 12 de outubro resume métodos do framework llm-d, como roteamento com reconhecimento de cache, descarregamento do cache KV, controle de fluxo e separação de fases. Os números de desempenho são atribuídos às demonstrações do llm-d citadas no texto.","tr":"IBM Research’in 12 Ekim tarihli açıklaması, llm-d çerçevesinde önbelleğe duyarlı yönlendirme, KV önbelleği boşaltma, akış kontrolü ve aşama ayrıştırma gibi yöntemleri özetliyor. Verimlilik rakamları, yazıda anılan llm-d gösterimlerine aittir."},"id":"reported-event","items":[{"date":"2026-10-12","description":{"ar":"تلخص مادة توضيحية من IBM Research بتاريخ 12 أكتوبر أساليب في إطار llm-d، منها التوجيه الواعي بالذاكرة المؤقتة وإخراج ذاكرة KV والتحكم في التدفق وفصل مراحل الاستدلال. وتعود أرقام الأداء إلى عروض llm-d التي يذكرها المنشور.","de":"Ein IBM-Research-Erklärbeitrag vom 12. Oktober fasst Methoden des llm-d-Frameworks zusammen, darunter cachebewusstes Routing, KV-Cache-Auslagerung, Flusssteuerung und Phasentrennung. Leistungswerte stammen aus den im Beitrag genannten llm-d-Demonstrationen.","en":"An IBM Research explainer dated October 12 outlines methods in the llm-d framework, including cache-aware routing, KV-cache offloading, flow control and phase disaggregation. Performance figures are attributed to llm-d demonstrations cited in the post.","es":"Un explicador de IBM Research del 12 de octubre resume métodos del marco llm-d, como enrutamiento consciente de la caché, descarga de la caché KV, control de flujo y separación de fases. Las cifras de rendimiento corresponden a demostraciones de llm-d citadas en la publicación.","fr":"Un explicatif d’IBM Research du 12 octobre présente des méthodes du cadre llm-d, dont le routage tenant compte du cache, le déchargement du cache KV, le contrôle des flux et la séparation des phases. Les chiffres de performance proviennent des démonstrations llm-d citées dans l’article.","pt":"Um explicador da IBM Research de 12 de outubro resume métodos do framework llm-d, como roteamento com reconhecimento de cache, descarregamento do cache KV, controle de fluxo e separação de fases. Os números de desempenho são atribuídos às demonstrações do llm-d citadas no texto.","tr":"IBM Research’in 12 Ekim tarihli açıklaması, llm-d çerçevesinde önbelleğe duyarlı yönlendirme, KV önbelleği boşaltma, akış kontrolü ve aşama ayrıştırma gibi yöntemleri özetliyor. Verimlilik rakamları, yazıda anılan llm-d gösterimlerine aittir."},"label":{"ar":"تاريخ نشر المصدر","de":"Veröffentlichungsdatum der Quelle","en":"Source publication date","es":"Fecha de publicación de la fuente","fr":"Date de publication de la source","pt":"Data de publicação da fonte","tr":"Kaynak yayın tarihi"},"status":"occurred"}],"methodology":{"ar":"تُنسب الادعاءات والتفاصيل التقنية إلى المصدر الأولي المشار إليه؛ وهذا ليس اختباراً مستقلاً.","de":"Aussagen und technische Angaben werden der zitierten Primärquelle zugeschrieben; dies ist kein unabhängiger Test.","en":"Claims and technical details are attributed to the cited primary source; this is not independent testing.","es":"Las afirmaciones y los detalles técnicos se atribuyen a la fuente primaria citada; no son pruebas independientes.","fr":"Les affirmations et détails techniques sont attribués à la source primaire citée ; il ne s’agit pas de tests indépendants.","pt":"As afirmações e os detalhes técnicos são atribuídos à fonte primária citada; isto não é um teste independente.","tr":"İddialar ve teknik ayrıntılar atıf yapılan birincil kaynağa dayanmaktadır; bu bağımsız bir test değildir."},"sourceUrls":["https://research.ibm.com/blog/distributed-inference"],"title":{"ar":"IBM تشرح أساليب الاستدلال الموزع للنماذج اللغوية الكبيرة","de":"IBM erklärt verteilte Inferenzverfahren für große Sprachmodelle","en":"IBM explains distributed inference methods for large language models","es":"IBM explica métodos de inferencia distribuida para modelos de lenguaje grandes","fr":"IBM explique les méthodes d’inférence distribuée pour les grands modèles de langage","pt":"IBM explica métodos de inferência distribuída para grandes modelos de linguagem","tr":"IBM, büyük dil modelleri için dağıtık çıkarım yöntemlerini açıklıyor"},"type":"events","updatedAt":"2026-10-15T21:44:05+00:00"}],"version":1},"data_downloads":[{"formats":{"csv":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026/data/reported-event?lang=en&format=csv","json":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026/data/reported-event?lang=en&format=json"},"id":"reported-event"}],"data_types":["events"],"editorial_illustration":{"alt":"Conceptual illustration of technology, processors and connected information.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"en","scope":"topic","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"event_id":"event-3eb3dcbe-d7dd-5060-a4bc-1d6962ec3285","event_period":"2026-10-12","event_period_basis":"source-review","id":"ibm-distributed-inference-llmd-explainer-2026","image":null,"importance":0.6,"language":"en","modified_at":"2026-10-15T22:00:39+00:00","publication_basis":"publication-record","published_at":"2026-10-15T22:00:39+00:00","reading_minutes":1,"region":"world","section_illustration":{"alt":"Conceptual illustration of technology, processors and connected information.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"en","scope":"section","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"slug":"ibm-distributed-inference-llmd-explainer-2026","sources":[{"checkedAt":"2026-10-15","name":"IBM Research","publishedAt":"2026-10-12","url":"https://research.ibm.com/blog/distributed-inference"}],"summary":"An IBM Research explainer dated October 12 outlines methods in the llm-d framework, including cache-aware routing, KV-cache offloading, flow control and phase disaggregation. Performance figures are attributed to llm-d demonstrations cited in the post.","text":"An IBM Research explainer dated October 12 outlines methods in the llm-d framework, including cache-aware routing, KV-cache offloading, flow control and phase disaggregation. Performance figures are attributed to llm-d demonstrations cited in the post.\n\nIBM Research described distributed inference through llm-d, an open-source, Kubernetes-native framework that coordinates groups of large-language-model servers rather than balancing each server in isolation. The explainer covers routing requests to servers that already hold shared prefixes, moving KV cache to CPU memory or storage, prioritizing request flow, pacing batch jobs, separating prefill from decode, and distributing mixture-of-experts components across nodes.\n\nIBM reports that one llm-d demonstration captured 97% of available prefix reuse in its tested workload; it also cites wide-expert-parallelism demonstrations with lower expert-weight memory per GPU and three to nine times the KV-cache capacity, depending on topology. These figures belong to demonstrations cited by the company; the post is not an independent benchmark study.","title":"IBM explains distributed inference methods for large language models","topics":[],"translations":[{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=tr","language":"tr","url":"https://newsverum.com/tr/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=en","language":"en","url":"https://newsverum.com/en/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=ar","language":"ar","url":"https://newsverum.com/ar/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=fr","language":"fr","url":"https://newsverum.com/fr/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=es","language":"es","url":"https://newsverum.com/es/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=de","language":"de","url":"https://newsverum.com/de/news/ibm-distributed-inference-llmd-explainer-2026"},{"api_url":"https://newsverum.com/api/v1/articles/ibm-distributed-inference-llmd-explainer-2026?lang=pt","language":"pt","url":"https://newsverum.com/pt/news/ibm-distributed-inference-llmd-explainer-2026"}],"url":"https://newsverum.com/en/news/ibm-distributed-inference-llmd-explainer-2026"},"version":"1"}