{"article":{"annotation":{"article_id":"gpu-tokenization-agentic-llm-serving-graphtok-2026","available_at":"2026-10-13T01:40:25.761594Z","geography_revision":"ef6ee0010dda5fe7b240fd514749b43e214c57da73222d2e40ec83b276dd0977","id":"f0313c359dc695e7c13f04814effcff2134b7608a877b9ee3c17e888b03d9f97","input_sha256":"752ddd38f81259a5a70d0dd89375a5751bd48779c681a4c9e94f57f5280dfaf5","limitations":["Rule-derived subjects are candidates, not verified editorial labels or calibrated model probabilities.","Coverage is distinct from event location. Unknown country/city/coordinates remain absent; no automatic geocoding.","Article geometries keep their stated precision and methodology; reproducing them is not independent coordinate verification."],"method":"controlled-rules-v1","places":[{"admin1":null,"city":null,"coordinate_note":null,"coordinate_reference_system":"OGC:CRS84","coordinate_source":null,"country":"SE","evidence":[{"field":"country","language":"en","note":"Explicit preserved editorial country/region; no city or national centroid inferred","source_urls":["https://arxiv.org/abs/2610.13325"]}],"geometry":null,"id":"iso3166-1:SE","kind":"country-or-area","labels":{"ar":"السويد","en":"Sweden","es":"Suecia","fr":"Suède"},"match_status":"candidate","precision":"regional","role":"coverage","uncertainty_m":null}],"rules_sha256":"fb5ad3c27c91cb840f7a0a05f7f64a5cb26ad0374409ff9457f277c7092e22b4","schema_version":1,"subjects":[{"concept_id":"medtop:20001298","confidence":null,"confidence_method":null,"evidence":[{"field":"en.title+en.summary","language":"en","note":"Subject suggested from the title or summary: AI","source_urls":["https://arxiv.org/abs/2610.13325"]}],"method":"controlled-rules-v1","rule_id":"artificial-intelligence","status":"candidate"}],"taxonomy_revision":"1f2db31d811fca07e99b16d402c10e2cc2965c9df1c7b41d9be348b40a24e7fb","taxonomy_version":"IPTC Media Topics 2026-Q2"},"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=de","category":"technology","content":{"key_points":[],"paragraphs":["KI-Agenten senden frühere Gespräche und Werkzeugausgaben mit jeder neuen Anfrage erneut. Ein Präfix-Cache kann Modellberechnungen für bereits verarbeiteten Text einsparen, doch die gesamte Anfrage muss vor der Cache-Abfrage in Token-IDs umgewandelt werden. GraphTok erledigt dies mit einem CUDA-Graphen auf der GPU und leitet kurze Eingaben an die CPU weiter.","In einem Dynamo-Frontend-Test verarbeitete GraphTok bei 121.000 Token Anfrageverkehr 518 Anfragen pro Sekunde innerhalb eines P99-Ziels von zwei Sekunden bis zum ersten Token; gigatoken erreichte 296 und der Standard-Tokenizer 18,5 Anfragen pro Sekunde. Die GPU-Worker wurden simuliert und die Korrektheit nur für getestete Tokenizer-Konfigurationen geprüft; die Werte sind keine allgemeine Kapazitätsangabe."],"sections":[]},"corrections":[],"country":"SE","data":{"blocks":[{"columns":[{"key":"context_tokens","label":{"ar":"عدد رموز السياق","de":"Anzahl der Kontexttokens","en":"Context token count","es":"Número de tokens del contexto","fr":"Nombre de jetons du contexte","pt":"Número de tokens de contexto","tr":"Bağlam belirteç sayısı"},"type":"number","unit":"count"},{"key":"load","label":{"ar":"نطاق الحمل المستدام","de":"Umfang der nachhaltigen Last","en":"Sustained-load scope","es":"Alcance de la carga sostenida","fr":"Portée de la charge soutenue","pt":"Âmbito da carga sustentada","tr":"Sürdürülen yük kapsamı"},"language":"en","type":"text"}],"description":{"ar":"يستخدم اختبار الحمل عمال GPU محاكين جزئياً.","de":"Der Lasttest verwendet teilweise simulierte GPU-Worker.","en":"The load test uses partly simulated GPU workers.","es":"La prueba de carga usa trabajadores GPU parcialmente simulados.","fr":"Le test de charge utilise des travailleurs GPU en partie simulés.","pt":"O teste de carga usa trabalhadores de GPU parcialmente simulados.","tr":"Yük testi kısmen benzetilmiş GPU çalışanları kullanır."},"id":"frontend-benchmark","methodology":{"ar":"في تجربة واجهة Dynamo بسياق مخبأ من 121 ألف رمز وهدف زمن أول رمز عند المئين 99 قدره ثانيتان، نفذت H100 NVL حقيقية الترميز بينما حوكيت عمال GPU.","de":"In einem Dynamo-Frontend-Test mit einem zwischengespeicherten Kontext von 121.000 Tokens und einem P99-Ziel von 2 Sekunden für das erste Token tokenisierte eine echte H100 NVL; die GPU-Worker wurden simuliert.","en":"In a Dynamo frontend experiment with a cached 121k-token context and a 2-second P99 time-to-first-token target, a real H100 NVL performed tokenization while GPU workers were simulated.","es":"En una prueba del frontend Dynamo con un contexto en caché de 121 000 tokens y un objetivo P99 de 2 segundos hasta el primer token, una H100 NVL real tokenizó mientras se simulaban los trabajadores GPU.","fr":"Dans un test du frontal Dynamo avec un contexte mis en cache de 121 000 jetons et un objectif de délai P99 de 2 secondes, un H100 NVL réel effectuait la tokenisation tandis que les travailleurs GPU étaient simulés.","pt":"Num teste do frontend Dynamo com contexto em cache de 121 mil tokens e meta P99 de 2 segundos até ao primeiro token, uma H100 NVL real fez a tokenização, enquanto os trabalhadores GPU foram simulados.","tr":"Dynamo ön ucunda, önbelleğe alınmış 121 bin tokenlık bağlam ve 2 saniyelik P99 ilk token süresi hedefiyle yapılan deneyde gerçek H100 NVL belirteçleme yaptı; GPU çalışanları benzetildi."},"rowTranslations":[{"load":{"ar":"518 طلباً في الثانية ضمن هدف P99 قدره ثانيتان؛ عمال GPU محاكَون","de":"518 Anfragen/s beim P99-Ziel von 2 Sekunden; GPU-Worker simuliert","en":"518 requests/s within 2-second P99 target; GPU workers simulated","es":"518 solicitudes/s con un objetivo P99 de 2 segundos; trabajadores GPU simulados","fr":"518 requêtes/s dans un objectif P99 de 2 secondes; travailleurs GPU simulés","pt":"518 pedidos/s dentro da meta P99 de 2 segundos; trabalhadores GPU simulados","tr":"2 saniyelik P99 hedefinde saniyede 518 istek; GPU çalışanları benzetilmiş"}}],"rows":[{"context_tokens":121000,"load":"518 requests/s within 2-second P99 target; GPU workers simulated"}],"sourceUrls":["https://arxiv.org/abs/2610.13325"],"title":{"ar":"مقارنة أداء الواجهة الأمامية","de":"Leistungsvergleich des Frontends","en":"Frontend performance comparison","es":"Comparación del rendimiento del frontend","fr":"Comparaison des performances du frontal","pt":"Comparação do desempenho do frontend","tr":"Ön uç performans karşılaştırması"},"type":"table","updatedAt":"2026-10-13T01:31:44Z"}],"version":1},"data_downloads":[{"formats":{"csv":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026/data/frontend-benchmark?lang=de&format=csv","json":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026/data/frontend-benchmark?lang=de&format=json"},"id":"frontend-benchmark"}],"data_types":["table"],"editorial_illustration":{"alt":"Konzeptionelle Darstellung von Technologie, Prozessoren und verbundenen Informationen.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"de","scope":"topic","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"event_id":"event-38db37c5-be8c-5dee-ba6f-4b6d6e21e130","event_period":"2026-10-09","event_period_basis":"source-review","id":"gpu-tokenization-agentic-llm-serving-graphtok-2026","image":null,"importance":0.6,"language":"de","modified_at":"2026-10-13T01:40:12+00:00","publication_basis":"publication-record","published_at":"2026-10-13T01:40:12+00:00","reading_minutes":1,"region":"world","section_illustration":{"alt":"Konzeptionelle Darstellung von Technologie, Prozessoren und verbundenen Informationen.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"de","scope":"section","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"slug":"gpu-tokenization-agentic-llm-serving-graphtok-2026","sources":[{"checkedAt":"2026-10-13","name":"arXiv","publishedAt":"2026-10-09","url":"https://arxiv.org/abs/2610.13325"}],"summary":"Forschende des KTH Royal Institute of Technology stellten GraphTok vor, ein System zur Tokenisierung von Anfragen großer Sprachmodelle auf einer GPU. In einem Dynamo-Frontend-Test tokenisierte eine echte H100 NVL; die GPU-Worker im Backend wurden simuliert.","text":"Forschende des KTH Royal Institute of Technology stellten GraphTok vor, ein System zur Tokenisierung von Anfragen großer Sprachmodelle auf einer GPU. In einem Dynamo-Frontend-Test tokenisierte eine echte H100 NVL; die GPU-Worker im Backend wurden simuliert.\n\nKI-Agenten senden frühere Gespräche und Werkzeugausgaben mit jeder neuen Anfrage erneut. Ein Präfix-Cache kann Modellberechnungen für bereits verarbeiteten Text einsparen, doch die gesamte Anfrage muss vor der Cache-Abfrage in Token-IDs umgewandelt werden. GraphTok erledigt dies mit einem CUDA-Graphen auf der GPU und leitet kurze Eingaben an die CPU weiter.\n\nIn einem Dynamo-Frontend-Test verarbeitete GraphTok bei 121.000 Token Anfrageverkehr 518 Anfragen pro Sekunde innerhalb eines P99-Ziels von zwei Sekunden bis zum ersten Token; gigatoken erreichte 296 und der Standard-Tokenizer 18,5 Anfragen pro Sekunde. Die GPU-Worker wurden simuliert und die Korrektheit nur für getestete Tokenizer-Konfigurationen geprüft; die Werte sind keine allgemeine Kapazitätsangabe.","title":"GraphTok verlagert die Tokenisierung langer KI-Eingaben auf die GPU","topics":[],"translations":[{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=tr","language":"tr","url":"https://newsverum.com/tr/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=en","language":"en","url":"https://newsverum.com/en/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=ar","language":"ar","url":"https://newsverum.com/ar/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=fr","language":"fr","url":"https://newsverum.com/fr/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=es","language":"es","url":"https://newsverum.com/es/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=de","language":"de","url":"https://newsverum.com/de/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},{"api_url":"https://newsverum.com/api/v1/articles/gpu-tokenization-agentic-llm-serving-graphtok-2026?lang=pt","language":"pt","url":"https://newsverum.com/pt/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"}],"url":"https://newsverum.com/de/news/gpu-tokenization-agentic-llm-serving-graphtok-2026"},"version":"1"}