{"article":{"annotation":{"article_id":"hugging-face-tokenizers-proposes-bert-normalization-speedups-2026","available_at":"2026-10-15T18:13:59.101303Z","geography_revision":"ef6ee0010dda5fe7b240fd514749b43e214c57da73222d2e40ec83b276dd0977","id":"8e4560ac6eb908d1cde96a5241c6685eddbd1b288bffe20273200fb83c015b56","input_sha256":"869d3f75de31680a9f87fbade215a718671e1cf331f92c27783d915947a56acb","limitations":["Rule-derived subjects are candidates, not verified editorial labels or calibrated model probabilities.","Coverage is distinct from event location. Unknown country/city/coordinates remain absent; no automatic geocoding.","Article geometries keep their stated precision and methodology; reproducing them is not independent coordinate verification."],"method":"controlled-rules-v1","places":[{"admin1":null,"city":null,"coordinate_note":null,"coordinate_reference_system":"OGC:CRS84","coordinate_source":null,"country":"US","evidence":[{"field":"country","language":"en","note":"Explicit preserved editorial country/region; no city or national centroid inferred","source_urls":["https://huggingface.co/blog/huggingface/tokenmaxxing-bert"]}],"geometry":null,"id":"iso3166-1:US","kind":"country-or-area","labels":{"ar":"الولايات المتحدة الأمريكية","en":"United States of America","es":"Estados Unidos de América","fr":"États-Unis d’Amérique"},"match_status":"candidate","precision":"regional","role":"coverage","uncertainty_m":null}],"rules_sha256":"fb5ad3c27c91cb840f7a0a05f7f64a5cb26ad0374409ff9457f277c7092e22b4","schema_version":1,"subjects":[{"concept_id":"medtop:13000000","confidence":null,"confidence_method":null,"evidence":[{"field":"category+topics","language":"en","note":"Preserved editorial category/topic; broad candidate only","source_urls":["https://huggingface.co/blog/huggingface/tokenmaxxing-bert"]}],"method":"controlled-rules-v1","rule_id":"technology-broad","status":"candidate"}],"taxonomy_revision":"1f2db31d811fca07e99b16d402c10e2cc2965c9df1c7b41d9be348b40a24e7fb","taxonomy_version":"IPTC Media Topics 2026-Q2"},"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=es","category":"technology","content":{"key_points":[],"paragraphs":["Una publicación del equipo Tokenizers de Hugging Face, fechada el 15 de octubre, describió cambios propuestos para acelerar la normalización de texto de tipo BERT. El método omite trabajo en secuencias ASCII, almacena en caché decisiones Unicode en páginas de 256 puntos de código y evita crear una cadena nueva si el texto no cambia. El perfil del equipo atribuyó el 88% del tiempo de codificación en inglés, tras el calentamiento, a la normalización, frente al 8% a la selección de tokens de WordPiece.","La publicación afirma que una rama de desarrollo codificó bert-base-uncased hasta unas 50 veces más rápido que Tokenizers 0.23 en las entradas probadas, con identificadores de tokens idénticos en los corpus comunicados. Las mayores mejoras se dieron en inglés y código; árabe, griego y ruso lograron entre 2,2 y 3,2 veces, mientras que el chino mejoró poco o nada. Las entradas pequeñas podían ser más lentas y los cambios seguían en fase de solicitud de incorporación, por lo que la cifra no garantiza el rendimiento general en producción."],"sections":[]},"corrections":[],"country":"US","data":{"blocks":[{"description":{"ar":"تختلف المكاسب حسب اللغة وحجم الإدخال، وكانت التحسينات ما تزال قيد المراجعة.","de":"Die Zugewinne hängen von Sprache und Eingabelänge ab; die Änderung wurde noch geprüft.","en":"The reported gains vary by language and input size, and the optimization was still under review.","es":"Las mejoras varían según el idioma y el tamaño de entrada; los cambios seguían en revisión.","fr":"Les gains rapportés varient selon la langue et la taille des entrées; l’optimisation restait en examen.","pt":"Os ganhos variam conforme o idioma e o tamanho da entrada; as alterações ainda estavam em análise.","tr":"Ölçülen kazanımlar dile ve girdi uzunluğuna göre değişti; iyileştirme hâlâ inceleme aşamasındaydı."},"id":"announcement-record","items":[{"date":"2026-10-15","description":{"ar":"تختلف المكاسب حسب اللغة وحجم الإدخال، وكانت التحسينات ما تزال قيد المراجعة.","de":"Die Zugewinne hängen von Sprache und Eingabelänge ab; die Änderung wurde noch geprüft.","en":"The reported gains vary by language and input size, and the optimization was still under review.","es":"Las mejoras varían según el idioma y el tamaño de entrada; los cambios seguían en revisión.","fr":"Les gains rapportés varient selon la langue et la taille des entrées; l’optimisation restait en examen.","pt":"Os ganhos variam conforme o idioma e o tamanho da entrada; as alterações ainda estavam em análise.","tr":"Ölçülen kazanımlar dile ve girdi uzunluğuna göre değişti; iyileştirme hâlâ inceleme aşamasındaydı."},"label":{"ar":"فريق Hugging Face يعلن تسريعاً مقترحاً لتقطيع نصوص BERT","de":"Hugging-Face-Team meldet vorgeschlagene Beschleunigung der BERT-Tokenisierung","en":"Hugging Face team reports faster BERT tokenization in proposed changes","es":"El equipo de Hugging Face propone acelerar la tokenización BERT","fr":"L’équipe de Hugging Face propose d’accélérer la tokenisation BERT","pt":"Equipa da Hugging Face propõe acelerar a tokenização BERT","tr":"Hugging Face ekibi önerilen değişikliklerle BERT tokenizasyonunun hızlandığını bildirdi"},"status":"occurred"}],"methodology":{"ar":"يلخص سجل الحدث إعلان المصدر ولا يضيف تحققاً مستقلاً أو تعميماً يتجاوز ما ورد فيه.","de":"Dieser Ereigniseintrag fasst die Quelle zusammen und ergänzt keine unabhängige Prüfung oder Verallgemeinerung.","en":"This event record summarizes the source announcement and adds no independent verification or generalization beyond it.","es":"Este registro resume el anuncio de la fuente sin añadir verificación independiente ni generalizaciones.","fr":"Cette fiche résume l’annonce de la source sans ajouter de vérification indépendante ni généralisation.","pt":"Este registo resume o anúncio da fonte sem acrescentar verificação independente ou generalização.","tr":"Bu olay kaydı kaynağın duyurusunu özetler; kaynakta belirtilmeyen bağımsız doğrulama veya genelleme eklemez."},"sourceUrls":["https://huggingface.co/blog/huggingface/tokenmaxxing-bert"],"title":{"ar":"ملخص الإعلان","de":"Zusammenfassung der Ankündigung","en":"Announcement summary","es":"Resumen del anuncio","fr":"Résumé de l’annonce","pt":"Resumo do anúncio","tr":"Duyuru özeti"},"type":"events","updatedAt":"2026-10-15T18:03:17Z"}],"version":1},"data_downloads":[{"formats":{"csv":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026/data/announcement-record?lang=es&format=csv","json":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026/data/announcement-record?lang=es&format=json"},"id":"announcement-record"}],"data_types":["events"],"editorial_illustration":{"alt":"Ilustración conceptual de tecnología, procesadores e información conectada.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"es","scope":"topic","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"event_id":"event-0ca089fe-d902-585a-87ae-23fb0a631aed","event_period":"2026-10-15","event_period_basis":"event-or-reference-period","id":"hugging-face-tokenizers-proposes-bert-normalization-speedups-2026","image":null,"importance":0.6,"language":"es","modified_at":"2026-10-15T18:13:43+00:00","publication_basis":"publication-record","published_at":"2026-10-15T18:13:43+00:00","reading_minutes":1,"region":"world","relations":[],"section_illustration":{"alt":"Ilustración conceptual de tecnología, procesadores e información conectada.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"es","scope":"section","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"slug":"hugging-face-tokenizers-proposes-bert-normalization-speedups-2026","sources":[{"checkedAt":"2026-10-15","name":"Hugging Face Tokenizers team","publishedAt":"2026-10-15","url":"https://huggingface.co/blog/huggingface/tokenmaxxing-bert"}],"summary":"Las mejoras varían según el idioma y el tamaño de entrada; los cambios seguían en revisión.","text":"Las mejoras varían según el idioma y el tamaño de entrada; los cambios seguían en revisión.\n\nUna publicación del equipo Tokenizers de Hugging Face, fechada el 15 de octubre, describió cambios propuestos para acelerar la normalización de texto de tipo BERT. El método omite trabajo en secuencias ASCII, almacena en caché decisiones Unicode en páginas de 256 puntos de código y evita crear una cadena nueva si el texto no cambia. El perfil del equipo atribuyó el 88% del tiempo de codificación en inglés, tras el calentamiento, a la normalización, frente al 8% a la selección de tokens de WordPiece.\n\nLa publicación afirma que una rama de desarrollo codificó bert-base-uncased hasta unas 50 veces más rápido que Tokenizers 0.23 en las entradas probadas, con identificadores de tokens idénticos en los corpus comunicados. Las mayores mejoras se dieron en inglés y código; árabe, griego y ruso lograron entre 2,2 y 3,2 veces, mientras que el chino mejoró poco o nada. Las entradas pequeñas podían ser más lentas y los cambios seguían en fase de solicitud de incorporación, por lo que la cifra no garantiza el rendimiento general en producción.","title":"El equipo de Hugging Face propone acelerar la tokenización BERT","topics":[],"translations":[{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=tr","language":"tr","url":"https://newsverum.com/tr/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=en","language":"en","url":"https://newsverum.com/en/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=ar","language":"ar","url":"https://newsverum.com/ar/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=fr","language":"fr","url":"https://newsverum.com/fr/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=es","language":"es","url":"https://newsverum.com/es/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=de","language":"de","url":"https://newsverum.com/de/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},{"api_url":"https://newsverum.com/api/v1/articles/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026?lang=pt","language":"pt","url":"https://newsverum.com/pt/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"}],"url":"https://newsverum.com/es/news/hugging-face-tokenizers-proposes-bert-normalization-speedups-2026"},"version":"1"}