Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: Bidirectional Encoders That Stay Fast at 8K Context on CPU

Liquid AI has released two open-weight bidirectional encoders, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M. Both are masked language models built on the LFM2 hybrid backbone. Both carry an 8,192-token context. Encoders sit underneath classifiers, intent routers, safety filters, and PII detectors. Those jobs run continuously, usually without a GPU, and increasingly on longer inputs. BERT established the class. ModernBERT pushed its accuracy, speed, and context. Liquid AI’s argument is that the LFM2 architecture continues that line, because its cost grows more slowly as inputs get longer. How a decoder became an encoder The encoders are not trained from scratch. They are initialized from the LFM2.5-230M and LFM2.5-350M decoder backbones, then converted with three changes: First, the causal attention mask is replaced with a bidirectional one, so every token attends to both sides. Second, the LFM2 short convolutions are made non-causal using symmetric center padding, so each token’s convolution mixes in neighbours on both sides. Third, the model is trained with a masked language modeling objective at a 30% mask rate. That is denser than BERT’s 15%, following evidence Liquid AI cites that a higher mask rate helps at this scale. Training runs in two stages: Stage one establishes general language competence with a short-context MLM objective on a large web corpus at 1,024 tokens. Stage two extends context to 8,192 tokens on the full data mix, strengthening factual, legal, and multilingual competence. Architecturally, the backbone interleaves gated short-convolution blocks with grouped-query attention, the same design described in the LFM2 technical report. Both checkpoints use a hidden size of 1024 and a 65,536-token vocabulary, and support 15 languages. The license is the LFM Open License v1.0. The embed below breaks down the conversion, the full ranking table, the per-task scores, and the CPU latency figures. (function(){ window.addEventListener("message", function(e){ if(!e.data || !e.data.lfmEncHeight) return; var f = document.getElementById("mtp-lfm-enc-1"); if(f) f.style.height = e.data.lfmEncHeight + "px"; }); })(); The benchmark results Liquid AI evaluated 14 models on 17 tasks pulled from GLUE, SuperGLUE, and multilingual classification. Every model is fully fine-tuned per task, and the reported score is that fine-tuned model’s result. LFM2.5-Encoder-350M posts a 17-task mean of 81.02 (±1.00), ranking fourth. The three models ahead of it are all larger. They are XLM-R XL at 3.5B (83.06), ModernBERT-large at 395M (81.68), and XLM-R large at 560M (81.34). The top model is nearly 10 times its size. LFM2.5-Encoder-230M posts 79.29 (±1.02), ranking sixth. It beats ModernBERT-base at 78.19 and every EuroBERT model in the table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19). Both new encoders also score above Liquid AI’s own retrieval siblings, LFM2.5-ColBERT-350M (76.18) and LFM2.5-Embedding-350M (75.68). That gap is the stated reason Liquid AI built a general-purpose encoder instead of reusing the retrievers. The methodology is the more interesting part, and it is open-sourced under Apache-2.0. Every model is loaded with fp32 master weights and bf16 autocast, so the table compares models rather than number formats. Every model uses the same AdamW recipe, taken from the EuroBERT card. Learning rate is selected per model and task across 10 rates and 3 seeds. Scores are then reported as the mean over 5 fresh seeds that never touched selection. The transformers version is pinned to 4.56.2 so dependency drift is not an uncontrolled variable. Use Cases and Deployment Environments The release names three settings. Edge and embedded devices come first. A car’s onboard compute or an industrial controller has no spare GPU, and cannot afford a cloud round trip. Regulated and on-premise systems in finance,