Sentence Segmentation Is Not as Solved as It May Seem

This article covers the methodology behind Ortix Seg LV, a state-of-the-art Latvian sentence boundary model released in the Ortix Collection.

Sentence segmentation sounds like the kind of NLP problem that should have been solved years ago. Find the periods, avoid a few abbreviations, split the text, move on...

That intuition holds until the input stops being clean prose.

Existing Latvian sentence segmentation tools and models portray quite strong performance on benchmark-style text: clean, prose-like input where punctuation and sentence structure usually agree. But dissertations, legal references, parliamentary transcripts, web pages, noisy parsed text, dictation artifacts, flattened layouts, broken line wraps, and punctuation that is missing or misleading expose a different problem. The model has to separate meaningful text units even when punctuation is not present, layout has not been preserved, and the boundary depends on document structure rather than grammar alone.

Why Sentence Boundaries Matter

Sentence segmentation is easy to underestimate because the output can look insignificant: a list of character offsets. But those offsets decide what every later step receives as input.

A clear downstream usage example is a grammatical error correction (GEC) model, which needs a coherent sentence-like unit. If the segmenter cuts a sentence in half, the GEC model loses the structure it needs to assess the sentence and apply appropriate corrections. If the segmenter joins unrelated rows together, the model receives text that was never meant to be corrected as one unit.

Retrieval, embeddings, deduplication, quality filtering have essentially the same dependency. They work best when the text unit they receive means something on its own.

What Counts as a Sentence?

Before training a model, it's wise to explicitly define what the model was supposed to predict.

That sounds obvious, but sentence segmentation becomes less obvious once the text comes from real documents. A period is not always a boundary, and a boundary does not always need a period. In clean prose, sentence boundaries mostly follow grammar. In parsed documents, they also follow layout, source structure, and whether a span can stand on its own.

For this dataset, a sentence was treated as either a complete written utterance or a standalone document unit. That means ordinary prose sentences are included, but so are headings, list items, agenda rows, speaker labels, captions, and similar units when the source presents them as independent text. The opposite is also true: not every period is a boundary. Dates, initials, abbreviations, references, and URLs often contain punctuation that belongs inside the same unit.

A boundary policy had to be stated upfront. If it is not stated, it still exists, but only as a collection of implicit choices, annotation habits, and source-specific exceptions. Some cases are genuinely ambiguous, so the goal was to make the policy explicit and keep it broad enough to work across document types without adding unnecessary bias toward one source format.

When looking at it from a practical perspective, the question shifts from "is there punctuation here?" to "does the text before this point form a meaningful unit, and does the text after it begin another one?"

Existing Work and Baselines

Sentence segmentation already has mature tooling. The important question was not whether existing systems are good, but what kind of data they were trained on, what a boundary means in that data, and how well those assumptions transfer to parsed academic, legal, parliamentary, and web-derived Latvian text for downstream use cases.

Most of the strong available baselines are tied to Universal Dependencies (UD): a multilingual treebank framework with consistent annotation for tokenization, morphology, syntax, and sentence structure across languages ^[1]. For Latvian, that usually means the Latvian Treebank (LVTB) in UD format ^[2]. This matters because models trained on the same treebank family should behave similarly on clean UD-style text. The larger gaps are expected when the input shifts from treebank prose to parsed documents, flattened layouts, references, transcripts, and web fragments.

Stanza

Stanza is Stanford's multilingual NLP pipeline, with pretrained models for many languages, including Latvian ^{[4] [5]}. Its tokenizer performs tokenization and sentence segmentation together as character-level tagging over raw text ^[3].

For this project, Stanza represented the mature BiLSTM-based non-transformer UD pipeline ^[3]. It was useful for understanding how far established Latvian UD tokenization carries on clean treebank-style text.

COMBO-SEG

COMBO-SEG is a character-level segmentation system built on transformer encoders. The Latvian model uses FacebookAI/xlm-roberta-base ^[9] as the base, is trained on UD_Latvian-LVTB v2.17, and is released as part of the COMBO-SEG UD 2.17 model set ^{[6] [7]}. The XLM-RoBERTa base encoder is interesting here because it is known to be a strong multilingual backbone ^[8].

The architecture detail worth noting is that XLM-RoBERTa uses a subword tokenizer, while COMBO-SEG does not simply split over those subword tokens. Instead it projects the transformer representations back to character positions and predicts word, token, and sentence-end labels there.

This made COMBO-SEG the most important clean Latvian UD comparison, with the same caveat as before: it may be an excellent model for clean Latvian treebank-style text, but not automatically aligned with our broader document-unit definition.

Segment Any Text (SaT)

Segment Any Text (SaT) is a multilingual sentence segmentation approach built around robustness to noisy input. Instead of assuming that sentence boundaries are mostly recoverable from clean punctuation and casing, it trains on corrupted variants where those cues are weakened or removed ^[10].

Their training setup includes punctuation and casing corruptions, supervised mixture training over clean and noisy styles, and adaptation experiments for domains such as legal text. The paper also reports strong sentence segmentation performance from compact 3-layer & 6-layer models, which made it a useful reference when deciding how much model capacity the task actually needed ^{[11] [12]}.

Estonian Web Text Evidence

The closest regional reference was the Estonian web text evaluation by Sirts and Peekman ^[13]. Their setup is useful because it asks almost the same question from another Baltic adjacent language: what happens when sentence segmentation systems trained or tuned on cleaner text are evaluated on manually annotated web text?

The paper separates orthographic sentence boundaries from syntactic sentence boundaries, which is close to the issue we faced with Latvian document units. A web paragraph can be one long orthographic sentence, but still contain several meaningful units that should be separated for downstream processing. They also show that paragraph boundaries matter: preserving layout gives segmenters useful boundary evidence, while losing it makes the task harder.

The useful warning was that domain shift is also annotation shift in practice. Clean treebank performance can look strong, but noisy text, domain-specific text, and web-derived text change what counts as a boundary, weaken punctuation assumptions, and make layout preservation part of the task.

What Existing Systems Taught Us

The lesson was not that existing sentence segmenters are weak. The lesson was that strong performance on clean UD-style text does not guarantee precise behavior across the broader range of documents where sentence segmentation is actually needed.

A general-purpose Latvian segmenter has to handle more than prose paragraphs. It has to work on academic text, legal references, parliamentary transcripts, web pages, copied fragments, flattened layouts, and text where punctuation is missing or misleading. That requires a boundary definition that is explicit, consistent, and broad enough to avoid turning every domain into a separate set of exceptions.

So the next question became practical: what data and training setup would make a Latvian model learn that broader definition directly?

Building the Dataset

The next step was not simply to find more Latvian text. More text only helps if it contains the boundary decisions the model needs to learn. Random clean paragraphs mostly reinforce the easy rule: split after terminal punctuation. The useful data had to include cases where that rule stops working.

The clean validated set combined local document sources with mined Latvian FineWeb-2 records ^[14]. The local sources included LU dissertations, Saeima material, Latvian articles, legal acts, Wikipedia, government and declassified documents, and judges archive records. FineWeb-2 supplied broader web variation: copied fragments, comments, service pages, reference pages, navigation leftovers, and extraction artifacts.

Before sampling, the local source datasets passed through the internal data processing pipeline (FineWeb-2 was already processed), which included exact duplicate removal and MinHash near-duplicate filtering. That kept repeated boilerplate, mirrored pages, and near-identical documents from dominating the candidate pool before hard boundary cases were mined.

The final clean validated dataset contained 539 manually validated records, about 683k characters, and 9,251 annotated sentence-end offsets. Roughly 60% of the records came from FineWeb-2. The rest kept more structured sources, which were distributed in balanced proportions: dissertations, legal and government text, parliamentary proceedings, Wikipedia, Latvian articles, and court-related material.

Each validated record was also assigned document-type metadata. That metadata was used when creating the train/dev/test split, so the held-out evaluation would still include different kinds of text rather than accidentally overrepresenting one source or genre. The final set includes commercial and service pages, reference articles, news, academic text, forum comments, legal and regulatory text, parliamentary proceedings, government policy, creative or review text, and a small number of other web fragments.

Mining Hard Boundary Cases

Random samples are useful for coverage, but they are inefficient for finding the boundaries that matter most. The review queues were built from disagreement: run several segmenters over the same text, compare their non-final boundary offsets, and queue records where the systems disagree.

This was useful because disagreement is not proof that one model is wrong. It is a cheap signal that the example contains a boundary decision worth reviewing.

Hard-case mining was applied across both the local document sources and FineWeb-2 examples. The initial local sample contained 3,000 records balanced across seven source groups, with length and newline-density buckets used to avoid sampling only one shape of document. From there, segmenter disagreement was used to prioritize review records. Later review queues raised the minimum number of disagreement required across models during mining, with the idea in mind to select more drastically challenging examples to include in the training regimen.

Manual Review and Validation

The mined examples still had to become 'gold' data. For this task, a small review application was developed to make each disagreement case inspectable and support final dataset curation. The review application made it easy to review the extracted text, boundary proposals from different models, and the places where those proposals diverged. Those proposals could be corrected manually, text artifacts could be cleaned before rerunning the segmenters, and records could be validated, marked as ambiguous, skipped, or excluded with notes.

Optional LLM annotations were useful as drafts (instructed to follow accordingly to the sentence definition) and greatly sped up the review process, but the final records were saved only after manual human examination.

Training the Boundary Model

Boundary Detection

Many sentence segmentation systems operate at character level. That is a natural fit for exact boundary prediction because the model does not have to rely on a tokenizer deciding where a word or subword begins and ends. The tradeoff is that character-level models usually process longer sequences and therefore cost more at training and inference time. Tokenization-free architectures such as CANINE exist partly to make character-level modeling more efficient, but they still represent a different speed and architecture tradeoff from ordinary subword transformers ^[15].

For this model, the practical question was whether XLM-RoBERTa subword tokenizer was good enough for Latvian boundary prediction. The concern was real, because if the tokenizer merged text in a way that hid important boundary positions, the model would be forced to predict from bad units. We checked our training data for problematic tokenization around sentence-end offsets and did not find this to be a blocker.

So the model kept the XLM-R tokenizer and treated sentence segmentation as token classification. For each supervised token position, it learned whether a boundary should appear after the current token, written here as <B>. Sometimes the boundary is after ordinary punctuation, as in teica.<B> Viņš, and sometimes it is after a word-like token that already includes punctuation, such as abc.<B> Nākamais. In noisier extracted text, the boundary can also sit at a damaged join such as teica<B>Viņš or teica<B> Viņš, where punctuation or layout has been lost.

XLM-RoBERTa Baseline

The first strong model family used FacebookAI/xlm-roberta-base with a token-level boundary classification head. This was the first task-learning stage: train the model to recognize Latvian sentence boundaries from UD/LVTB-style data before adapting it to the manually validated document-unit dataset.

The encoder-depth experiments were partly motivated by SaT. One of the useful findings from the SaT paper was that sentence segmentation can remain highly competitive with smaller 3-layer and 6-layer models, while becoming much cheaper to run ^[10]. That made it worth testing whether the full 12-layer XLM-RoBERTa encoder was actually needed for this task.

It was not. Early ablations showed that retaining the first six pretrained transformer layers gave a better efficiency and quality tradeoff. We also tested a DistilBERT-inspired variant that kept every second pretrained layer (1, 3, 5, 7, 9, 11) instead of the first six layers, but it performed worse in this setup ^[19].

Before joint training on UD/LVTB plus the clean manually validated data, the strongest base checkpoint used weighted cross entropy plus Dice loss with dice_weight=0.2. That objective fit the label shape well: sentence boundaries are sparse, so cross entropy kept token-level probability learning stable, while the Dice term added a direct overlap signal for the rare boundary class ^[24].

This stage gave a strong UD218 model and a stable anchor for the rest of the work. The next problem was adaptation: improving on document-unit boundaries without losing the clean treebank behavior.

Joint Training

The adaptation stage had to avoid a simple failure mode: learning the manually validated document-unit data while degrading the behavior already learned from UD-style Latvian text. Neural models can suffer from catastrophic forgetting when continued training shifts weights toward a new task or domain and weakens earlier behavior. Methods such as Learning without Forgetting address this with distillation losses when the old training data is not available ^[16].

In this case, the old data was available and the tasks were close. Instead of relying on a distillation objective, the XLM-RoBERTa branch used joint training: the clean manually validated document-unit data was mixed with the original UD/LVTB-style data. UD/LVTB stayed visible as the treebank anchor, while the clean validated data supplied the broader document-boundary behavior.

The raw sources were strongly imbalanced: 4,995 UD/LVTB training windows versus 456 clean manually validated windows. With 0.7/0.3 source weights, each epoch sampled 3,816 clean manually validated windows and 1,635 UD/LVTB windows. In practice, the smaller clean set was intentionally replayed while UD/LVTB was sampled without replacement. A more formal mixture-optimization approach, such as RegMix or Olmix, could be useful later, but the dataset sizes were small enough that direct sweeps were sufficient for this stage ^{[17] [18]}.

The final XLM-RoBERTa adaptation recipe came after many ablation runs over encoder depth, loss objectives, data mixture, calibration, layer-wise learning-rate decay, SWA, and checkpoint averaging. Focal loss replaced the earlier CE+Dice baseline for the joint stage. The selected setup used gamma=1.0 and boundary positive weight 10, which kept training focused on hard boundary positions instead of letting easy non-boundary tokens dominate the objective ^[20].

Layer-wise learning-rate decay was set to 0.90. The classifier head and highest retained encoder layer moved the most, while lower transformer layers were updated more conservatively. This was useful for preserving stable UD validation loss while adapting toward the manually validated document-unit data. The idea is close to discriminative fine-tuning: lower pretrained layers should usually change less than task-adjacent layers during adaptation ^[21].

The run trained for 10 epochs total and used a late SWA (Stochastic Weight Averaging) phase: epoch_start=8, one annealing epoch, and top SWA learning rate 1e-5. SWA averages weights from the same optimization trajectory and can improve generalization without changing inference cost ^[22]. We also tested model-soup style averaging across different candidate runs. Some soups, especially between similar layer-wise LR/SWA runs, were competitive with the selected checkpoint. Others regressed, which is consistent with the model-soup assumption that weight averaging works best when models are in a compatible region of the loss landscape ^[23].

Calibration

During training, the main validation metric was PR-AUC. Accuracy was not useful for this task because most token positions are non-boundaries, and fixed-threshold F1 would have assumed that the operating threshold was already known. PR-AUC was a better training-time signal because it measured whether the model ranked boundary positions well without committing to one threshold too early.

During joint training, validation had to account for both sources. A model that improved on clean manually validated data but regressed on UD/LVTB was not acceptable, and the opposite was not enough either. The validation score was therefore weighted across the clean manually validated data and UD/LVTB data, so checkpoint selection reflected the same tradeoff as the training mix.

After training, the decision threshold was calibrated on held-out dev data (mixture of clean manually validated and UD/LVTB data) and then frozen for test evaluation. Final promotion was based on calibrated model performance.

SaT Baselines

At this point, SaT became the most important external comparison. It was the strongest model family we had evaluated so far, and it was also the clearest target to beat. The SaT paper was relevant because it described the models being trained with punctuation and casing corruptions, supervised mixture training, and robustness in mind ^{[10] [11]}.

The result changed the direction of the work. On UD218, the XLM-RoBERTa project checkpoint was still stronger: it had higher precision, higher F1, and better exact match than the tested SaT models. But on the manually validated document test set, SaT was clearly ahead. The gain came mostly from recall. SaT found many more standalone document-unit boundaries, especially in web and layout-like text, but it also produced many more false positives.

The UD218 comparison showed the opposite tradeoff:

This provided intuitive insights. First, it showed that the XLM-RoBERTa model was too conservative on the broader clean validated set. Second, it showed that SaT's stronger recall came with a different precision profile, especially on UD218 and some document-unit edge cases. Before trying to reproduce SaT's corruption pipeline, the lighter experiment was to test whether sat-12l-sm could be used as a teacher through soft-label distillation.

SaT Distillation

Since sat-12l-sm had the strongest clean validated recall, the next question was whether its probability estimates could be used without copying its full error profile over. The experiment did not use SaT hard predictions as new labels. It used soft-label distillation: gold boundary labels remained the main supervision signal, while an auxiliary loss encouraged the student to stay closer to SaT's boundary probabilities ^[25].

The setup was kept lightweight. sat-12l-sm probabilities were cached once over the training and development data, so the teacher did not have to run during every training step. SaT predicts boundary probabilities at character positions, while the project model predicts after supervised token positions. The alignment therefore gathered the teacher value at the character just before the token end: sat_probs[token_end - 1].

The distillation loss was source-dependent. SaT had better recall on the clean validated document set, but weaker precision on UD218. Because of that, the main sweeps tested separate UD and clean teacher weights. The best token-level sweep region used very small clean validated KD weights, around 0.003 to 0.005, with 0.03 on UD/LVTB (which was slightly counterintuitive because the experiment was motivated by SaT's higher recall on the manually validated document set, but the sweeps preferred a larger teacher weight on UD/LVTB). Strong 0.4 teacher-weight runs were added later as stress tests, not because they were the most promising sweep result.

After checkpointed training, threshold calibration, and decoded sentence-offset evaluation, the KD candidates did not improve over the original XLM-RoBERTa project checkpoint.

This was still informative. The KD objective changed token-level ranking, but that change did not translate into better decoded document-unit boundaries. Stronger teacher pressure also failed to recover the clean validated gap; it shifted the precision/recall balance without improving the final calibrated result.

The likely reason is that SaT was not only providing useful uncertainty estimates. It was also encoding a partly different boundary policy. Its extra recall helped identify missing boundary types, but some of that probability mass likely pointed toward splits outside the document-unit definition used here. For this setup, direct SaT distillation was therefore not selected as the next model path.

Encoder Adaptation

The distillation results did not show a promising path for recovering the missing recall on noisier document text. Stronger SaT guidance did not improve the final decoded result, and reproducing SaT's corruption-training pipeline would have opened a much larger experiment than we initially wanted to commit to. The next practical question was whether the limitation came partly from the multilingual encoder itself.

The motivation was distribution mismatch. Don't Stop Pretraining shows that pretrained language models often improve when the pretraining distribution is adapted toward the target domain or task data ^[26]. In this project, we did not run a new unlabeled domain-adaptive pretraining stage. Instead, we used the same intuition more directly: replace the multilingual XLM-RoBERTa backbone with a Latvian-specific encoder that should already start closer to Latvian text.

The Latvian encoder suite by Znotins introduced RoBERTa-, DeBERTaV3-, and ModernBERT-based encoders pretrained on a large Latvian mixture, including web-scale data and curated Latvian resources ^[27]. The paper reports lv-deberta-base as the strongest overall encoder in that suite, which made it the most natural candidate for the next training run.

The comparison was kept controlled by preserving the same boundary-detection formulation and reusing the same two-stage training structure. First, train on UD/LVTB-style sentence windows to learn the core sentence segmentation task. Then adapt with the joint UD/LVTB plus clean manually validated setup. That kept the experiment focused on the encoder: would better Latvian pretrained representations make the same boundary detector more reliable?

DeBERTa Fine-Tuning

The Latvian DeBERTa run changed the result more than the earlier loss and distillation experiments did. The same calibration methodology described above was used, but the model family changed from multilingual XLM-RoBERTa to Latvian lv-deberta-base.

The improvement was not just on one side of the tradeoff. The 6-layer DeBERTa checkpoint beat the tested SaT variants on the broader clean validated document set and on the UD218 benchmark. Compared with sat-12l-sm, it kept nearly the same clean validated recall, but reduced false positives substantially. Compared with the earlier XLM-RoBERTa project checkpoint, it recovered many more document-unit boundaries while also improving the already high UD218 score.

The 3-layer DeBERTa variant also presented compelling capabilities. It positions itself as competitive and faster alternative to 6-layer version. Since most earlier sweeps were performed with XLM-RoBERTa, the DeBERTa result should be read as a transferred-recipe result rather than proof that the setup is fully optimized. The training validation loss was also less stable than in the final XLM-RoBERTa runs, which suggests that the training configuration may not transfer perfectly across encoder families. For time and resource reasons, the heavy ablation sweeps were not repeated for DeBERTa.

Results

Evaluated Suite

The headline result is that the promoted Latvian DeBERTa checkpoint is the strongest model in the evaluated Latvian sentence-boundary suite. It beats the tested SaT models on the manually validated document set, while also improving over the UD218 benchmark baselines.

That matters because the two test sets stress different behavior. UD218 measures clean treebank-style sentence segmentation. The clean validated document set measures the broader target: academic, legal, parliamentary, web-derived, and layout-damaged text where sentence boundaries are often document-unit boundaries.

SaT had been the strongest external baseline on the clean validated document set because it found many more boundaries than the XLM-RoBERTa model. The promoted DeBERTa checkpoint keeps almost the same recall as sat-12l-sm, but with far fewer false positives: 49 instead of 204 on the clean validated test set.

The same pattern holds on UD218. SaT reaches very high recall, but its extra splits had reduced its precision. The DeBERTa checkpoint gives the best exact-match count and F1 there as well, which means the improvement didn't sacrifice clean treebank behavior.

FineWeb Source Slice

If the DeBERTa gain came mostly from dissertations, legal text, or other structured sources, it would be less convincing as evidence of robust web-document segmentation. The clean validated score therefore needed a source-level check. FineWeb-derived records make up most of that test set: 72 / 108 records.

This confirms that the final result is not carried by easier non-web examples. On FineWeb-derived records, the promoted DeBERTa checkpoint keeps SaT-level recall while sharply reducing false positives. That is precisely the behavior we wanted: recover the missing web/layout boundaries.

Limitations and Next Steps

The promoted final model sets the strongest result in the evaluated Latvian sentence-boundary suite: UD218, the manually validated document set and its FineWeb source slice, and the selected Stanza, COMBO-SEG, SaT, XLM-RoBERTa, and DeBERTa baselines. In that sense, it is a state-of-the-art Latvian sentence segmentation result. The scope is still important: the claim is tied to this boundary policy, this benchmark suite, and the systems evaluated here.

The manually validated dataset is relatively small. It contains 539 records, with 108 records allocated to the test split. It also does not necessarily represent a natural frequency sample of all Latvian text. It includes cleaner sources, such as parliamentary material, but the curation process deliberately included hard-mined model-disagreement cases. That makes the evaluation more informative for robustness, while less suitable for estimating real-world boundary-type frequencies.

The next dataset improvement is not only more annotation, but annotation reliability. Multi-annotator review would make it possible to measure where the boundary policy is obvious, where annotator preference starts to matter, and where the text itself is genuinely underdetermined. It would make it possible to estimate annotator reliability: some annotators may be more consistent on legal references, others on transcript labels or web fragments. For sentence segmentation, that may be more useful than treating every boundary as a perfectly binary fact. Ambiguous headings, references, transcript labels, list rows, and damaged layout cases could benefit from agreement statistics or probabilistic treatment before becoming hard labels.

Calibration also deserves more systematic validation. During training, PR-AUC was useful because it avoided choosing a fixed threshold too early. But PR-AUC and the later token-level max-F1 calibration did not always rank checkpoints the same way: a model could rank boundary candidates well overall, yet produce a worse precision/recall balance once a single threshold was chosen.

The DeBERTa result should also not be treated as fully optimized. Most of the expensive ablations were run against XLM-RoBERTa, and the DeBERTa runs reused that transferred recipe. Since DeBERTa is a different encoder family and its validation loss appeared less stable, there is probably still room for DeBERTa-specific tuning over learning rate, retained layer count, layer-wise learning-rate decay, loss settings, SWA, and calibration strategy.

SaT leaves another strong open direction. Its corruption-based training approach clearly produced a robust segmenter: on the manually validated set it recovered many boundaries that the first XLM-RoBERTa model missed. The remaining question is how to adapt that idea without importing too many false positives for the boundary policy used here.

Finally, the model still needs downstream validation. The practical question is whether better sentence boundaries improve grammatical correction, retrieval chunks, document cleanup, embedding quality, and filtering decisions.

So, is sentence segmentation finally solved?

Only if "sentence" means clean prose with 'polite' punctuation. For 'real' documents, the boundary is still a modeling decision, a data decision, and a policy decision at the same time...

References

[1] J. Nivre et al., "Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection," in Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 4034-4043, 2020. https://aclanthology.org/2020.lrec-1.497/

[2] L. Pretkalniņa, L. Rituma, G. Nešpore-Bērzkalne, B. Saulīte, A. Znotiņš, and N. Grūzītis, "UD Latvian LVTB." https://github.com/UniversalDependencies/UD_Latvian-LVTB

[3] P. Qi, T. Dozat, Y. Zhang, and C. D. Manning, "Universal Dependency Parsing from Scratch," in Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 160-170, 2018. https://doi.org/10.18653/v1/K18-2016

[4] P. Qi, Y. Zhang, Y. Zhang, J. Bolton, and C. D. Manning, "Stanza: A Python Natural Language Processing Toolkit for Many Human Languages," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 101-108, 2020. https://doi.org/10.18653/v1/2020.acl-demos.14

[5] Stanford NLP Group, "Model Performance," Stanza documentation. https://stanfordnlp.github.io/stanza/performance.html

[6] M. Ulewicz and A. Wróblewska, "COMBO-SEG Models Trained on UD v2.17," Zenodo, 2026. https://doi.org/10.5281/zenodo.19651441

[7] CLARIN-PL, "COMBO-SEG Model for Latvian," Hugging Face. https://huggingface.co/clarin-pl/combo-seg-xlm-roberta-base-latvian-lvtb-ud2.17

[8] A. Conneau et al., "Unsupervised Cross-lingual Representation Learning at Scale," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440-8451, 2020. https://doi.org/10.18653/v1/2020.acl-main.747

[9] FacebookAI, "XLM-RoBERTa base," Hugging Face. https://huggingface.co/FacebookAI/xlm-roberta-base

[10] M. Frohmann, I. Sterner, I. Vulić, B. Minixhofer, and M. Schedl, "Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation," arXiv:2406.16678, 2024. https://arxiv.org/abs/2406.16678

[11] Segment Any Text contributors, "wtpsplit." GitHub. https://github.com/segment-any-text/wtpsplit

[12] Segment Any Text, "sat-3l-sm," Hugging Face. https://huggingface.co/segment-any-text/sat-3l-sm

[13] K. Sirts and K. Peekman, "Evaluating Sentence Segmentation and Word Tokenization Systems on Estonian Web Texts," arXiv:2011.07868, 2020. https://arxiv.org/abs/2011.07868

[14] G. Penedo, H. Kydlíček, V. Sabolčec, B. Messmer, N. Foroutan, A. H. Kargaran, C. Raffel, M. Jaggi, L. Von Werra, and T. Wolf, "FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language," arXiv:2506.20920, 2025. https://arxiv.org/abs/2506.20920

[15] J. H. Clark, D. Garrette, I. Turc, and J. Wieting, "CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation," arXiv:2103.06874, 2021. https://arxiv.org/abs/2103.06874

[16] Z. Li and D. Hoiem, "Learning without Forgetting," in Computer Vision -- ECCV 2016, pp. 614-629, 2016. https://arxiv.org/abs/1606.09282

[17] Q. Liu, X. Zheng, N. Muennighoff, G. Zeng, L. Dou, T. Pang, J. Jiang, and M. Lin, "RegMix: Data Mixture as Regression for Language Model Pre-training," arXiv:2407.01492, 2024. https://arxiv.org/abs/2407.01492

[18] M. F. Chen, T. Murray, D. Heineman, M. Jordan, H. Hajishirzi, C. Ré, L. Soldaini, and K. Lo, "Olmix: A Framework for Data Mixing Throughout LM Development," arXiv:2602.12237, 2026. https://arxiv.org/abs/2602.12237

[19] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter," arXiv:1910.01108, 2019. https://arxiv.org/abs/1910.01108

[20] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, "Focal Loss for Dense Object Detection," arXiv:1708.02002, 2017. https://arxiv.org/abs/1708.02002

[21] J. Howard and S. Ruder, "Universal Language Model Fine-tuning for Text Classification," arXiv:1801.06146, 2018. https://arxiv.org/abs/1801.06146

[22] P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson, "Averaging Weights Leads to Wider Optima and Better Generalization," arXiv:1803.05407, 2018. https://arxiv.org/abs/1803.05407

[23] M. Wortsman et al., "Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time," arXiv:2203.05482, 2022. https://arxiv.org/abs/2203.05482

[24] F. Milletari, N. Navab, and S.-A. Ahmadi, "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation," arXiv:1606.04797, 2016. https://arxiv.org/abs/1606.04797

[25] G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv:1503.02531, 2015. https://arxiv.org/abs/1503.02531

[26] S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks," arXiv:2004.10964, 2020. https://arxiv.org/abs/2004.10964

[27] A. Znotins, "Pretraining and Benchmarking Modern Encoders for Latvian," in Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026), pp. 461-470, 2026. https://doi.org/10.18653/v1/2026.loreslm-1.40