Dissertation directed by: Marine Carpuat Department of Computer Science While much natural language processing work focuses on analyzing language content …

… the training data as in-domain and either Common Crawl or ICWSM as out-of- … Mono-TL Common Crawl 26,788,048 … Mono-SW Common Crawl 12,158,524 …


Dissertation directed by: Professor Marine Carpuat Department of Computer Science Cross-lingual resources such as parallel corpora and bilingual dictionaries are …

… Common Crawl corpus contains sentence-aligned parallel documents automati- … trained on the exact same parallel corpora (OpenSubtitles or CommonCrawl for … On the CommonCrawl test set, the examples with disagreement are more …


Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling

S Goyal, D Lopez-Paz, K Ahuja - arXiv preprint arXiv:2509.01649, 2025

… More broadly, current pretraining datasets have largely been curated from common crawl with standard next-token pretraining paradigms in mind. Moving forward, a highly promising research direction would be the development of …


Divergent thinking in groups

MK Smith, R Weller, T Duong, R McClintock… - 2025

Methods: To examine whether the severity of cold shock response impairs higherlevel thinking in a group, 29 active duty service members completed a group format Divergent Association Task (DAT; 4–5 per group) prior to and during a 13-min …


Diversity at the top: leveraging language for inclusion

M Bannò, A Franzoni, C Leggerini, M Rosola - Journal of Management and …, 2025

This study explores how gendered job titles in Italian, particularly feminine forms used for leadership positions, are represented and perceived on social media like Twitter. While feminine titles align with Italian grammatical norms, they are often …


Do Chinese models speak Chinese languages?

AW Wen-Yi, UES Jo, D Mimno - arXiv preprint arXiv:2504.00289, 2025

The release of top-performing open-weight LLMs has cemented China's role as a leading force in AI development. Do these models support languages spoken in China? Or do they speak the same languages as Western models? Comparing …


Do Not Trust Licenses You See—Dataset Compliance Requires Massive-Scale AI-Powered Lifecycle Tracing

J Kim, S Sohn, GJ Jo, J Choi, K Bae, H Lee, Y Park…

This paper argues that a dataset’s legal risk cannot be accurately assessed by its license terms alone; instead, tracking dataset redistribution and its full lifecycle is essential. However, this process is too complex for legal experts to handle manually …


DocHPLT: A Massively Multilingual Document-Level Translation Dataset

D O'Brien, B Malik, O de Gibert, P Chen, B Haddow… - arXiv preprint arXiv …, 2025

Existing document-level machine translation resources are only available for a handful of languages, mostly high-resourced ones. To facilitate the training and evaluation of document-level translation and, more broadly, long-context modeling …


Document Matching for Contradiction Detection in Low-Resource Legislative Texts With Self-Training and Augmentation Using Transformer Model

DA Navastara, S Abdillah, D Benito, IG Adillion… - Jurnal Nasional Pendidikan …, 2025

… Meanwhile, XLM-RoBERTa is a multilingual model pretrained on approximately 2.5TB of filtered CommonCrawl data across 100 languages, making it particularly effective in lowresource language settings [34]. The highest-performing model was …


Does Grammatical Gender Influence Implicit Gender Attitudes? Evidence from Sequential Bi/Multilingual Speakers from Afghanistan

… Using word embeddings from large-scale datasets like Wikipedia and Common Crawl, they analyzed associations between gender words (eg, he, she) and valence words (positive/negative terms). They found that 60% of gendered languages exhibit …


Does Synthetic Data Help Named Entity Recognition for Low-Resource Languages?

G Kamath, S Vajjala - arXiv preprint arXiv:2505.16814, 2025

Named Entity Recognition(NER) for low-resource languages aims to produce robust systems for languages where there is limited labeled training data available, and has been an area of increasing interest within NLP. Data augmentation for …


Doing More with Less--Implementing Routing Strategies in Large Language Model-Based Systems: An Extended Survey

C Varangot-Reille, C Bouvard, A Gourru, M Ciancone… - arXiv preprint arXiv …, 2025

Large Language Models (LLM)-based systems, ie interconnected elements that include an LLM as a central component (eg, conversational agents), are typically monolithic static architectures that rely on a single LLM for all user queries. However …


Domain Specific Finetuning of LLMs Using PEFT Techniques

DK Gajulamandyam, S Veerla, Y Emami, K Lee, Y Li… - 2025 IEEE 15th Annual …, 2025

As Large Language Models (LLMs) like ChatGPT and Gemini gain widespread adoption across industries, organizations increasingly seek methods to customize these models for domain-specific applications. This research evaluates four …


Domain-Adaptive Pretraining of Transformer-Based Language Models on Medical Texts: A High-Performance Computing Experiment

CK Gitonga, LG Mugao - European Journal of Information Technologies and …, 2025

This research was to investigate the effect of utilizing high-performance computing (HPC) resources to enhance the adaptability and performance of transformer-based language models. The research was done through intensive domain-specific …


Domain-Specific Text Embedding Models for Information Retrieval

Large Language Models (LLMs) have shown advanced capabilities across various fields. However, using these models out of the box, especially in specialized domains like chemistry, often leads to issues such as context limitations …


DomainHarvester: Uncovering Trustworthy Domains Beyond Popularity Rankings

D Chiba, H Nakano, T Koide - IEEE Access, 2025

… They serve as data sources for web crawling in projects like Common Crawl [39], which provides data for AI training, including models like ChatGPT. Under the assumption that popularity suggests safety, top lists have often been used as de …


Don't Let Copyright Kill American AI

… Based on the New York Times complaint, which claims tens of millions of violations of its copyrights through datasets such as Common Crawl and WebText2, if a judge embraced a maximalist approach to statutory damages, the bill could …


Don't Score too Early! Evaluating Argument Mining Models on Incomplete Essays

NJ Schaller, Y Ding, T Jansen, A Horbach

… 2020) explicitly examined sentence versus token classification for argument recognition on annotated Common Crawl data (IAA αunom = .61). Their experiments with various BERT and FLAIR models showed that a BERT_LARGE sentence …


DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off

J Zhang, Y Fan, K Cai, Z Huang, X Sun, J Wang… - arXiv preprint arXiv …, 2025

This paper introduces DrDiff, a novel framework for long-text generation that overcomes the efficiency-quality trade-off through three core technologies. First, we design a dynamic expert scheduling mechanism that intelligently allocates …


Dream-Coder 7B: An Open Diffusion Language Model for Code

Z Xie, J Ye, L Zheng, J Gao, J Dong, Z Wu, X Zhao… - arXiv preprint arXiv …, 2025

We present Dream-Coder 7B, an open-source discrete diffusion language model for code generation that exhibits emergent any-order generation capabilities. Unlike traditional autoregressive (AR) models that decode strictly left-to-right, Dream-Coder …


Dual-Modality Integration Attention with Graph-Based Feature Extraction for Visual Question and Answering

J Lu, C Wu, L Wang, R Li, X Shen - Tsinghua Science and Technology, 2025

Visual Question and Answering (VQA) has garnered significant attention as a domain that requires the synthesis of visual and textual information to produce accurate responses. While existing methods often rely on Convolutional Neural …


DUKweb, diachronic word representations from the UK Web archive corpus

Lexical semantic change (detecting shifts in the meaning and usage of words) is an important task for social and cultural studies as well as for Natural Language Processing applications. Diachronic word embeddings (time-sensitive vector …


Dutch CrowS-Pairs: Adapting a Challenge Dataset for Measuring Social Biases in Language Models for Dutch

E Strazda, G Spanakis - arXiv preprint arXiv:2507.16442, 2025

Warning: This paper contains explicit statements of offensive stereotypes which might be upsetting. Language models are prone to exhibiting biases, further amplifying unfair and harmful stereotypes. Given the fast-growing popularity and …


DutchCrows: A Benchmark for Measuring Dutch Stereotypes in Large Language Models

With Large Language Models (LLMs) increasingly used worldwide, including in the Netherlands, there is a growing need to evaluate them on harmful biases such as stereotyping. While many benchmarks exist for English, non-English bias …


DweshVaani: An LLM for Detecting Religious Hate Speech in Code-Mixed Hindi-English

V Srivastava - Proceedings of the First Workshop on Challenges in …, 2025

… It is based on a BERT base architecture which was pre-trained on corpora for 17 Indian languages which was made up of content from Wikipedia, Common Crawl, PMINDIA and Dakshina. For training, this monolingual text corpora was augmented …


Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models

A Yeginbergen, M Oronoz, R Agerri - arXiv preprint arXiv:2503.05328, 2025

This paper investigates the role of dynamic external knowledge integration in improving counter-argument generation using Large Language Models (LLMs). While LLMs have shown promise in argumentative tasks, their tendency to generate …


Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining

D Sow, H Woisetschläger, S Bulusu, S Wang… - arXiv preprint arXiv …, 2025

Pretraining large language models (LLMs) on vast and heterogeneous datasets is crucial for achieving state-of-the-art performance across diverse downstream tasks. However, current training paradigms treat all samples equally, overlooking the …


Dynaword: From One-shot to Continuously Developed Datasets

K Enevoldsen, KN Jensen, J Kostkan, B Szabó… - arXiv preprint arXiv …, 2025

… Danish Dynaword notably excludes social media data from Twitter ( 32M tokens), copyrighted samples from OpenSubtitles (<1M tokens), and common crawl segments ( 100M tokens) following the principles of traceable and open licensing …


Early Christian history

Recorded Irish history begins with the introduction of Christianity and Latin literacy, beginning in the 5th century or possibly slightly before. When compared to neighbouring Insular societies, early Christian Ireland is well documented, at least …


Early life and education

Margaret Gillespie, from an Irish Protestant family, 4 was born at Boyle, County Roscommon, 5 and educated locally and in Derry. 6 She studied music at the Royal University of Ireland in Dublin, graduating in 1902, and became a teacher. As a …


Early life

Over a long career at Kasturba Medical College, Mangalore, Hegde served in various positions such as professor, principal and dean. He was appointed the vice chancellor of Manipal Academy of Higher Education in 1999 and served till 2003. 9 …


Eco-friendly LLMs: Can memory-based large language models pave the way towards sustainable AI?

This research studies the viability of memory-based large language models (LLMs) as an eco-friendly alternative to transformer-based LLMs, addressing the increasing environmental concerns associated with the energetic demands of AI. The study …


Ecom100B: A 100 B-Token Customer-Service Corpus

R Zhao, Y Liu, C Yang, Y Huang, X Zhong - 2025

… Abstract: We introduce E-Com100B, a 100-billion-token English-centric corpus distilled from Common Crawl for pre-training customer-service-oriented language models. Following the FineWeb-Edu recipe, we prompt a lightweight scorer (Qwen3-1.7B) …


Economics of Sourcing Human Data

S Santy, P Bhattacharya, MH Ribeiro, K Allen, S Oh - arXiv preprint arXiv:2502.07732, 2025

Progress in AI has relied on human-generated data, from annotator marketplaces to the wider Internet. However, the widespread use of large language models now threatens the quality and integrity of human-generated data on these very platforms …


EDITED BY RENÉ KÖNIG AND MIRIAM RASCH INC READER# 9

… Common Crawl represents an important development. The project has done … See, http://commoncrawl.org/our-work/. …


Education and early career

BET Morgue, G Coast, N Ghanaian, E Sutherland

Returning to Ghana in 1951, she taught first at Fijai Secondary School at Sekondi, then at St. Monica's School (1951–54), and also began writing for children. 24 She would later say:" I started writing seriously in 1951. I can even remember the precise …


Effectiveness of Bi-GRU and FastText in Sentiment Analysis of Shopee App Reviews

RF Rahmanda, Y Sibaroni, SS Prasetiyowati - Sinkron: jurnal dan penelitian teknik …, 2025

… This research uses a pre-trained corpus trained from Wikipedia and Common Crawl with the CBOW approach. The corpus is used to expand features through Top-n similar words available in the corpus (Abasan & Setiawan, 2024). The model can …


Efficiency in Computer Vision: From Compute and Memory to Robustness

Deep learning has been the biggest success story in the past decade. It is now part of nearly every facet of our lives. Along with growth in popularity, deep learning has also seen growth in terms of model and data sizes and the scale of their training …


Efficient Elicitation of Fictitious Nursing Notes from Volunteer Healthcare Professionals

JV Bornerup, C Hardmeier - Proceedings of the Joint 25th Nordic Conference on …, 2025

Reliable automatic solutions to extract structured information from free-text nursing notes could bring important efficiency gains in healthcare, but their development is hampered by the sensitivity and limited availability of example data. We describe a …


Efficient Expert Pruning for Pre-Training of Mixture-of-Experts Large Language Models

S Wu, J Luo, T Yu, X Chen, S Wang, X Zhao, L Li… - 2025

… Our work shows that a larger proportion of data are necessary to support the model in pre-training to obtain satisfactory code capabilities, and that appropriately reducing text data, especially the reduction of low-quality text data (eg common crawl) …


Efficient Hate Speech Detection: Evaluating 38 Models from Traditional Methods to Transformers

M Abusaqer, J Saquer, H Shatnawi - Proceedings of the 2025 ACM Southeast …, 2025

The proliferation of hate speech on social media necessitates automated detection systems that balance accuracy with computational efficiency. This study evaluates 38 model configurations in detecting hate speech across datasets ranging from 6.5K …


Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining: Method, Evaluation and Applications

S Kim, Y Na, K Kim, H Cho, G Lim, M Kim, S Park… - arXiv preprint arXiv …, 2025

The emergence of open-source large language models (LLMs) has expanded opportunities for enterprise applications; however, many organizations still lack the infrastructure to deploy and maintain large-scale models. As a result, small LLMs (sLLMs) …


Efficient Large Language Models with Conditional Computation

This PhD thesis examines the potential of improving the efficiency of language models by incorporating Conditional Computation into the architecture. Deep Learning methods have taken over the Natural Language Processing (NLP) field in …


Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration

T Bai, L Yang, ZH Wong, F Sun, X Zhuang, J Peng… - Proceedings of the 63rd …, 2025

… As shown in Figure 4, we first cluster 1.4 billion documents obtained from Common Crawl (Project… This diagram shows the process of training a BERT-based topic classifier using CommonCrawl data. 1.44 billion documents are clustered to …


Efficient Scaling of Language Models

Large language models (LLMs) are progressively reshaping how humans interact with information, offering increasingly sophisticated access to knowledge through natural language interfaces and advancing reasoning capabilities across diverse …


Efficient Sign Language Recognition with Skeleton Data: A Study of Keypoint Selection, Pose Estimators, and GCN Models

VHT Anh, Q Le Duc, TO Nguyen - … Conference on Multimedia Analysis and Pattern …, 2025

… Using FastText [18] trained on the Common Crawl dataset to obtain feature representations for each word, with each word represented by a 300-dimensional vector. The feature representation of the entire vocabulary in the dataset is denoted as E ∈ RN×300 …


EfficientLLM: Efficiency in Large Language Models

Z Yuan, W Sun, Y Liu, H Zhou, R Zhou, Y Li, Z Zhang… - arXiv preprint arXiv …, 2025

Large Language Models (LLMs) have driven significant progress, yet their growing parameter counts and context windows incur prohibitive compute, energy, and monetary costs. We introduce EfficientLLM, a novel benchmark and the first …


EfficientUICoder: Efficient MLLM-based UI Code Generation via Input and Output Token Compression

J Xiao, Z Zhang, Y Wan, Y Huo, Y Liu, MR Lyu - arXiv preprint arXiv:2509.12159, 2025

… Design2Code [25] introduced the first real-world benchmark with 484 manually curated web pages from Common Crawl, while WebCode2M [9] scaled this to 20,000 samples for comprehensive training and evaluation. Specialized …


Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects

X Chen, X Wang, A Colacelli, M Lee, L Xie - arXiv preprint arXiv:2509.07218, 2025

The rapid growth of artificial intelligence (AI) is driving an unprecedented increase in the electricity demand of AI data centers, raising emerging challenges for electric power grids. Understanding the characteristics of AI data center loads and their …


Embedding-Based Deep Learning Frameworks for Multimodal Oncology Data Integration

This dissertation presents a cohesive set of novel frameworks developed to address critical challenges in oncology data integration, representation learning, and clinical information extraction. The work encompasses four interconnected projects: MINDS (Multimodal …


Embryology of a Language Model

G Wang, G Baker, A Gordon, D Murfet - arXiv preprint arXiv:2508.00331, 2025

Understanding how language models develop their internal computational structure is a central problem in the science of deep learning. While susceptibilities, drawn from statistical physics, offer a promising analytical tool, their full potential for …


Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective

S Han, J Song, J Gore, P Agrawal - Forty-second International Conference on Machine …

Autoregressive transformers exhibit adaptive learning through in-context learning (ICL), which begs the question of how. Prior works have shown that transformers represent the ICL tasks as vectors in their representations. In this paper, we leverage the …


Emerging Properties in Unified Multimodal Pretraining

C Deng, D Zhu, K Li, C Gou, F Li, Z Wang, S Zhong… - arXiv preprint arXiv …, 2025

… We build upon OmniCorpus [39], a large-scale dataset preprocessed from Common Crawl [14], which provides a vast collection of web documents with interleaved text and images. We additionally include open-source image editing …


Emotion-based Multimodal Music Classifier for Recommender Systems

In recent years, advancements in artificial intelligence have driven a growing demand for personalized user experiences across various digital platforms. In the music domain, this trend is reflected in the need for more sophisticated …


Empirical Evaluation of Knowledge Distillation from Transformers to Subquadratic Language Models

P Haller, J Golde, A Akbik - arXiv preprint arXiv:2504.14366, 2025

Knowledge distillation is a widely used technique for compressing large language models (LLMs) by training a smaller student model to mimic a larger teacher model. Typically, both the teacher and student are Transformer-based architectures …


Employing device inventory and failure data for test configuration discovery and device utilization

Within network integration testing, device capacity is a key constraint for comprehensive testing. This thesis investigates how fault tickets and device inventory data could be utilized to automatically discover test configurations …


Empowering Enterprises with Lightweight Large Language Models: Automated" Rule Card" Extraction from Grant Documents

In the face of rapidly expanding unstructured data, organizations, especially small and medium-sized enterprises (SMEs), require automated solutions that not only offer accurate information extraction but also preserve data privacy. This thesis …


Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

W An, J Nie, Y Wu, F Tian, S Lu, Q Zheng - arXiv preprint arXiv:2508.10955, 2025

… DATACOMP [37] utilizes cc2dataset, an Apache Spark-based library, to extract pairs of image URLs and nonempty alttext from all Common Crawl snapshots to collect image-text data pairs. LAION [36] collects multimodal data from the Common …


Empowering Sentiment Analysis in African Low-Resource Languages through Transformer Models and Strategic Language Selection

N Raychawdhary, S Bhattacharya, C Seals, G Dozier - IEEE Access, 2025

… The training data was primarily sourced from the Common Crawl Corpus and articles from the BBC news platform. … XLM-R [31] is the multilingual [36] version of RoBERTa, pretrained on a massive dataset that includes 2.5 terabytes of filtered …


Empowering sentiment analysis in social media: a comprehensive approach to enhance the classification of abusive Tamil comments using transformer models

M Subramanian… - Journal of Big Data, 2025

… A multilingual variant of RoBERTa is referred to as XLMRoBERTa [57], which was trained on a substantial 2.5 TB dataset sourced from CommonCrawl, encompassing content from 100 different languages. Similar to the RoBERTa model, XLMRoBERTa …


Encoding/decoding in artificial intelligence: Global AI and local languages

… The top languages on the Common Crawl dataset of web pages are English (44%), followed by other European languages (20%), Russian (6%), Japanese (5%), and Chinese (5%) (Common Crawl, 2025). In the field of artificial intelligence, the …


EnerGIZAr: Leveraging GIZA++ for Effective Tokenizer Initialization

P Singh, E Agirre, G Azkune, O De Clercq, E Lefever - Findings of the Association for …, 2025

… Wikipedia was chosen over CommonCrawl, C4 or OSCAR as it significantly reduces the duration of experimentation, allowing us to iterate … size of Common Crawl for Hindi is approximately 1.8 billion tokens – roughly 40 times larger. While …


Engineering AI Systems: Architecture and DevOps Essentials

Chapter 1: Introduction Chapter 2: Software Engineering Background Chapter 3: AI Background Chapter 4: Foundation Models Chapter 5: AI Model Lifecycle Chapter 6: System Lifecycle Chapter 7: Reliability Chapter 8: Performance Chapter 9: Security …


Enhanced phishing URL identification using an integrated attention-based LSTM-CNN with hybrid features

SK Birthriya, P Ahlawat, AK Jain - International Journal of Security and Networks, 2025

Phishing attacks continue to pose a significant threat to online security, targeting users' personal and financial information through deceptive URLs and websites. This study proposes a robust hybrid deep learning model for phishing URL detection …


Enhancement of Implicit Emotion Recognition in Arabic Text: Annotated dataset and baseline models

H Boutouta, A Lakhfif, F Senator, C Mediani - IEEE Access, 2025

… The model was trained on 2.5 TB of Wikipedia and CommonCrawl data spanning 100 languages, including Arabic. … The model was trained on various public data sources, including CommonCrawl, GitHub, and Wikipedia web pages. …


Enhancing Bidirectional Encoder Representations From Transformers (BERT) With Frame Semantics to Extract Clinically Relevant Information From German …

D Reichenpfader, J Knupp, SU von Däniken, R Gaio… - Journal of Medical Internet …, 2025

… GPT-3, the predecessor of GPT-4, however, was pretrained on the CommonCrawl dataset of 45 TB of data [12]. According to Zhang et al [51], pretraining PaLM [a decoder-based model] requires around 2.5×10 24 floating-point operations and …


Enhancing Cross-Language Understanding: A Machine Learning-Based Approach to Multilingual Identification

T Harinadh, MA Valli, AV Chowdary, NJKS Abhinav…

In today’s interconnected world, multilingual identification has become an essential component of various applications, ranging from search engines and virtual assistants to online customer support and security surveillance. As globalization …


Enhancing Embedding-based Product Search using Large Language Models and Synthetic Query

E-commerce platforms have become integral components of daily life, requiring advanced retrieval systems to ensure an optimal user experience. Traditional lexical term matching methods, such as BM25, struggle with semantic nuances and fail to …


Enhancing Fake News Detection with Transformer Models and Summarization

A Saadi, H Belhadef, A Guessas, O Hafirassou - Engineering, Technology & Applied …, 2025

This study evaluates the performance of transformer-based models such as BERT, RoBERTa, and XLNet for fake news detection. Using supervised and unsupervised deep learning techniques, we optimized classification accuracy while reducing …


Enhancing Hate Speech Detection in Mixed-Language Texts: A Comparative Study of BLOOM and XLM-RoBERTa Models

A Wicaksana, K Sorensen, F Dinarta - 2025 17th International Conference on …, 2025

Hate speech detection is essential in combating online toxicity, particularly in mixed-language or code-switched texts prevalent on social media. Traditional natural language processing (NLP) models often struggle with these complex linguistic structures due …


Enhancing Health Information Retrieval with RAG by Prioritizing Topical Relevance and Factual Accuracy

R Uapadhyay, M Viviani - arXiv preprint arXiv:2502.04666, 2025

The exponential surge in online health information, coupled with its increasing use by non-experts, highlights the pressing need for advanced Health Information Retrieval models that consider not only topical relevance but also the factual …


Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation

S Singh, R Mishra, US Tiwary - arXiv preprint arXiv:2507.16002, 2025

One major challenge in natural language processing is named entity recognition (NER), which identifies and categorises named entities in textual input. In order to improve NER, this study investigates a Hindi NER technique that makes use of Hindi-specific …


Enhancing Impression Change Prediction in Speed Dating Simulations Based on Speakers' Personalities

K Matsuo, Y Ishii, A Otsuka, R Ishii, H Sugiyama… - arXiv preprint arXiv …, 2025

This paper focuses on simulating text dialogues in which impressions between speakers improve during speed dating. This simulation involves selecting an utterance from multiple candidates generated by a text generation model that …


Enhancing Interactive English Learning through Embedded Artificial Intelligence Technology

X Chen - International Journal of High Speed Electronics and …, 2025

The purpose of this research is to assess the extent to which an AI-integrated, self-regulated English learning system is effective in enhancing learners’ grammar, vocabulary, fluency, and pronunciation skills. To prove the effectiveness of AI in language …


Enhancing Khmer-English Machine Translation via Document Analysis Techniques

R Buoy, S Chenda, N Taing, M Kong, M Iwamura…

While modern communication technologies have made online information more accessible to Cambodians, the lack of robust Khmer machine translation (MT) tools continues to limit the serviceability of content that is primarily available in English …


Enhancing Language Models via HTML DOM Tree for Text Structure Understanding

H Xing, Z Shao, F Gao, J Bu, Z Yu, Q Zheng, J Gu, X Liu - IEEE Transactions on Audio …, 2025

… 1) Data: Our corpus is built from the Common Crawl dataset1 and Wikipedia dumps2. We randomly select approximately 3 million web pages from Common Crawl, and use the similar Wikipedia corpus with BERT. We remove the pages …


Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks

H Shen, B Huang, X Wan - arXiv preprint arXiv:2507.06274, 2025

Watermarking is a promising defense against the misuse of large language models (LLMs), yet it remains vulnerable to scrubbing and spoofing attacks. This vulnerability stems from an inherent trade-off governed by watermark window size: smaller windows …


Enhancing LLMs via High-Knowledge Data Selection

F Duan, X Zhang, S Wang, H Que, Y Liu, W Rong… - Proceedings of the AAAI …, 2025

The performance of Large Language Models (LLMs) is intrinsically linked to the quality of its training data. Although several studies have proposed methods for high-quality data selection, they do not consider the importance of knowledge richness in text …


Enhancing Low-Resource Language Performance in Multilingual Large Language Models

… Wikipedia and CommonCrawl are extremely larger than other parallel datasets. For AMBER, we only list the parallel data size of the … Leveraging the implementation within the Hugging Face framework [103], this model was trained on …


Enhancing Masked Language Modeling in BERT Models Using Pretrained Static Embeddings

A Mištera, P Král - International Conference on Text, Speech, and …, 2025

This paper explores the integration of pretrained static fastText word vectors into a simplified Transformer-based model to improve its efficiency and accuracy. Despite the fact that these embeddings have been outperformed by large models based on …


ENHANCING MULTILINGUAL COMMUNICATION WITH MACHINE LEARNING-DRIVEN LANGUAGE IDENTIFICATION

RS Sirisati, V Navyasri, T Swathi, M Akhila

Multilingual identification is a vital component in enabling seamless communication across linguistically diverse populations. This study presents the development of an efficient language identification system leveraging machine learning algorithms—namely …


Enhancing Multilingual Information Extraction Towards Global Linguistic Inclusivity In our interconnected world, the diversity of around 7,000 languages presents …

M Nguyen, TH Nguyen, TH Nguyen, D Lowd, K Kyle

… Common Crawl’s 2019 and 2020 releases. The pages were split into passages …


Enhancing Multilingual LLM Pretraining with Model-Based Data Selection

B Messmer, V Sabolčec, M Jaggi - arXiv preprint arXiv:2502.10361, 2025

… (2020) already observed the importance of using a cleaned version of Common Crawl for improved performance, the high cost of LLM training has further motivated research into better … In order to pretrain LLMs on a large amount of diverse texts …


Enhancing NLP for Indic Languages With Limited Resources: A Study of Transformer Models for Translation and Summarization

A Bhati, VK Gupta, HR Shah, R Nagar, S Nimawat - 2025 International Conference on …, 2025

… Clean Crawled Corpus (C4), a filtered subset of the Common Crawl [5] dataset that is publicly available, containing large-scale web crawled data. This approach allows T5 to deal with a broad range of tasks - translation, summarization, and …


Enhancing Password Security and Memorability Using Machine Learning and Linguistic Patterns

In the digital age, text-based passwords remain a primary method for securing online accounts. Yet, users frequently face a dilemma between creating passwords that are easy to remember and sufficiently secure against cyberattacks. This …


Enhancing Product Categorization with LLMs

This research explored techniques to improve Large Language Models performance for Hierarchical Product Classification (HPC), including optimized fine-tuning, optimal prompting techniques, taxonomy-specific Knowledge Graphs, leveraging …


Enhancing Product Categorization with Retrieval Augmented Generation: A Comparative Study of Architectures, Techniques and Strategies


Enhancing QA System Evaluation: An In-Depth Analysis of Metrics and Model-Specific Behaviors

H Kim, A Ademola - Journal of Information Science Theory and Practice, 2025

The purpose of this study is to examine how evaluation metrics influence the perception and performance of question answering (QA) systems, particularly focusing on their effectiveness in QA tasks. We compare four different models: BERT …


Enhancing QoS in Edge Computing through Federated Layering Techniques: A Pathway to Resilient AI Lifelong Learning Systems

C Han - arXiv preprint arXiv:2507.20444, 2025

… Datasets used were diverse, including ImageNet for image classification, along with OpenSubtitles and CommonCrawl for text analysis tasks. … ImageNet, OpenSubtitles, and CommonCrawl were used for various classification tasks …


Enhancing Rumor Detection Methods with Propagation Structure Infused Language Model

C Cui, S Li, K Ma, C Jia - Proceedings of the 31st International Conference on …, 2025

… 2015) or Project Gutenberg1), Wikipedia corpora, and web-crawled data (like Common Crawl2), with language that is generally more formal, grammatically correct, and skewed towards the written form. However, texts in posts on social platforms is …


Enhancing Sentiment-driven Recommender Systems with LLM-Based Feature Engineering: A Case Study in Drug review Analysis

SM Kangoni, OT Tshipata, PS Nzakuna, V Paciello… - IEEE Access, 2025

… For this study, we employed 300-dimensional FastText embeddings pre-trained on the Common Crawl corpus, which consists of 2 million word vectors. These embeddings capture subword information by breaking down words into n-grams …


Enhancing Skin Disease Classification with MetaCLIP Models: A Multi-Dataset Analysis Across 15 Sources

BC Mohan, TN Malleswari, O Oyerinde, K Bagadi… - Authorea Preprints, 2025

… This “Quality Data” is the output of the Curation Algorithm, that is, a cleaned dataset that was much better, cleaner and much less dirty compared to the Common Crawl. High-quality data will likely serve as the foundation for developing a more …


Enhancing Small Language Models for Graph Tasks Through Graph Encoder Integration

D Oh, S Kang, H Kim, D Oh - Applied Sciences, 2025

Small language models (SLMs) are increasingly utilized for on-device applications due to their ability to ensure user privacy, reduce inference latency, and operate independently of cloud infrastructure. However, their performance is often limited …


Enhancing text quality evaluation with integrating content security attributes

Y Sun, J Zhao - Expert Systems with Applications, 2025

With the rapid development of large language models (LLMs), the problem of generating unsecure and illegal content has become increasingly severe. Therefore, it is especially important to evaluate the security and compliance of content …


Enhancing waste recognition with vision-language models: A prompt engineering approach for a scalable solution

HJ Malla, M Bazli, M Arashpour - Waste Management, 2025

Conventional unimodal computer vision models, trained on limited bespoke waste datasets, face significant challenges in classifying waste images in material recovery facilities, where waste appears in diverse forms. Maintaining performance of these …


Enigma@ ElCardioCC: bridging NER and ICD-10 entity linking-A hybrid method for greek clinical narratives

B Velichkov, A Datseris, S Vassileva, S Boytcheva - CLEF, 2025

… , European Parliament Proceedings Parallel Corpus, and the Greek portion of filtered CommonCrawl. It has shown improved results on the general domain Greek NER task. • XLM-RoBERTa Large [3] 21 - a multilingual model, trained on 2.5TB of …


Enough Coin Flips Can Make LLMs Act Bayesian

R Gupta, R Corona, J Ge, E Wang, D Klein, T Darrell… - arXiv preprint arXiv …, 2025

Large language models (LLMs) exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning (ICL). We investigate whether LLMs utilize ICL to perform structured reasoning in ways that …


Enriched Image Captioning based on Knowledge Divergence and Focus

AA Liu, Q Wu, N Xu, H Tian, L Wang - IEEE Transactions on Circuits and Systems for …, 2025

… For instance, GPT-3 [5] boasts a massive 175 billion parameters and has been trained on extensive text data corpora from various sources including Common Crawl [52], WebText2, books [5], and Wikipedia, providing it with a broad knowledge …


Enriching sentence-level machine translation

Neural Machine Translation (MT) has long been established as a successful paradigm to produce high-quality MT across many languages and domains. However, it suffers from one significant limitation – it is too often formulated as a task …


Ensemble Machine Learning Algorithms for Fake News Detection

G Preethi, S Vasanthakumar - 2025 3rd International Conference on Sustainable …, 2025

The fake news is becoming more complex, and this presents a severe threat to the credibility and faith of digital citizens. The present paper relies on the idea of forming a hybrid ensemble model that would enhance the accuracy and interpretation of …


Ensembling Sparse Autoencoders

S Gadgil, C Lin, SI Lee - arXiv preprint arXiv:2505.16077, 2025

… Its diverse components include academic papers (eg, arXiv, PubMed Central), books (eg, Books3, BookCorpus2), code (from GitHub), web content (eg, a filtered version of Common Crawl called Pile-CC, OpenWebText2), and other sources like …


Entity-aware Cross-lingual Claim Detection for Automated Fact-checking

R Panchendrarajan, A Zubiaga - arXiv preprint arXiv:2503.15220, 2025

… XLMR was trained on CommonCrawl data supporting 100 languages, and mBERT was trained on Wikipedia data containing 104 languages. Both models generate embeddings of size 768 for each tokenized word. …


Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations

PA Irawan, R Diandaru, BJB Syuhada, RZ Suchrady… - arXiv preprint arXiv …, 2025

We introduce Entropy2Vec, a novel framework for deriving cross-lingual language representations by leveraging the entropy of monolingual language models. Unlike traditional typological inventories that suffer from feature sparsity and static …


ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining

MI Bal, V Cevher, M Muehlebach - arXiv preprint arXiv:2505.19893, 2025

Large language model pretraining is compute-intensive, yet many tokens contribute marginally to learning, resulting in inefficiency. We introduce Efficient Selective Language Modeling (ESLM), a risk-aware algorithm that improves training efficiency …


Essential-Web v1. 0: 24T tokens of organized web data

A Hojel, M Pust, T Romanski, Y Vanjani, R Kapila… - arXiv preprint arXiv …, 2025

… In addition to evaluating EAI-TAXONOMY in domains with existing, multi-billion-token baseline datasets curated from Common Crawl, we also select a domain where we wish there was a large-scale dataset available. Given the importance of STEM …


Ethical Challenges and Bias in NLP Models: A Python-Based Investigation

… Many NLP models are trained on large corpora scraped from the internet, such as Wikipedia, news articles, or Common Crawl. These sources often reflect real-world disparities in representation—underrepresenting minorities or reinforcing gender …


Ethical Issues in Large Language Models: A Systematic Literature Review

Large Language Models (LLMs), and Generative AI (GenAI) more generally, have been the center of much attention in both media and research following recent technical advances. In the wake of the recent surge of users services like ChatGPT …


ETHICS-2025 Session G1-Panel: Scraping the Surface: Ethical Collection Practices in the Age of AI: ETHICS-2025 Special Session, Sunday, June 8 2025, 2: 45-4: 15 …

G Lindahl, D Mazia, L Rosenthol, J Levy, K Natana - 2025 IEEE International …, 2025

… By freeing up access, Lindahl explained, Common Crawl hopes to equalize opportunity across research institutions and industries. However, he acknowledged that even responsibly collected public data can provoke controversy, especially …


Evaluating and comparing gender bias across four text-to-image models

Z Hammad, NL Sowah - arXiv preprint arXiv:2509.08004, 2025

As we increasingly use Artificial Intelligence (AI) in decision-making for industries like healthcare, finance, e-commerce, and even entertainment, it is crucial to also reflect on the ethical aspects of AI, for example the inclusivity and fairness of the …


Evaluating Binary Decision Biases in Large Language Models: Implications for Fair Agent-Based Financial Simulations

A Vidler, T Walsh - arXiv preprint arXiv:2501.16356, 2025

… To gain a context on natural language bias, we perform a sampling of the Common crawl (Common Crawl 2024) as recent research by (Tessema, Kedia, and Chung 2024) has found that it can provide a valuable data source for fine tuning …


Evaluating Code-Mixing in LLMs Across 18 Languages

Y Yang, Y Chai - arXiv preprint arXiv:2507.18791, 2025

Code-mixing, the practice of switching between languages within a conversation, presents unique challenges for traditional natural language processing. Existing benchmarks, such as LinCE and GLUECoS, are limited by narrow language …


Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning

T Mackintosh, HT Madabushi, C Bonial - arXiv preprint arXiv:2509.16422, 2025

We probe large language models' ability to learn deep form-meaning mappings as defined by construction grammars. We introduce the ConTest-NLI benchmark of 80k sentences covering eight English constructions from highly lexicalized to highly …


Evaluating Dutch Speakers and Large Language Models on Standard Dutch: a grammatical Challenge Set based on the Algemene Nederlandse Spraakkunst

J Pestel, J Bloem, RG Alhama - Computational Linguistics in the Netherlands Journal, 2025

This study evaluates the linguistic knowledge of Dutch Large Language Models (LLMs) by introducing a novel challenge set based on the Algemene Nederlandse Spraakkunst (ANS). The ANS is a comprehensive resource of Dutch prescriptive …


Evaluating GPT-and Reasoning-based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Implications for Educational …

P Tschisgale, H Maus, F Kieser, B Kroehs, S Petersen… - arXiv preprint arXiv …, 2025

… Moreover, many of the problems are not publicly shared through the Internet, and thus likely not part of the Common Crawl of the Internet, which is part of the training data for LLMs. The average difficulty of the problems increases across the …


Evaluating Large Language Models as Raters in Large-Scale Writing Assessments: A Psychometric Framework for Reliability and Validity

Y Wang, J Huang, L Du, Y Guo, Y Liu, R Wang - Computers and Education: Artificial …, 2025

In large-scale international writing assessments, human raters often exhibit inconsistency, undermining reliability and validity. Large language models (LLMs) offer a potential solution, but their assessment reliability remains underexplored …


Evaluating Large Language Models in Mongolian

This paper presents a comprehensive evaluation for assessing large language model (LLM) capabilities in the Mongolian language, addressing a critical gap in multilingual LLM evaluation. We introduce MonMLU, a novel benchmark derived …


Evaluating LLMs on Chinese Idiom Translation

C Yang, Y Dou, D Heineman, X Wu, W Xu - arXiv preprint arXiv:2508.10421, 2025

Idioms, whose figurative meanings usually differ from their literal interpretations, are common in everyday language, especially in Chinese, where they often contain historical references and follow specific structural patterns. Despite recent progress …


Evaluating Multimodal AI Systems: A Comparative Analysis of Large Languagel Model-Based Models for Text, Image, and Video Generation

In the era of rapid technological advancement, efficient content generation, application development, and data management are crucial for meeting the demands of dynamic digital environments. This thesis uses state-of-the-art models to …


Evaluating the Ability of Large Language Models to Self-Improve on Forecasting Future Events

This thesis investigates whether large language models (LLMs) can improve their forecasting capabilities through self-reflection. While recent studies have explored LLMs' ability to predict future events, they typically rely on a limited set of human-crafted …


EVALUATING THE ACCURACY OF GENERATIVE AI IN PRODUCING CULTURALLY RELEVANT RECIPE RECOMMENDATIONS FOR HEALTH APPLICATIONS

Culturally relevant meal recommendations have been shown to aid in long-term diet adherence and positive attitudes towards healthy eating. However, as interest in AI-driven food and health recommendations grows, the cultural accuracy of popular large …


Evaluating the Effectiveness of Existing Phishing Detectors on AI Generated Phishing Emails

This research evaluates the effectiveness of phishing detection models when faced with phishing emails rephrased by large language models (LLMs). The findings show that LLM-generated rephrased emails impact detection accuracy, though the …


Evaluating the Efficacy of Machine Learning and Neural Networks in Cross-Language Translation and Security Applications

A Zokirjon, E Shokhruza, G Aggarwal, D Ather - Data Mining and Information Security …, 2025

In this paper, comparative analysis of various machine learning (ML) algorithms and neural network (NN) for cross-language translation and security applications is discussed in detail. The development of the Internet as a medium for international …


Evaluating the Impact of Advanced LLM

S Kahl¹, F Löffler, M Maciol, F Ridder, M Schmitz… - AI in Education and Educational …

This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation …


Evaluating the Impact of Data Scarcity on Model Performance in a Low-Resource Afrikaans Question Answering Model

TG Moape, F Mthombeni, A Stoman - 2025 Conference on Information …, 2025

This paper evaluates the impact of data scarcity on a generative question-answering (QA) model for Afrikaans, using a hybrid architecture that combines BERT for contextual encoding and GPT-2 for generation. Trained on a limited dataset of 1,800 …


Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain

S Amirshahi, A Bigdeli, CLA Clarke, A Ghenai - arXiv preprint arXiv:2509.03787, 2025

… The TREC 2020 track consists of 46 queries on COVID-19 treatments (eg, “Can pneumococcal vaccine prevent COVID-19?”), with candidate documents sourced from the Common Crawl News dataset2, covering the early months of the pandemic …


Evaluating Virtual Reality and Artificial lntelligence as Emerging Digital Tools for Mental Health Care

… COMMON CRAWL Common Crawl is a nonprofit organization that provides alarg e, freely accessible repository of web crawl data. This … Founded in 2007, Common Crawl has continuously collected and archived web data, gathering petabytes of …


Evaluation of a Node-based Automatic Short Answer Tool “NodeGrade”

DV Fischer, J Haug, P Schoppel, J Abke, M Becker… - Proceedings of the 6th …, 2025

NodeGrade tries to provide a suitable solution for the problem of time-intensive short answer grading. This research focuses simultaneously on performance, functionality and user experience, which is underlined by a triangulated approach. The …


Evaluation of the phi-3-mini SLM for identification of texts related to medicine, health, and sports injuries

C Brogly, S Rjaibi, C Liang, E Lam, E Wang, A Levitan… - arXiv preprint arXiv …, 2025

Small Language Models (SLMs) have potential to be used for automatically labelling and identifying aspects of text data for medicine/health-related purposes from documents and the web. As their resource requirements are significantly lower than …


Even Small Reasoners Should Quote Their Sources: Introducing the Pleias-RAG Model Family

PC Langlais, P Chizhov, M Nee, CR Hinostroza… - arXiv preprint arXiv …, 2025

We introduce a new generation of small reasoning models for RAG, search, and source summarization. Pleias-RAG-350m and Pleias-RAG-1B are mid-trained on a large synthetic dataset emulating the retrieval of a wide variety of multilingual open …


Event knowledge and object-scene knowledge jointly influence fixations in scenes

S Heer, MA Pedziwiatr, P Bex, A Coutrot, I Mareschal - Visual Cognition, 2025

Viewers of real-world scenes typically have knowledge about preceding events (“event knowledge”) and the relationships between objects (“object-scene knowledge”). We examined how these knowledge types interact to influence gaze. We recorded eye …


Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs

L Team, B Zeng, C Huang, C Zhang, C Tian, C Chen… - arXiv preprint arXiv …, 2025

… The majority of raw data used in this study were obtained from publicly available sources, including Common Crawl (CC), coding platforms, and … The data collection process leveraged publicly available repositories such as Common Crawl …


Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM

L Team, W Cai, Y Cao, C Chen, C Chen, S Chen, Q Cui… - arXiv preprint arXiv …, 2025

… For instance, we convert the Common Crawl data from HTML pages into plain text, and … Specifically, we leverage open-source LLMs to score randomly sampled Common Crawl pages … Notably, we find that the quality of parsing raw Common …


Evidencing Unauthorized Training Data from AI Generated Content using Information Isotopes

Q Tao, Y Jinhua, C Dongqi, X Yueqi, W Huili… - arXiv preprint arXiv …, 2025

… Half of these articles were published in 2022 (a time period commonly associated with AI training datasets) and can be found in the common crawl dataset (a public dataset widely used for training the large language models)68, serving as the …


Examining the Impact and Limitations of Distributed Large Language Models and Multimodal Systems

… Large-scale corpora such as C4, The Pile, and Common Crawl provide extensive text data from books, web pages, and code repositories. However, data quality filtering is crucial to remove noise and biases. Additionally, tokenization strategies …


Expanding the paradigm: Generative artificial intelligence and US privacy norms

E Zeide - Cambridge Forum on AI: Law and Governance, 2025

Generative artificial intelligence (AI) systems, such as large language models, image synthesis tools, and audio generation engines, present remarkable possibilities for creative expression and scientific discovery but also pose pressing challenges for …


ExPe: Exact Positional Encodings for Generative Transformer Models with Extrapolating Capabilities

A Datseris, S Vassileva, I Koychev, S Boytcheva - arXiv preprint arXiv:2509.19569, 2025

This paper introduces a novel approach to position embeddings in transformer models, named "Exact Positional Embeddings" (ExPE). An absolute positional embedding method that can extrapolate to sequences of lengths longer than the …


Expert in the Loop: LLM Assistance for Technical Documentation Writing Case Study at Saab AB

… For instance, in the training methodology of previous-generation GPT-3 (Brown at al., 2020), a filtered version of the Common Crawl1 dataset was used as a part of the complete training data. This Common Crawl subset alone consisted of around 410 …


Explainability and Interpretability of Multilingual Large Language Models: A Survey

Multilingual large language models (MLLMs) demonstrate state-of-the-art capabilities across diverse cross-lingual and multilingual tasks. Their complex internal mechanisms, however, often lack transparency, posing significant …


Explainable Prediction of User Post Popularity: An Analysis of the One Million Posts Corpus

Discussions in newspaper comment sections significantly influence public opinion. The methods used to sort and display user posts impact these discussions and can propagate certain opinions. However, sorting is often partially done by forum …


Explaining How Visual, Textual and Multimodal Encoders Share Concepts

C Cornet, R Besançon, HL Borgne - arXiv preprint arXiv:2507.18512, 2025

… DFN is a CLIP-like model trained from 2B image-text pairs, resulting from the filtering of a pool of 12.8 billion uncurated image-text pairs of CommonPool, collected from Common Crawl. This last is itself part of DataComp, a benchmark for …


ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models

M Miliani, S Auriemma, A Bondielli, E Chersoni… - arXiv preprint arXiv …, 2025

Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. In this paper, we introduce ExpliCa, a new dataset for evaluating LLMs in explicit causal reasoning. ExpliCa uniquely integrates both …


Exploring a Gamified Personality Assessment Method through Interaction with Multi-Personality LLM Agents

B Zhang, X Li, C Zhou, X Gai, Z Liao, J Liu, X Yang… - arXiv preprint arXiv …, 2025

The execution of effective and imperceptible personality assessments is receiving increasing attention in psychology and human-computer interaction fields. This study explores an interactive approach for personality assessment, focusing on the …


Exploring AI Tools and Large Language Models for Students' Performance Enhancement in Riddle Based Logical Reasoning

… RoBERTa is a highly optimized variant of BERT that has been trained on a much larger corpus (Common Crawl) for much longer training duration than BERT. RoBERTa can absorb this much larger amount of training data, allowing it to more …


Exploring Approaches for Measuring Risk in the News

This paper presents a preliminary investigation into automatic approaches that rely solely on content-based features to compute indicators such as the “risk indicator”, which aims to provide a measure of the extent to which risk is present/evoked in a …


Exploring Causes of Representational Similarity in Machine Learning Models

ZM Li, HA Vu, D Awofisayo, E Wenger - arXiv preprint arXiv:2505.13899, 2025

… For example, many models are trained on subsets of Common Crawl, a massive dataset composed of most of the internet. Many are also … For example, GPT [4], Jamba [58], Llama [60], PaLM [7], and Phi [1] are all trained on subsets of …


Exploring Gen-AI applications in building research and industry: A review

H Wan, J Zhang, Y Chen, W Xu, F Feng - Building Simulation, 2025

This paper investigates the transformative potential of Generative AI (Gen-AI) technologies, particularly large language models, within the building industry. By leveraging these advanced AI tools, the study explores their application across key …


Exploring geometric compression across languages in multilingual language models

This study explores geometric compression of linguistic data across languages in multilingual language models using the Europarl corpus, focusing on three models: BLOOM, XLMRoBERTa, and Mistral. We estimate the intrinsic dimension (ID) of …


Exploring LLM Embedding Potential for Dementia Detection Using Audio Transcripts

BA Llaca-Sánchez, LR García-Noguez… - Eng, 2025

… The GloVe model—trained on data from Wikipedia 2014, Gigaword 5 archive of newswire text data, and the Common Crawl web pages dataset—is based on a co-occurrence matrix constructed from a large text corpus, which captures how frequently pairs of …


Exploring Multimodal Humor Detection in Latin-American Spanish with LS-FUNNY

E Herrera-Alba, R Manrique - SN Computer Science, 2025

… To encode each transcript we employed the XLM-RoBERTa transformer [13], a multilingual extension of RoBERTa pre-trained on 2.5TB of filtered CommonCrawl text spanning 100 languages. The model is trained with a masked-language-modeling …


Exploring sentence embeddings for better argument relation classification

This thesis investigates methods to improve argument relation classification by augmenting sentence embeddings and leveraging Large Language Models (LLMs). Argument relation classification, a subtask of argument mining, involves predicting …


Exploring Sentiment Analysis for Spanglish: Why Sociolinguistic Context Still Matters for NLP

This thesis explores the role of sociolinguistic context in natural language processing (NLP), with a specific focus on sentiment analysis of Spanish-English code-switched language. Despite recent advancements in large language models (LLMs) …


Exploring sentiment patterns in social

O El Azzouzy, T Chanyour, SJ Andaloussi, K El - … and Optimization in 5G and Beyond, 2025

… Trained on large datasets such as Common Crawl or Reddit, these models offer remarkable computational efficiency and effectiveness, particularly when refined on smaller, domain-specific linguistic resources, suggesting further performance …


Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders

S Wang, T Loakman, Y Lei, Y Liu, B Yang, Y Zhao… - arXiv preprint arXiv …, 2025

Large Language Models (LLMs) are traditionally viewed as black-box algorithms, therefore reducing trustworthiness and obscuring potential approaches to increasing performance on downstream tasks. In this work, we apply an effective LLM …


Exploring The Effectiveness of In-Context Methods in Human-Aligned Large Language Models Across Languages

UA Prathama, A Purwarianti, S Cahyawijaya - JUTI: Jurnal Ilmiah Teknologi Informasi, 2025

… From these drop-off points we can infer a practical resource threshold for current LLMs which is language with roughly a Joshi’s Class value of 4 (which corresponds to a certain typological and corpus-size bracket) and at least 0.5 % coverage in the …


Exploring the Effectiveness of Multilingual and Generative Large Language Models for Question Answering in Financial Texts

A Al-Laith - Proceedings of the Joint Workshop of the 9th Financial …, 2025

This paper investigates the use of large language models (LLMs) for financial causality detection in the FinCausal 2025 shared task, focusing on generative and multilingual question answering (QA) tasks. Our study employed both generative …


Exploring the Impact of Attention Mechanisms in Big Data Analysis and Large Language Models

Z Mahal - American-Eurasian Journal of Scientific Research

… We utilized multiple large-scale datasets for this study, including textual corpora from open repositories (eg, Common Crawl and Wikipedia) and structured big data sources such as financial transactions and sensor logs. The goal was to test the …


Exploring the potential and limitations of large language models as virtual respondents for social science research

Z Rakovics, M Rakovics - Intersections. East European Journal of Society and …, 2024

Social and linguistic differences encoded in various textual content available on the internet represent certain features of modern societies. For any scientific research which is interested in social differences mediated by language, the advent of large …


Exploring the Potential of DeepSeek-R1 Model in Transforming Healthcare Solutions: An Overview

MR Raza, S Ahmed, FA Khokhar, A Varol - … on Digital Forensics and Security (ISDFS), 2025

Over the past few decades, artificial intelligence (AI) has become more integrated into healthcare, with Large Language Models (LLMs) being a key component in improving healthcare decision-making. These LLMs' capacity to produce and …


Exploring the Utility of Embedding Similarity for Contract Tasks

J Donnelly, A Roegiest - Proceedings of the 2025 International ACM SIGIR …, 2025

With the increasing use of text embeddings motivated by the adoption of Retrieval Augmented Generation (RAG) in applied domains, this work investigates whether the semantic aspects of text embeddings correspond to the colloquial understanding …


Exploring training data-free video generation from a single image via a stable diffusion model

X Han, H Sheng, C Bai - Journal of Visual Communication and Image …, 2025

Video generation is typically performed by incorporating frame information into a model and combining it with optical flow or warping operations. However, this approach requires extensive training on multiple frames, making it time intensive …


Exploring Transfer Learning in a Bidirectional Myanmar-Tedim Chin Machine Translation with the mT5 Transformer

CZ Man, SSM Win, KLL Khine - 2025 IEEE Conference on Computer Applications …, 2025

Machine translation (MT) plays a crucial role in bridging linguistic gaps, particularly for underrepresented languages. This paper investigates the feasibility and performance of the mT5-transformer model in machine translation task from …


EXPLORING WORD EMBEDDINGS FOR SENTIMENT ANALYSIS OF MARATHI POLITICAL TWEETS: A MACHINE LEARNING APPROACH

Sentiment analysis of textual data is becoming increasingly significant in research. Many researchers are developing new technologies to enhance the accuracy and performance of sentiment analysis. This process is particularly vital in analysing …


Exploring Zero-Shot Prompting for Generating Data Format Descriptions

P Anantharaman, V Varadharaju - 2025 IEEE Security and Privacy Workshops (SPW), 2025

… Finally,instead of relying on generating inputs from the specificationusing tools such as 3DTestGen, we constructed a test corpususing publicly available data via GovDocs, CommonCrawl,and packet captures.We explore how accurate parser …


EXPORTINGIDEOLOGIESVIAAI? EARLYASSESSMENTOF OPEN-SOURCE CHINESELARGE LANGUAGEMODELSIN JAPAN

This study examines how the influence of large language models (LLMs) developed in China is starting to spread beyond the country’s borders. Since 2022, the Chinese government has accelerated its promotion of generative artificial intelligence (AI) …


Exposing the Guardrails: Reverse-Engineering and Jailbreaking Safety Filters in DALL· E Text-to-Image Pipelines

… Our first metric to categorize languages is based on the Common Crawl dataset [11], which contains over 250 billion pages downloaded through extensive crawls of the Internet and is commonly used for training LLMs. The dominant language detected …


EXPRESS: From Bytes to Biases. Investigating the Cultural Self-Perception of Large Language Models

W Messner, T Greene, J Matalone - Journal of Public Policy & Marketing, 2025

Large language models (LLMs) are able to engage in natural-sounding conversations with humans, showcasing unprecedented capabilities for information retrieval and automated decision support. They have disrupted human-technology …


External links

M Kuraguchi, XP Wang, RT Bronson, R Rothenberg…

PAM Muhammad, S Ali, B Cairo, A Bakr, AS Hajji…

Nakajima D, Okazaki N, Yamakawa H, Kikuno R, Ohara O, Nagase T (2003)." Construction of expression-ready cDNA clones for KIAA genes: manual curation of 330 KIAA cDNA clones". DNA Res. 9 (3): 99–106. CiteSeerX 10.1. 1.500. 923. doi …


Extracting cross-modal semantic incongruity with attention for multimodal sarcasm detection

S Aggarwal, A Pandey, DK Vishwakarma - Applied Intelligence, 2025

… With over 2 terabytes of cleaned CommonCrawl data [47], the masked language model [46] has already been pre-trained on over a hundred languages, including Hindi. We chose this particular variant of [48] for our investigation since the majority …


Extracting Entity Mentions

CT Tsai, S Upadhyay, D Roth - Multilingual Entity Linking, 2025

The first step of the entity linking pipeline is to locate phrases in text which we would like to disambiguate. These phrases are usually called mentions of entities or concepts. Mention extraction could be very challenging depending on the …


Facilitating Judicial Cooperation in the EU. A Computable Approach to Mutual Recognition in Criminal Matters

G Lasagni, G Contissa, M Caianiello - 2025

The volume develps an innovative analysis of EU cooperation mechanisms in the criminal matter through the lenses of a computational approach to the law. This multi-level research tackles both EU and national legislation. The comparative analysis of the …


Facing GAIa: A Tale of Three ChatGPToxicities

V Galanos, I Xi - Deleuze and Guattari Studies, 2025

In light of generative artificial intelligence’s (GAI) recent successful applications and associated pair of hype and scepticism, we revitalise Félix Guattari’s three ecologies framework, suggesting its usefulness as a mapping and decision-making tool …


Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models

J Vladika, M Dhaini, F Matthes - arXiv preprint arXiv:2509.04304, 2025

The growing capabilities of Large Language Models (LLMs) show significant potential to enhance healthcare by assisting medical researchers and physicians. However, their reliance on static training data is a major risk when medical …


Factual Knowledge Assessment of Language Models Using Distractors

HA Khodja, F Bechet, Q Brabant, A Nasr, G Lecorvé - Proceedings of the 31st …, 2025

… For example, an LM trained on a QA dataset is more likely to generate the correct answer after a question compared to an LM trained on CommonCrawl, not because it “knows” the fact better, but because it was conditioned to answer questions with …


FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes

C Constantinides, D Patel, S Lin, C Guerrero, SD Patil… - arXiv preprint arXiv …, 2025

We introduce FailureSensorIQ, a novel Multi-Choice Question-Answering (MCQA) benchmarking system designed to assess the ability of Large Language Models (LLMs) to reason and understand complex, domain-specific scenarios in Industry 4.0. Unlike …


Fair Text Classification via Transferable Representations

T Leteno, M Perrot, C Laclau, A Gourru, C Gravier - arXiv preprint arXiv:2503.07691, 2025

Group fairness is a central research topic in text classification, where reaching fair treatment between sensitive groups (eg, women and men) remains an open challenge. We propose an approach that extends the use of the Wasserstein …


FairLangProc: A Python package for fairness in NLP

A Pérez-Peralta, S Benítez-Peña, RE Lillo - arXiv preprint arXiv:2508.03677, 2025

The rise in usage of Large Language Models to near ubiquitousness in recent years has risen societal concern about their applications in decision-making contexts, such as organizational justice or healthcare. This, in turn, poses questions about the …


Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance

J Zuo, M Velikanov, I Chahed, Y Belkada, DE Rhayem… - arXiv preprint arXiv …, 2025

… The multilingual data corpus draws from diverse sources—mainly Common Crawl and a range of curated datasets. For multilingual web data from Common Crawl, language identification was first performed at the HTML level using pycld2, then …


Fanar: An Arabic-Centric Multimodal Generative AI Platform

F Team, U Abbas, MS Ahmad, F Alam, E Altinisik… - arXiv preprint arXiv …, 2025

We present Fanar, a platform for Arabic-centric multimodal generative AI systems, that supports language, speech and image generation tasks. At the heart of Fanar are Fanar Star and Fanar Prime, two highly capable Arabic Large Language Models …


Fast and Lightweight Distributed Suffix Array Construction

M Haag, F Kurpicz, P Sanders, M Schimek - 33rd Annual European Symposium on …, 2025

… https://commoncrawl.org, 2019. Downloaded WET files from segments: https://data.commoncrawl.org/crawl-data/CC-MAIN-2019-09/segments/1550247479101.30/wet/CC-MAIN-… Only textual content was retained; HTML tags and Common Crawl metadata were removed. …


Fast Approximate Similarity Join in Vector Databases

J Xie, JX Yu, Y Liu - Proceedings of the ACM on Management of Data, 2025

Recent advancements in deep learning, particularly in embedding models, have enabled the effective representation of various data types such as text, images, and audio as vectors, thereby facilitating semantic analysis. A large number of massive …


Faster Wavelet Tree Queries

F Kurpicz, A Savino, R Venturini - Software: Practice and Experience, 2025

… CC is a concatenation of the WET files of the Common Crawl corpus, ie, a web crawl without HTML tags. Here, we removed all meta-information added by the corpus. Sources is a concatenation of plaintext files from some of the biggest open …


Feature Engineering Trends in Text-Based Affective Computing: Rules to Advance Deep Learning Models

G Pattun, P Kumar - International Research Journal of Multidisciplinary …, 2025

Understanding emotions in textual data, particularly within dynamic social media platforms such as YouTube, Facebook, and Twitter, presents significant challenges. This paper aims to provide a comprehensive review of emotion detection techniques …


Feature-based Media Bias Detection

T Spinde - Automated Detection of Media Bias, 2025

Thus far, we have presented a comprehensive literature review on media bias in Chap. 2, evaluated reliable measures for understanding media bias perception in Chap. 3, and introduced our two new datasets, MBIC and BABE, in Chap. 4. We now …


FED: Fast and Efficient Dataset Deduplication Framework with GPU Acceleration

Y Son, C Kim, J Lee - arXiv preprint arXiv:2501.01046, 2025

… The RealNews dataset is a large English corpus of news articles from Common Crawl, and C4 is a filtered version of Common Crawl. For C4, we sampled 100GB out of the total 700GB. We choose these datasets for two reasons. …


FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models

B Pang, Y Ouyang, H Xu, Z Jia, P Li, S Wen, L Wang… - arXiv preprint arXiv …, 2025

Advancements in reasoning for large language models (LLMs) have lead to significant performance improvements for LLMs in various fields such as mathematics and programming. However, research applying these advances to the …


Fighting Fire with Fire: Journalistic Investigations of Artificial Intelligence Using Artificial Intelligence Techniques

… – the Common Crawl. However, GPT-3's training process involved a narrower selection, with slightly more than 900,000 Dutch language pages (OpenAI Citation2020). This disparity stems from the stricter filtering criteria applied by the GPT-3 creators to …


Figurative Archive: an open dataset and web-based application for the study of metaphor

M Bressler, V Mangiaterra, P Canal, F Frau, F Luciani… - arXiv preprint arXiv …, 2025

… Semantic distance between the topic and vehicle was calculated using the Italian word embeddings from fastText58, a set of pre-trained word vectors based on Common Crawl and Wikipedia. The web interface provides access to these …


Filter Like You Test: Data-Driven Data Filtering for CLIP Pretraining

M Shechter, Y Carmon - arXiv preprint arXiv:2503.08805, 2025

We introduce Filter Like You Test (FLYT), a method for curating large-scale vision-language datasets that learns the usefulness of each data point as a pretraining example. FLYT trains a scoring model that learns to weigh each example using gradient …


FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models

X Xu, F Wen, B Chu, Z Fu, Q Lin, J Liu, B Fei, Z Yang… - arXiv preprint arXiv …, 2025

In natural language processing (NLP), the focus has shifted from encoder-only tiny language models like BERT to decoder-only large language models(LLMs) such as GPT-3. However, LLMs' practical application in the financial sector has revealed …


Fine-grained Fallacy Detection with Human Label Variation

A Ramponi, A Daffara, S Tonelli - arXiv preprint arXiv:2502.13853, 2025

We introduce Faina, the first dataset for fallacy detection that embraces multiple plausible answers and natural disagreement. Faina includes over 11K span-level annotations with overlaps across 20 fallacy types on social media posts in Italian …


Fine-grained sentiment analysis based on cross-modal information translation

S Zhang, P Du, X Cui, H Lin, L Yang - Multimedia Systems, 2025

Fine-grained sentiment analysis has been a hot topic in the field of natural language processing in recent years, and the diverse ways people express sentiments necessitate multi-modal sentiment analysis with text modality enhancement. In this …


Fine-Grained Sentiment Analysis on COVID-19 Tweets Using Deep Learning Techniques

P Appalanaidu, KDK Yadav, PM Manohar - Advances in Machine Learning and Big …, 2025

With the outbreak of COVID-19, we understand how social media played a crucial role by providing a platform as a means of emotional outlet for peer support and relief in the event of a health crisis. Nowadays there has been an exponential growth …


Fine-Tuned Large Language Models for Enhanced Automated Academic Advising

H Ismail - 2025 IEEE Global Engineering Education Conference …, 2025

The rapid increase in student enrollment at universities across the UAE has underscored the limitations of traditional academic advising methods, including long wait times and overburdened advisors. This paper presents a finetuned Academic …


Fine-Tuning Deep Learning Models for Sentiment Analysis: A Study on Movie Titles

H Qasim, M Zain, L Aziz, M Ayaz - Iqra Journal of Engineering and Computing, 2025

… Pre-trained GloVe embeddings obtain their main strength from their training against extensive Wikipedia and Common Crawl datasets that strengthen their linguistic understanding. Due to previous training GloVe embeddings acquire …


Fine-tuning Open-source Large Language Models for Processing Open-vocabulary Commands for Robotic Navigation

This thesis investigates using fine-tuned open-source Large Language Models (LLMs) for interpreting open-vocabulary commands for robotic navigation tasks. In this study, this means retrieving objects from scene graphs based on freeform language …


Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective

R Aralimatti, SAG Shakhadri, KR Kruthika, K Angadi - 2025

… While the general-purpose pre-training corpus includes sources such as Common Crawl and curated datasets, the Shakti-250M model incorporates domain-specific texts to enhance applicability in specialized fields such as healthcare, finance, and …


FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training

T Cheng, Y Wang, W He, Q Wang, Y Cheng, Y Zhang… - Second Conference on Language …

… 2024), we aim to use internet corpora (eg, Common Crawl, CC) as the foundation for our medical knowledge texts. CC inherently includes large-scale question-answer pairs and knowledge-rich textbooks (Shao et al.…


FineMedLM-o1: Enhancing the Medical Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training

H Yu, T Cheng, Y Cheng, R Feng - arXiv preprint arXiv:2501.09213, 2025

Recent advancements in large language models (LLMs) have shown promise in medical applications such as disease diagnosis and treatment planning. However, most existing medical LLMs struggle with the advanced reasoning required for …


FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering

E Henriksson, O Tarkka, F Ginter - arXiv preprint arXiv:2501.07314, 2025

… Since 2008, CommonCrawl has collected a corpus of approximately 10 petabytes of web content (Baack, 2024). Despite its size, CommonCrawl … C4 uses the WET files provided by CommonCrawl, which come with pre-extracted plaintext, whereas …


FineScope: Precision Pruning for Domain-Specialized Large Language Models Using SAE-Guided Self-Data Cultivation

C Bhattacharyya, Y Kim - arXiv preprint arXiv:2505.00624, 2025

Training large language models (LLMs) from scratch requires significant computational resources, driving interest in developing smaller, domain-specific LLMs that maintain both efficiency and strong task performance. Medium-sized …


Finetuning LLMs for Grammatical Error Correction in English and Greek Texts

D Kapelles, A Andriopoulos, D Koutsomitropoulos - 2025

… T5 was pre-trained on 750 GB of English-language text derived from the public web Common Crawl. mT5 was pre-trained on data from all 71 monthly web data published by Common Crawl so far, which is more than the source data used by T5. …


FineWeb-Conv: A Method for Finding Good Conversation Data

RJ Moore, S An, JP Gala, D Jadav - Workshop on Preparing Good Data for …, 2025

… Initially, it can be employed to identify high-quality conversation data within a collection of diverse documents, like Fineweb or Common Crawl. Here, quality refers to the presence of natural interaction patterns, not the information or knowledge …


FineWeb2: One Pipeline to Scale Them All--Adapting Pre-Training Data Processing to Every Language

G Penedo, H Kydlíček, V Sabolčec, B Messmer… - arXiv preprint arXiv …, 2025

… Finally, we use our pipeline to process almost 100 Common Crawl1 snapshots spanning the summer of 2013 to April 2024 to create … We extend our gratitude to the Common Crawl project for freely providing and maintaining their regular crawls …


First polarization study of the M87 jet and active galactic nuclei at submillimeter wavelengths with ALMA

C Goddi, DF Carlos - arXiv preprint arXiv:2505.10181, 2025

We present full-polarization observations at $\lambda = 0.87$ mm (345 GHz) conducted with the Atacama Large Millimeter/submillimeter Array (ALMA) toward Messier 87 (M87) and seven other radio-loud active galactic nuclei (AGN). We …


FlexOlmo: Open Language Models for Flexible Data Use

W Shi, A Bhagia, K Farhat, N Muennighoff, P Walsh… - arXiv preprint arXiv …, 2025

We introduce FlexOlmo, a new class of language models (LMs) that supports (1) distributed training without data sharing, where different model parameters are independently trained on closed datasets, and (2) data-flexible inference, where …


Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning

AT Ghahrizjani, A Taban, Q Wang, S Ye, A Mirzaei… - arXiv preprint arXiv …, 2025

Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs), notably enhancing their capacity to acquire domain-specific knowledge while preserving or potentially augmenting their general-purpose …


Form and function: automatic methods for prediction of functions

From the viewpoint of Systemic Functional Linguistics (SFL), language has evolved in society to provide means for negotiating with others about offering and requesting information or actions. These communicative needs are realised through the options …


Formalising lexical and syntactic diversity for data sampling in French

L Estève, M Scholivet, A Savary - arXiv preprint arXiv:2501.08003, 2025

Diversity is an important property of datasets and sampling data for diversity is useful in dataset creation. Finding the optimally diverse sample is expensive, we therefore present a heuristic significantly increasing diversity relative to random sampling. We …


Formalizing Complex Mathematical Statements with LLMs: A Study on Mathematical Definitions

L Zhang, M Valentino, A Freitas - arXiv preprint arXiv:2502.12065, 2025

… DeepSeekMath-7B is an open-sourced LLM trained specifically for mathematics using mathematical contents from Common Crawl. As a smaller model, it has demonstrated comparable mathematical reasoning performance as in GPT-4 (OpenAI et al.…


Formatting the Visible. In the Factory of Photo Datasets (2005‑2021)

T Sugitani - Transbordeur. Photographie histoire société, 2025

… In order to feed gargantuan datasets (400 million “URL-text” pairs for LAION-400M;45 5.85 billion for its successor LAION-5B), the work of collecting, processing, and annotating has been automated: LAION-400M needs to use Common Crawl, a …


Fostering Digital Inclusion for Low-Resource Nigerian Languages: A Case Study of Igbo and Nigerian Pidgin

E Nwafor, MP Nguyen - Proceedings of the Eighth Workshop on Technologies …, 2025

… This dataset includes 5,000 EnglishIgbo parallel sentences collected and pre-processed from sources like Wikipedia, CommonCrawl, and local media. Translation and quality checks were performed, including manual review and intertranslator …


Found in Translation: Sourcing parallel corpora for low-resource language pairs

This paper describes the sourcing, processing, and application of parallel text data for Icelandic and Polish for the purpose of bilingual lexicon induction (BLI), demonstrating how a parallel corpus can be compiled for a low-to-medium resource …


Foundation Model for Generative AI

A foundation model is a broad-scale trained model that can be adapted to various downstream activities, Scale, and capacity to carry out tasks beyond training. A large amount of unlabeled data is used in the training process of foundation models to get …


Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring

BS Korkmaz, R Nair, EM Daly, E Anagnostopoulos… - arXiv preprint arXiv …, 2025

Foundation models require fine-tuning to ensure their generative outputs align with intended results for specific tasks. Automating this fine-tuning process is challenging, as it typically needs human feedback that can be expensive to acquire. We present …


Foundation Models for Tabular Data within Systemic Contexts Need Grounding

T Klein, J Hoffart - arXiv preprint arXiv:2505.19825, 2025

… Large-scale efforts like WebTables [17], which contains 233 million tables from the Common Crawl project, and TabLib [36], with 627 million tables sourced from GitHub and Common Crawl, provide vast quantities of webscraped tables. However …


Foundations and Frontiers of Transfer Learning in NLP: A Comprehensive Review

MV Suryawanshi, A Kaiwade - Advances in Computational Sciences and Technology, 2025

… These models are initially trained using tasks like masked language modeling (MLM), causal language modeling (CLM), or denoising autoencoding on large unsupervised corpora (like Wikipedia and Common Crawl). They are refined on …


Foundations of Unknown-aware Machine Learning

X Du - arXiv preprint arXiv:2505.14933, 2025

Ensuring the reliability and safety of machine learning models in open-world deployment is a central challenge in AI safety. This thesis develops both algorithmic and theoretical foundations to address key reliability issues arising from …


Fragile by Design: Formalizing Watermarking Tradeoffs via Paraphrasing

A Falahati, L Golab - ICML Workshop on Technical AI Governance (TAIG)

… 2020), a large common-crawl based dataset, and generate corresponding watermarked outputs using the method’s default configuration. To simulate a paraphrasing attack, we apply a state-of-the-art publicly available paraphraser, Parrot (Damodaran…


FRAMES: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy

X Zhang, F Duan, L Xu, Y Zhou, S Wang, R Weng… - arXiv preprint arXiv …, 2025

Large language models (LLMs) have significantly advanced human language understanding and generation, with pretraining data quality and organization being crucial to their performance. Multi-stage pretraining is a promising approach, but …


Freak folk

… Reference.org uses data and images under license from Common Crawl, Getty Images, MusicBrainz, TMDB, Unsplash, Wikipedia …


From Acceleration to Saturation: Scaling Behavior of Bootstrapped Language Model Pretraining

SP Liew, T Kato - NeurIPS 2025 Workshop on Evaluating the Evolving …

… For the first-stage pretraining, we use the CommonCrawl portion of the Slimpajama-DC dataset [41], 73 containing 368B tokens in total. These models serve as base checkpoints for the second-stage 74 bootstrapped pretraining experiments. …


From Bias to Balance How Multilingual Dataset Composition Affects Tokenizer Performance Across Languages

A Selvamurugan, R Dandekar, R Dandekar, S Panat - NeurIPS 2025 Workshop on …

Tokenization serves as a crucial preprocessing step in multilingual language models, affecting performance in both high-resource and low-resource languages. However, current tokenizers seem to adopt language biases due to unbalanced training …


From ChatGPT to DeepSeek AI: A Comprehensive Analysis of Evolution, Deviation, and Future Implications in AI-Language Models

S Singh, S Bansal, AE Saddik, M Saini - arXiv preprint arXiv:2504.03219, 2025

… Training was carried out on the huge data, comprising of approximately 570GB of text after filtering, sourced from Common Crawl (60% of the training mix), WebText2 (19 billion, 22% training weight), Books1 (19 billion, 8% training weight), Books2 (55 …


From classification to taxonomy: Automated structuring of vehicle repair names in multilingual corpora

SV Mashtalir, OV Nikolenko - Вісник сучасних інформаційних технологій, 2025

This study introduces and rigorously validates a hybrid, five-stage Natural Language Processing pipeline that transforms unstructured, bilingual repair-order text into fully navigable, hierarchical action taxonomy – bridging the gap between flat keyword …


From Data to Grassroots Initiatives: Leveraging Transformer-Based Models for Detecting Green Practices in Social Media

A Glazkova, O Zakharova - Proceedings of the 1st Workshop on Ecology …, 2025

Green practices are everyday activities that support a sustainable relationship between people and the environment. Detecting these practices in social media helps track their prevalence and develop recommendations to promote eco-friendly …


From Embeddings to Explainability: A Tutorial on Large-Language-Model-Based Text Analysis for Behavioral Scientists

R Debelak, TK Koch, M Aßenmacher, C Stachl - Advances in Methods and Practices …, 2025

Large language models (LLMs) are transforming research in psychology and the behavioral sciences by enabling advanced text analysis at scale. Their applications range from the analysis of social media posts to infer psychological traits to the …


From Embeddings to Explainability: A Tutorial on LLM-Based Text Analysis for Behavioral Scientists

R Debelak, TK Koch, M Aßenmacher, C Stachl


From Geospatial Data to Narrative: A GIS-LLM Pipeline for Generating Personalised Outdoor Route Descriptions

I Ilyankou, J Haworth, T Cheng, S Cavazzi

With the growing availability of detailed geospatial data and the rise of generative AI, there is increasing potential to enhance how route-based information is communicated to end users. Current cartographic practices rarely incorporate …


FROM IA_ARCHIVER TO OPENAI: THE PASTS AND FUTURES OF AUTOMATED DATA SCRAPERS

K Mackinnon, E Maemura - AoIR Selected Papers of Internet Research, 2024

Data scraping practices have recently come under scrutiny, as datasets scraped from the web’s social spaces are the basis of new generative AI tools like Google’s Gemini, Microsoft’s Copilot, and OpenAI’s ChatGPT. These practices of scrapers and …


From inpainting to painting: exploring conservation of Chinese paintings with generative artificial intelligence

Chinese painting conservation faces several challenges, such as the inherent conflict between the conservation principles of minimal intervention, recognizability, and reversibility (Muñoz-Viñas, 2012), and the traditional pursuit of completeness in …


From keywords to key embeddings–contrasting French and Swedish web registers using multilingual deep learning

S Hellström, V Skantsi, A Salmela, V Laippala - Corpus Linguistics and Linguistic …, 2025

The pervasiveness of the internet has given web language use a central role in society. However, the lack of multilingual corpora and scalable methods has led to the focus on English in web language research. To address this gap, the present …


From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications

F Jiang, C Pan, L Dong, K Wang, OA Dobre, M Debbah - arXiv preprint arXiv …, 2025

With the advent of 6G communications, intelligent communication systems face multiple challenges, including constrained perception and response capabilities, limited scalability, and low adaptability in dynamic environments. This tutorial …


From Origins to Future: The Evolution and Prospects of Artificial Intelligence in the Reasoning Era

B Yang, J Qu - J. Int. Eco. Glo. Gov, 2025

With the release of the OpenAI o1 model, artificial intelligence (AI) technology has ushered in a new era of Reasoning. This article reviews the development history of AI technology, from the early days of symbolic reasoning and logic programming, to …


From Past to Present: A Survey of Malicious URL Detection Techniques, Datasets and Code Repositories

Y Tian, Y Yu, J Sun, Y Wang - arXiv preprint arXiv:2504.16449, 2025

Malicious URLs persistently threaten the cybersecurity ecosystem, by either deceiving users into divulging private data or distributing harmful payloads to infiltrate host systems. Gaining timely insights into the current state of this ongoing …


From PMI to Bots

K Church - International Journal of Lexicography, 2025

My paper with Patrick Hanks on PMI (pointwise mutual information) was the most successful paper I ever wrote, or ever will write. I believe the paper was successful because it appealed to a number of different audiences for a number of different …


From Pre-Trained Language Models to Agentic AI: Evolution and Architectures for Autonomous Intelligence

In this position paper, we present a comprehensive analysis of the evolution of artificial intelligence from pre-trained language models to agentic AI systems designed for autonomous intelligence. This evolution is structured across seven …


From Scarcity to Capability: Empowering Fake News Detection in Low-Resource Languages with LLMs

HM Shibu, S Datta, MS Miah, N Sami, MS Chowdhury…

The rapid spread of fake news presents a significant global challenge, particularly in lowresource languages like Bangla, which lack adequate datasets and detection tools. Although manual fact-checking is accurate, it is expensive and slow to prevent …


From Small to Large Language Models: Revisiting the Federalist Papers

SW Jeong, V Rockova - arXiv preprint arXiv:2503.01869, 2025

… et al., 2019; Brown et al., 2020), BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), BART (Lewis et al., 2019), and LLaMA (Touvron et al., 2023) rely heavily on massive training datasets sourced from diverse corpora, including BookCorpus …


From Translation to Generative LLMs: Classification of Code-Mixed Affective Tasks

A Yadav, T Garg, M Klemen, M Ulcar, B Agarwal… - IEEE Transactions on …, 2025

… It was trained on large Indian corpora consisting of English and 16 Indian languages where they utilize publicly available corpora from Wikipedia and Common Crawl, including Hindi and Tamil, which we consider in this work. We use …


From Tweets to Insights: Social Opinion Mining on Corporate Social Responsibility

C Leggerini, M Bannò - Corporate Social Responsibility and Environmental …, 2025

Corporate Social Responsibility (CSR) has become increasingly critical as firms seek to balance financial goals with social and environmental responsibilities. Our study introduces a three‐phase structured method to analyze stakeholders' opinions …


From Voice to Safety: Language AI Powered Pilot-ATC Communication Understanding for Airport Surface Movement Collision Risk Assessment

Y Pang, AP Kendall, A Porcayo, M Barsotti, A Jain… - arXiv preprint arXiv …, 2025

This work integrates language AI-based voice communication understanding with collision risk assessment. The proposed framework consists of two major parts, (a) Automatic Speech Recognition (ASR); (b) surface collision risk modeling. ASR …


Frustratingly Simple Retrieval Improves Challenging, Reasoning-Intensive Benchmarks

X Lyu, M Duan, R Shao, PW Koh, S Min - arXiv preprint arXiv:2507.01297, 2025

… To ensure wide coverage, we start with Common Crawl, which is widely used for pre-training and also constitutes 70% of MASSIVEDS [8]. However, we hypothesize that much of it is low-… Overall, this process reduces the size of Common Crawl …


FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow

H Sun, HW Wang, J Gu, L Li, Y Cheng - arXiv preprint arXiv:2505.17399, 2025

Front-end engineering involves a complex workflow where engineers conceptualize designs, translate them into code, and iteratively refine the implementation. While recent benchmarks primarily focus on converting visual designs to code, we present …


Functions and organization

The mandate of OHCHR derives from Articles 1, 13 and 55 of the Charter of the United Nations, the Vienna Declaration and Programme of Action and General Assembly resolution 48/141 of 20 December 1993, by which the Assembly …


Fusing Geoscience Large Language Models and Lightweight RAG for Enhanced Geological Question Answering

… pre-training corpus for the geosciences, comprising three main tiers: (1) Academic Literature: integrating approximately 288,000 Open Access scientific papers from 12 major publishers; (2) Web Text: including geoscience-related …


Future-Proof Yourself: An AI Era Survival Guide

T Kim - arXiv preprint arXiv:2504.04378, 2025

Future-Proof Yourself is a practical guide that helps readers navigate the fast-changing world of artificial intelligence in everyday life. The book begins by explaining how computers learn from data in simple, relatable terms, and gradually introduces the …


FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

S Zhu, T Dong, B Li, D Xiong - arXiv preprint arXiv:2505.14256, 2025

In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy to train FuxiMT. We first pre-train the model on a massive …


Gaining the Edge: Visualizing Information Advantage through Machine Learning-Driven Dashboards

A El Ouadi, W Knowlton, A Pimentel, D Beskow - 2025

… , two straightforward ways to access news data are through the Common Crawl News (CC-News) feed or the Global Database of … Common Crawl News and GDELT data for diverse academic, commercial, and government use cases. This …


GENDER BIAS DETECTION IN GREEK LANGUAGE MODELS

Gender bias in language models has emerged as a critical ethical and technical challenge in Natural Language Processing (NLP). This thesis investigates the presence and extent of gender bias in Greek language models, focusing on two …


Gender bias in language and artificial intelligence tools

This master’s thesis represents an interdisciplinary approach to understanding gender bias manifested in the output of artificial intelligence tools, which are based on language models. Biases and stereotypes become problematic when we …


Gender Bias in Translation Automation: Addressing Bias and Inequality

MG González - The Social Impact of Automating Translation, 2024

Machine translation (MT) has become an essential tool for overcoming language barriers and facilitating cross-cultural communication. However, it has also raised significant concerns, particularly regarding gender bias—the tendency of MT …


General purpose models for the chemical sciences

N Alampara, A Aneesh, M Ríos-García, A Mirza… - arXiv preprint arXiv …, 2025

… One can utilize a “top-down” approach where a large and diverse pool of data—eg, results from web-crawled resources such as CommonCrawl… filtered CommonCrawl for mathematical text using a combination of regular expressions …


Generalizable Cross-Lingual Cognitive Distortion Detection with Standardized Annotations and Multi-Task Learning

H Qi, N Bai, J Li, W Zhai, Q Zhao, Q Gao, BX Yang… - Findings of the Association …, 2025

… ), pre-trained on 2.5 TB of CommonCrawl data1 from 100 languages. Its key features include extended training steps, dynamic masking, and unigram SentencePiece tokenization, enabling consistent crosslanguage processing. 1https://commoncrawl.org/ …


Generate-Distill: Training Cross-Language IR Models with Synthetically-Generated Data

D Lawrie, E Kayi, E Yang, J Mayfield, DW Oard, S Miller - Proceedings of the 48th …, 2025

Most pretrained language models that support neural information retrieval are fine-tuned on the MS MARCO dataset. MS MARCO is expressed in English, so it naturally supports monolingual English retrieval. However, for Cross-Language Information …


Generating language assessment content free from representational harms

Today’s language models can produce syntactically accurate and semantically coherent texts. This capability presents new opportunities for generating content for language assessments, which have traditionally required intensive expert resources …


Generating targeted and tailored health communication narratives with AI

Customized narratives are effective tools to promote risk prevention behaviors in populations. However, the development of such narratives is resource‐intensive. Advances in generative artificial intelligence (AI) offer promising solutions to these …


Generative AI and Democratic Culture

J Branford, E Soulier, L Fichtner - Philosophy & Technology, 2025

The purported threats that the algorithmic creation, ordering, and manipulation of information in the digital sphere may pose to democracy have received considerable academic attention in recent years. In seeking to extend this discussion beyond the …


Generative AI Chatbots in Higher Education: ATAM-Based Analysis of Discipline-Specific Adoption Patterns for Students at the University of Borås

… For example, LLMs are trained on text from sources like Wikipedia and the Common Crawl (a collection of web pages). They predict the probability of words appearing in a particular context, generating text by selecting words based on these …


Generative AI Decision-Making Attributes in Complex Health Services: A Rapid Review

… -3, the third generation of the Generative Pre-trained Transformer, can retrieve information from a large corpus of books, articles, websites, and many other sources of text data; its primary source being the repository of documents and web pages …


Generative AI for Industry Transformation: A Systematic Review of ChatGPT's Capabilities and Integration Challenges

S Salih, O Husain, EAM Abdalla, AO Ibrahim… - International Journal of …, 2025

The rapid advancement of Generative Artificial Intelligence (GAI), particularly OpenAI's ChatGPT, has significantly transformed various industries by enhancing efficiency, reducing operational costs, and fostering innovation. This systematic …


Generative AI in Academic Writing: A Comparison of DeepSeek, Qwen, ChatGPT, Gemini, Llama, Mistral, and Gemma

Ö Aydın, E Karaarslan, FS Erenay, NB Džakula

… The team developed a FastText-based classifier to filter mathematical content at scale, starting with a robust seed dataset comprising OpenWebMath as positive examples and Common Crawl as negatives. This approach enabled the extraction …