Generative AI in Focus: A Comprehensive Review of Leading Models Across Modalities

S Aishwarya, C Selvamurugan, KG Parthiban… - 2024 4th International …, 2024

GenAI has revolutionized the generation of realistic and imaginative data in ways that were previously beyond the capabilities of other machine learning algorithms. This area is rapidly gaining traction, with extensive research currently underway to …


Generative AI Unleashed: A Multi-Domain Journey of Successful Implementations of Large Language Models

N Kumar, A Barthwal, S Mishra, A Jain - … : Large Language Models and Their Real …, 2025

The entire work describes various fields addressed by generative artificial intelligence and provides a cross-disciplinary approach not limited to a particular discipline. As this chapter showcases examples of using generative AI in contexts …


Generative AI's Copyright Enigma: A Comparative Study of Fair Use and Fair Dealing

… Rather, LAION-5B, assisted by the CLIP (Contrastive Language-Image Pre-training) exploit images collected by Common Crawl to create a … Common Crawl itself does not engage in any reproduction or copyright infringement because they do not …


Generative AI: Techniques, Models and Applications

The rapid advancement of artificial intelligence (AI) has ushered in a new era of technological innovation, with generative AI standing at the forefront of this transformation. This book, Generative AI—Techniques, Models and Applications …


Generative Artificial Intelligence Data Risks and Governance Pathways

B Jiang, X Ye - Beijing Law Review, 2025

Generative AI, exemplified by ChatGPT, offers societal benefits while posing challenges to data governance. Addressing data risks is vital for its healthy development. This paper examines the technical framework and pre-training data …


Generative Artificial Intelligence in Academic Surgery: Ethical Implications and Transformative Potential

JR Robinson, A Stey, DF Schneider, AN Kothari… - Journal of Surgical …, 2025

Artificial intelligence (AI) is rapidly being used in medicine due to its advanced capabilities in image and video recognition, clinical decision support, surgical education, and administrative task automation. Large language models such as …


Geosocial media's perspective on energy: a text classification approach using natural language processing

J Verdoodt, K Milleville, H Huang, C Vandeviver… - Journal of Location Based …, 2025

This study examines public opinion on various energy sources through Twitter data, focusing on fossil fuels, nuclear energy, and renewable energy sources like solar and wind. Utilizing natural language processing techniques, specifically BERTweet …


Geospatiality: the effect of topics on the presence of geolocation in English text data

J Mast, R Lemoine-Rodríguez, V Rittlinger… - International Journal of …, 2025

Geolocated text data are a promising data source for spatial analyses in many fields, from disease surveillance to the spatial humanities. This study investigates the relationship between texts’ thematic categories and their likelihood of containing …


GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay

Y Zhang, S Jiang, M Zhao, Y Li, Y Fan, X Wu, Q Chen - arXiv preprint arXiv …, 2025

The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence. However, continual fine-tuning LLMs across various domains often suffers from catastrophic forgetting, characterized by: 1) …


Getting Publii CMS Running

B Moore - Designing Websites with Publii and GitHub Pages, 2025

… The options include toggles for "Noindex website," "Block GPTBot bot," "Block ChatGPT-User bot," and "Block Common Crawl bots." Each option has a description explaining its function, such as preventing … The same is true for the fourth option …


Getting the balance right–Copyright and AI

… One survey of 1,158 news publishers found that approximately 55% of them have instructed OpenAI, Google AI or the non-profit Common Crawl to stop scanning their sites.On the other hand, a 2024 study by Cloudflare found that out of the top …


GigaChat Family: Efficient Russian Language Modeling Through Mixture of Experts Architecture

M Valentin, E Kosarev, G Leleytner, I Shchuckin… - arXiv preprint arXiv …, 2025

Generative large language models (LLMs) have become crucial for modern NLP research and applications across various languages. However, the development of foundational models specifically tailored to the Russian language has been limited …


GLiDRE: Generalist Lightweight model for Document-level Relation Extraction

R Armingaud, R Besançon - arXiv preprint arXiv:2508.00757, 2025

… 2024), a high-quality dataset built from filtered and deduplicated English Common Crawl archives. We prompt the Mistral-Small-24B-Instruct-2501 model to generate annotations for both entities and the relations between them in a structured …


GLM-4.1 V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

W Hong, W Yu, X Gu, G Wang, G Gan, H Tang… - arXiv preprint arXiv …, 2025

… We begin by extracting URLs from a recent CommonCrawl snapshot and capturing corresponding webpage screenshots via automated tools. Going beyond static captures, we employ the Playwright framework to deeply interact with …


GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

A Zeng, X Lv, Q Zheng, Z Hou, B Chen, C Xie, C Wang… - arXiv preprint arXiv …, 2025

We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters, featuring a hybrid reasoning method that supports both thinking and direct response modes. Through …


GLOVE: GLOBAL VECTORS FOR WORD REPRESENTATION

AM Rakhmatillayevich - AMERICAN JOURNAL OF MULTIDISCIPLINARY …, 2025

… Efficient Training: It can be trained efficiently even on large corpora (eg, Wikipedia, Common Crawl with billions of words). Cooccurrence … Therefore, pre-trained GloVe vectors (eg, trained on Wikipedia, Common Crawl, Twitter) are often used …


GneissWeb: Preparing High Quality Data for LLMs at Scale

HE Gohari, SR Kadhe, SYS Adam, A Adebayo… - arXiv preprint arXiv …, 2025

… These datasets are mainly derived by processing text from the Common Crawl [12] and optionally mixing some high-quality data sources (eg, GitHub). However, majority of these datasets are less than 5T tokens which limits their suitability for pre-training …


Going over Fine Web with a Fine-Tooth Comb: Technical Report of Indexing Fine Web for Problematic Content Search and Retrieval

I Altemir Marinas, A Kucherenko, A Kucharavy - arXiv e-prints, 2025

IA Marinas, A Kucherenko, A Kucharavy - arXiv preprint arXiv:2508.21788, 2025

Large language models (LLMs) rely heavily on web-scale datasets like Common Crawl, which provides over 80\% of training data for some modern models. However, the indiscriminate nature of web crawling raises challenges in data quality, safety …


Governance of discriminatory content in conversational AIs: a cross-platform and cross-cultural analysis

N Ta, J Zeng, Z Li - Information, Communication & Society, 2025

As the widespread adoption of conversational artificial intelligence (AI) systems has raised concerns about social bias, especially towards vulnerable groups, this study explores how these systems respond to and regulate discriminatory content …


Governmental Internet censorship and its circumvention: Case of the Great Firewall of China

Internet censorship can be defined as the practice of removing, manipulating, or blocking access to information on the Internet. Internet censorship overall ranges from companies regulating what is allowed within their company’s digital premises …


GPT-3-Based AI Cover Letter Generator: A Feasibility Study & Implementation

Natural Language Processing (NLP) is the emerging field research studies of the interaction between human and computing systems. With advancement of NLP techniques, machines are becoming increasingly proficient in understanding …


GPU Implementation of the Wavelet Tree

M Franzreb, M Burtscher, S Rudolph - arXiv preprint arXiv:2505.03372, 2025

I present a new GPU implementation of the wavelet tree data structure. It includes binary rank and select support structures that provide at least 10 times higher throughput of binary rank and select queries than the best publicly available CPU …


Gradient Weight-normalized Low-rank Projection for Efficient LLM Training

E Kanoulas, JIAH HUANG, Y Shen, H Zhu, S Rudinac - Greeks in AI Symposium 2025

JH Huang, Y Shen, H Zhu, S Rudinac, E Kanoulas - arXiv preprint arXiv:2412.19616, 2024

… 2020b), a cleaned version of Common Crawl’s web corpus, with perplexity as the performance metric. In our fine-tuning experiments, we use BERTbase (Devlin et al. 2018), RoBERTabase, RoBERTalarge (Liu et al. 2019a), and BARTbase (Lewis et al …


Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval

T Zheng, Y Zhang, X An, Z Feng, K Yang, Q Ding - arXiv preprint arXiv:2509.09118, 2025

… 2022), a large-scale dataset that contains 747M image-text pairs collected from CommonCrawl, as our web-crawled images source. To filter high-quality person-centric images, we initially deploy the YOLOv11 model (Jocher and Qiu, 2024) to detect …


Grammar or Crammer? The Role of Morphology in Distinguishing Orthographically Similar but Semantically Unrelated Words

We show that n-gram-based distributional models fail to distinguish unrelated words due to the noise in semantic spaces. This issue remains hidden in conventional benchmarks but becomes more pronounced when orthographic similarity is high. To …


Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence

GV Team, L Karlinsky, A Arbelle, A Daniels, A Nassar… - arXiv preprint arXiv …, 2025

… A large portion of the DocFM dataset was obtained from the Common Crawl corpus which is an open repository of web data. It is freely … by Common Crawl from the actual URL content. By adhering to the Robots Exclusion Protocol (Koster et …


GRAPE: Optimize Data Mixture for Group Robust Multi-target Adaptive Pretraining

S Fan, MI Glarou, M Jaggi - arXiv preprint arXiv:2505.20380, 2025

The performance of large language models (LLMs) across diverse downstream applications is fundamentally governed by the quality and composition of their pretraining corpora. Existing domain reweighting algorithms primarily optimize data …


Grid based hybrid search for spatio-textual data

In this thesis, we present a new approach to the approximate similarity search problem over spatio-textual data, where queries involve both geographic locations and semantically rich text. Unlike traditional approaches that rely on exact keyword …


Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model

C Li, Y Deng, J Zhang, C Zong - arXiv preprint arXiv:2506.12388, 2025

The curse of multilinguality phenomenon is a fundamental problem of multilingual Large Language Models (LLMs), where the competition between massive languages results in inferior performance. It mainly comes from limited capacity and …


Guardrails for safe implementations of AI-based services

DC Verma, R Ratnaparkhi - Assurance and Security for AI-enabled Systems 2025, 2025

… In some cases, an enterprise may also want to use publicly available data on the Internet, eg common crawl data and its derivatives.The … [14] Foundation, CC, “Common crawl data,” (2023). Accessed: 2025-02-18. [15] Gutiérrez-Fandino, A., Pérez-Fernández …


GUICourse: From General Vision Language Model to Versatile GUI Agent

W Chen, J Cui, J Hu, Y Qin, J Fang, Y Zhao, C Wang… - Proceedings of the 63rd …, 2025

… We collected 4M URLs from the Cleaned Common Crawl Corpus (Raffel et al.… However, some screenshots in our GUIEnv dataset are collected from the Cleaned Common Crawl Corpus, so we cannot guarantee that these website screenshots are …


Guided by Style: Fine-Grained Modulation in Multi-Style Artistic Transfer

We propose a novel diffusion-based framework for artistic multi-style transfer that uniquely combines compositional denoising and classifier-free guidance (CFG) to enable fine-grained control over both content preservation and stylistic blending …


HAGEN//ANALYTICS

AI refers to a computer’s ability to emulate human intelligence and thought. When people refer to AI, they are often referring to the concept of Generative AI (GenAI), which refers to a computer’s ability to create new content out of synthesized data …


Hajj-FQA: A benchmark Arabic dataset for developing question-answering systems on Hajj fatwas: H. Aleid and A. Azmi

HA Aleid, AM Azmi - Journal of King Saud University Computer and …, 2025

Deep learning has significantly advanced the question-answering (QA) systems across various sectors. However, Arabic-language systems for Hajj-related fatwas (non-binding Islamic legal opinions issued by muftis) remain underdeveloped. This paper …


Hardwired-Neurons Language Processing Units as General-Purpose Cognitive Substrates

Y Liu, Y Chen, Y Zhao, Y Hao, Z Zheng, W Kong, Z Li… - arXiv preprint arXiv …, 2025

… For instance, Common Crawl [12] has amassed an 8 PB text corpus from web pages, growing consistently by approximately 250 TB per month. This unparalleled scale of pre-training data is instrumental in enabling the zero-shot generalization …


Harnessing Large Language Models and Deep Neural Networks for Fake News Detection

E Papageorgiou, I Varlamis, C Chronis - Information, 2025

… It was trained on the Colossal Clean Crawled Corpus (C4) dataset, a 750 GB dataset created from Common Crawl’s web-extracted text. The model architecture is similar to the original … It was trained on the RealNews dataset, created from …


Harvard Data Science Review â¢ Special Issue 5: Grappling With the Generative AI Revolution

The rise of powerful large language models (LLMs) brings about tremendous opportunities for innovation but also looming risks for individuals and society at large. We have reached a pivotal moment for ensuring that LLMs and LLM-infused …


Has My Code Been Stolen for Model Training? A Naturalness Based Approach to Code Contamination Detection

HA Khan, Y Jiang, Q Umer, Y Zhang, W Akram, H Liu - Proceedings of the ACM on …, 2025

It is often valuable to know whether a given piece of source code has or hasn’t been used to train a given deep learning model. On one side, it helps avoid data contamination problems that may exaggerate the performance of evaluated models …


HASTIKA: hate speech and target identification in Kannada-English code-mixed text

S Kavatagi, R Rachh - Language Resources and Evaluation, 2025

In the modern era, the widespread use of social media has facilitated connections among millions of people worldwide. However, these platforms have also been exploited for spreading hate speech, particularly in multilingual contexts. The …


Hate Speech Detection in Code-Mixed Datasets Using Pretrained Embeddings and Transformers

T Sohail, A Aiman, E Hashmi, AS Imran, SM Daudpota… - … International Conference on …, 2024

… The model utilizes FastText’s unsupervised learning method, trained on data from Common Crawl and Wikipedia, to embed words into 300-dimensional vectors. By integrating character n-grams, it enhances its grasp of word morphology and …


Health Sentinel: An AI Pipeline For Real-time Disease Outbreak Detection

D Pant, RR Grandhe, V Samaria, M Paul, S Kumar… - arXiv preprint arXiv …, 2025

… We identified regional news websites that are often overlooked by platforms like Common Crawl or Google News. To address this, we developed a custom crawler that manually collects articles from these sources, improving regional representation. 3. …


Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training

Y Zhuang, J Yang, H Jiang, X Liu, K Cheng… - arXiv preprint arXiv …, 2025

Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability. We introduce …


Hermes: Algorithm-System Co-design for Efficient Retrieval-Augmented Generation At Scale

M Shen, M Umar, K Maeng, GE Suh, U Gupta - 2025

… less than 10B tokens we use a subset of Common Crawl [36]. We generate a synthetic set of … indices from the 10B token subset of Common Crawl, ranging from 5GB to 11GB each and … of indices built using the 10B token Common Crawl …


Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations

H Li, F Fu, H Ge, S Lin, X Wang, J Niu, X Miao, B Cui - arXiv preprint arXiv …, 2025

… 100 steps on the CommonCrawl and GitHub datasets using 32 H20 GPUs, with a batch size of 200K tokens per step, for different context lengths (32K and 16K). Figure 16 further depicts the sequence length distribution per step for the 32K …


Hey ChatGPT—Is a Louis Vuitton Bag an Investment? Evaluating LLM Readiness for Use in Financial Literacy and Education

S Taylor, S Taylor, S Lin, V Keselj - Journal of Emerging Technologies in Accounting, 2025

The prevalence of large language models (LLMs) such as ChatGPT has wowed the world with its ability to generate text in a human-like manner. While educators evaluate how AI will impact the future of learning, we identify mistakes ChatGPT has …


High-Accuracy Transition-Based Constituency Parsing

J Bauer, CD Manning - Proceedings of the 18th International Conference on …, 2025

Constituency parsers have improved markedly in recent years, with the F1 accuracy on the venerable Penn Treebank reaching 96.47, half of the error rate of the first transformer model in 2017. However, while dependency parsing frequently uses …


High-Fidelity Simultaneous Speech-To-Speech Translation

T Labiausse, L Mazaré, E Grave, P Pérez, A Défossez… - arXiv preprint arXiv …, 2025

We introduce Hibiki, a decoder-only model for simultaneous speech translation. Hibiki leverages a multistream language model to synchronously process source and target speech, and jointly produces text and audio tokens to perform speech-to-text …


History of Large Language Models

F De Luzi - Engineering Information Systems with Large Language …, 2025

This chapter explores the historical development of artificial intelligence (AI) and natural language processing (NLP), focusing on the evolution of language modeling. We begin by outlining the foundations of AI, from symbolic approaches to the …


Homophonic Pun Generation in Code Mixed Hindi English

YR Sarrof - Proceedings of the 1st Workshop on Computational …, 2025

In this study, we investigate Hinglish—a blend of Hindi and English commonly found in informal online communication—with a particular focus on automated pun generation. Our work examines the applicability and adaptability of existing English …


Horizon-scale variability of from 2017--2021 EHT observations

K Akiyama, E Albentosa-Ruíz, A Alberdi, W Alef… - Astronomy & Astrophysics

We report three epochs of polarized images of at 230,GHz using data from the Event Horizon Telescope (EHT) taken in 2017, 2018, and 2021. The baseline coverage of the 2021 observations is significantly improved through the addition of two new EHT …


Hot ChipsKeynote

N Shazeer - 2025 IEEE Hot Chips 37 Symposium (HCS), 2025

… Trillions of words of text available(Common Crawl, etc)●Generative pretraining, then finetune on many tasks(Radford et al, OpenAI)Your computer is too slow.●Training cost is quadratic in #parameters:(more operations/token) * (more training to fill brain)●109params …


HOW CONVERSATIONAL SYSTEMS ARE BUILT USING LANGUAGE MODELS

DB Hydyrova, D Jumayeva - ОБРАЗОВАНИЕ И НАУКА В XXI ВЕКЕ, 2025

Conversational systems, also known as chatbots or virtual assistants, have evolved significantly with the advancement of large language models (LLMs). These systems rely on natural language processing (NLP), deep learning, and reinforcement …


How Long Do Financial Markets Need to Fully Respond to FOMC Announcements?


How Much Do LLMs Hallucinate across Languages? On Multilingual Estimation of LLM Hallucination in the Wild

A Lauscher, G Glavaš - arXiv preprint arXiv:2502.12769, 2025

… Across all 30 languages, however, we find no correlation between the hallucination rates and measures of language “resourceness”: (i) proportion of language-specific data in Common Crawl and (ii) number of articles in the language-specific …


How to Compare Things Properly? A Study of Argument Relevance in Comparative Question Answering

I Nikishina, S Anwar, N Dolgov, M Manina, D Ignatenko…

… using the Comparative Argumentative Machine (CAM 2.0), which retrieves relevant content from CommonCrawl (Schildwächter et al.… 2019), which involves retrieving relevant sentences from the CommonCrawl corpus, sentence classification …


How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapters

R Oji, J Kunz - arXiv preprint arXiv:2501.06025, 2025

This paper investigates the optimal use of the multilingual encoder model mDeBERTa for tasks in three Germanic languages -- German, Swedish, and Icelandic -- representing varying levels of presence and likely data quality in …


How well can LLMs Grade Essays in Arabic?

R Ghazawi, E Simpson - arXiv preprint arXiv:2501.16516, 2025

This research assesses the effectiveness of state-of-the-art large language models (LLMs), including ChatGPT, Llama, Aya, Jais, and ACEGPT, in the task of Arabic automated essay scoring (AES) using the AR-AES dataset. It explores various evaluation …


HPLT's Second Data Release

N Arefyev, M Aulamo, M Bañón, L Burchell, P Chen… - Proceedings of Machine …, 2025

We describe the progress of the High Performance Language Technologies (HPLT) project, a 3-year EU-funded project that started in September 2022 with two main objectives: derive monotexts and bitexts for multiple languages from web crawls at …


HQ-CLIP: Leveraging Large Vision-Language Models to Create High-Quality Image-Text Datasets and CLIP Models

Z Wei, G Wang, X Ma, K Mei, H Chen, Y Jin, F Rao - arXiv preprint arXiv:2507.22431, 2025

… CommonPool aggregates web-crawled image-text pairs from Common Crawl dumps spanning 2014-2022. We offer three standardized benchmark scales: small (12.8M pairs), medium (128M pairs), and large (1.28B pairs). To ensure direct comparability …


HTMLDownloader: An open-source tool for dynamic web scraping and archiving using WebView2

BV Truong, LTT Nguyen, P Pham, B Vo - SoftwareX, 2025

The increasing complexity and dynamism of modern websites present major challenges for traditional web scraping tools such as Scrapy, BeautifulSoup and wget, which often fail to capture dynamic content or offer accessible user interfaces …


Hugging Face Diffusers-Chapter 01

1Introduction to Hugging Face Diffusers LibraryThe Hugging Face Diffusers library has become a cornerstone in the field of natural language processing (NLP) , offering innovative tools for training, fine-tuning, and deploying transformer-based …


Human Perspectives and Social Infrastructures: Prioritising People in GLAM Digitisation

M Gooch, R Kahn, E Kugeler - Journal of Open Humanities Data, 2025

Much discussion in current digital humanities research and funding is concerned with creating, using and maintaining technical and research infrastructures. These large-scale projects are often ambitious–designed to bring tools, data and …


Human vs. Machine: The Future of Translation in an AI-Driven World

A Falempin, D Ranadireksa - … Conference on Engineering 2024 (WICOENG 2024), 2024

The era of digitalization and advanced technology has revolutionized information exchange, leading to unprecedented efficiency and transformative shifts. Advanced language models are capable of automating routine translation tasks and facilitating …


Human-Centered AI in Computational Social Science: Evaluating Automated Annotation with Large Language Models

Computational social scientists are increasingly incorporating text as data into their research. A typical framework for working with large text data sets involves hiring human annotators to read a subset of the text samples and then building a statistical …


Human-like conceptual representations emerge from language prediction

N Xu, Q Zhang, C Du, Q Luo, X Qiu, X Huang, M Zhang - arXiv preprint arXiv …, 2025

Recent advances in large language models (LLMs) provide a new opportunity to address the long-standing question of how concepts are represented and organized in the mind, which is central to unravelling the nature of human cognition. Here, we …


Humanitarian classification of crisis-related microblogs in Bengali: A comparison of multilingual pre-trained language models

K Das, D Datta, M Basu, S Ghosh - International Journal of Disaster Risk Reduction, 2025

During a crisis or disaster event, humanitarian organizations need various types of situational information that is essential for planning relief efforts. Social media platforms like X (erstwhile Twitter) have proven to be effective platforms for …


Hybrid AI for Large-Scale Foundation Models

… o Common Crawl: A massive corpus of web data that serves as a source for training transformer-based models like GPT-4, which requires large-scale text data for pretraining on diverse content. o Wikipedia: Used to fine-tune LLMs for domain-specific …


Hybrid natural language processing tool for semantic annotation of medical texts in Spanish

L Campillos-Llanos, A Valverde-Mateos… - BMC Bioinformatics, 2025

… CLIN-X-ES is derived from the XML RoBERTA multilingual model (originally pre-trained on 2.5 terabytes of the CommonCrawl corpus for 100 languages), by continuous pre-training on a corpus of medical texts from SciELO, MedlinePlus, EMEA or PubMed. This …


HYLR-FO: Hybrid Approach Using Language Models and Rule-Based Systems for On-Device Food Ordering

S Yang, D Kim, S Lee - Electronics, 2025

… Its multilingual counterpart, mT5, is trained on a dataset derived from Common Crawl, covering 101 languages [30]. MASS (masked sequence-to-sequence pre-training) employs an encoder–decoder framework to reconstruct missing segments within a …


IDENTIFYING ARGUMENTATIVE CLAIMS IN BIOMEDICAL RESEARCH ARTICLES

… Common Crawl is a vast dataset of web crawls that contains a variety of web pages, offering more diversity in text and gathering various online content. The combined data results in a large and diverse collection of text containing hundreds …


Identifying Data Contamination in LLMs for Mathematical Benchmarks

… For web pages, large-scale datasets such as CommonCrawl -which include snapshots of most web content on the Internet, although with significant amounts of noisy and low-quality information - are widely used. Filtered subsets of such datasets …


Identifying School Shooter Threats Through Online Texts

OJ Liahagen, MJ Nilsen, B Gambäck - … in Natural Language Processing and Social …, 2025

… GloVe embeddings were extracted using Wikipedia 2014 + Gigaword 53 and the Common Crawl 840B3 sets as frozen embedding layers. Two different vector dimensionalities, 50 and 300, were utilized to study their effects on prediction …


Idiosyncrasies in Large Language Models

M Sun, Y Yin, Z Xu, JZ Kolter, Z Liu - arXiv preprint arXiv:2502.12150, 2025

In this work, we unveil and study idiosyncrasies in Large Language Models (LLMs) -- unique patterns in their outputs that can be used to distinguish the models. To do so, we consider a simple classification task: given a particular text output, the objective …


IFEvalCode: Controlled Code Generation

J Yang, W Zhang, S Liu, L Chai, Y Tan, J Liu, G Zhang… - arXiv preprint arXiv …, 2025

… Forward Constraints Generation Given the recalled code-related documents from Common Crawl, we adopt Qwen2.5-Coder-32B to create new questions by drawing inspiration from the coderelated documents for a general question. To effectively …


Impact of Deep Learning for Multilingual Natural Language Processing in Educational Applications

P Tamilarasan, V Selvaraj, LR Buckingham - … of International Conference on Recent Trends …

Over the past few years, the field of educational technology has experienced notable progress by incorporating advanced deep learning methods, namely, in the area of multilingual Natural Language Processing (NLP). This work examines the utilization …


Impact of Pretraining Word Co-occurrence on Compositional Generalization in Multimodal Models

H Qu, SM Xie - arXiv preprint arXiv:2507.08000, 2025

… LAION-400M [21] is a dataset of 400 million image-text pairs curated from Common Crawl by filtering out pairs with CLIP embedding cosine similarity below 0.3. LAION-400M was created to emulate the closed-source WIT-400M [1] dataset used to train the …


Implicit Evaluation of Health Answers from Large Language Models

… Scripts are provided to download the documents from the CommonCrawl archive, as well as the social media content directly over the … The web documents are easily accessible via the CommonCrawl archive. Before accessing all relevant …


Implicit knowledge-augmented prompting for commonsense explanation generation

Y Ge, HT Yu, C Lei, X Liu, A Jatowt, K Kim, S Lynden… - Knowledge and Information …, 2025

… OPT’s pre-training primarily involves English text, though a small amount of non-English data from CommonCrawl is present in the training corpus. It is pre-trained with a causal language modeling objective and is a decoder-only model, similar to GPT-3. …


Improving Acoustic Recognition Models/Author Paul Primus

Sound is one of the fundamental signals through which we perceive our surroundings and consequently, humans have evolved to perform complex auditory tasks effortlessly. The field of intelligent audio processing aims to replicate these …


Improving complex reasoning in large language models

This thesis studies complex reasoning in language models. We use the term reasoning to refer to tasks that would require a human to perform slow deliberate, step-by-step thinking (instead of providing an intuitive and instantaneous response) …


Improving critical infrastructure security through hybrid embeddings for vulnerability classification

AB Yahya, H El Akhal, AEB El Alaoui - Journal of Information Security and …, 2025

The growing prevalence of vulnerabilities in embedded devices poses a significant risk to critical infrastructure. While deep learning has advanced vulnerability classification, its effectiveness is often hindered by limitations in word representation …


Improving Experimental Methods to Capture Real-World Human-AI Perceptions and Interactions

AI agents are being increasingly used in production settings, but our understanding of how humans expect AI to behave, and how AI usage influences human behavior, falls short because of the gap between controlled laboratory studies and real-world …


Improving Fluency Of Neural Machine Translation Using Large Language Models

J He, W Pan, SP Jijia Yang, X Jia - Proceedings of Machine Translation Summit XX …, 2025

… Common Crawl for training in De–En. The training data have totally 4.6 million sentences. We use Newstest2014 for validation, and Newstest2021 for testing in De–En. For Ru–En, ParaCrawl v9, News-commentary-v10, and Common Crawl … v10, and …


Improving Informally Romanized Language Identification

A Benton, A Gutkin, C Kirov, B Roark - arXiv preprint arXiv:2504.21540, 2025

… The first is MADLAD-400, a filtered subset of Common Crawl that covers a wider range of languages than multilingual C4. The release … GlotCC: An open broad-coverage CommonCrawl corpus and pipeline for minority languages. In Proceedings of the …


Improving LLMs for Machine Translation Using Synthetic Preference Data

D Vajda, D Vreš, M Robnik-Šikonja - arXiv preprint arXiv:2508.14951, 2025

… By prompting both GaMS-9B-Instruct [4] and EuroLLM-9B-Instruct [15] to translate English Wikipedia articles [30] and a collection of English news articles from Common Crawl (CC-News dataset), we generate dual translations for each article …


Improving LLMs' Generalized Reasoning Abilities by Graph Problems

Q Zhang, N Chen, Z Li, M Peng, J Tang, J Li - arXiv preprint arXiv:2507.17168, 2025

Large Language Models (LLMs) have made remarkable strides in reasoning tasks, yet their performance often falters on novel and complex problems. Domain-specific continued pretraining (CPT) methods, such as those tailored for mathematical …


Improving Machine Translation Formality with Large Language Models

M Yang, F Li - Computers, Materials and Continua, 2025

Preserving formal style in neural machine translation (NMT) is essential, yet often overlooked as an optimization objective of the training processes. This oversight can lead to translations that, though accurate, lack formality. In this paper, we propose …


Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentations

L Ranaldi, F Ranaldi, FM Zanzotto, B Haddow, A Birch - arXiv preprint arXiv …, 2025

Retrieval-augmented generation (RAG) is key to enhancing large language models (LLMs) to systematically access richer factual knowledge. Yet, using RAG brings intrinsic challenges, as LLMs must deal with potentially conflicting knowledge, especially in …


Improving Text Recognition Accuracy for Serbian Legal Documents Using BERT

M Bogdanović, M Frtunić Gligorijević, J Kocić… - Applied Sciences, 2025

… The primary dataset selected for the training phase was the OSCAR [33] dataset, which represents a comprehensive collection of open data, generated through linguistic classification from the Common Crawl corpus [34]. Additionally, we utilized …


Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics

A Fernando, S Ranathunga, N de Silva - arXiv preprint arXiv:2502.19074, 2025

… 2020) extracts bitext from Common Crawl 6 using document-level and sentence-level alignment based on multilingual embeddings. Though it improves alignment quality over global bitext-mined corpora, it still contains significant noise, requiring careful …


In Generative AI We Trust: Measuring the Potential for Deception in LLM-Generated Health Information Using Computational Content Analysis

Misleading health information remains a central concern in medical sociology and public health due to its harmful effects on individuals and society. As health information-seeking increasingly shifts to digital platforms, Large Language Models (LLMs)—now …


In-Context Learning as Conditioned Associative Memory Retrieval

W Wu, TY Hsiao, JYC Hu, W Zhang, H Liu - Forty-second International Conference on …

We provide an exactly solvable example for interpreting In-Context Learning (ICL) with one-layer attention models as conditional retrieval of dense associative memory models. Our main contribution is to interpret ICL as memory reshaping in the modern …


Incident Cause Classification in Insurance claims using Generative AI

Automation plays a critical role in modern insurance operations, enabling companies like OP-Pohjola to process claims more rapidly, reduce manual workloads, and improve customer satisfaction. However, claim automation rates are …


Incorporating Symmetry and Constraints Into Machine Learning for Molecular and Solid-State Systems

This thesis focuses on the development and application of ML models incorporating physics constraints and symmetry for predicting complex physical quantities of both solid-state condensed matter systems and molecules with the aim to accelerate the …


Incremental Tensor Induction through Unbounded Pseudo-Contextualization in Pretrained Language Models

O Strickland, H Whitlam, R Cattermole, S Chilvers…

… Model Selection and Modification The base model selected for architectural modification was a 7B-parameter causal decoder-only transformer pretrained on a mixture of Common Crawl, Wikipedia, and high-quality instructional corpora …


Indian integration

… Reference.org uses data and images under license from Common Crawl, Getty Images, MusicBrainz, TMDB, Unsplash, Wikipedia …


Indian Legal Judgment Summarization using LEGAL-BERT and BiLSTM model with Adaptive Length

V Naik, K Rajeswari - EPJ Web of Conferences, 2025

… While pretrained general corpora (eg, Wikipedia, BooksCorpus, and Common Crawl) trained language models have proven effective across generalized tasks, they often fall short when considering domain-specific tasks that require in-domain …


Indo-Aryan Languages: A Transformer-Based Survey

S Roy, JR Saini - Intelligent System and Data Analysis: SSIC 2023 …

… It has been trained on large (2.5 TB) Common Crawl Data [30]. It has performed well for all multiple cross-lingual benchmarks. This model consists of 12 … Dirt cheap web-scale parallel text from the common crawl. In: Proceedings of the 51st …


Indonesian Abstractive Text Summarization Using Stacked Embeddings and Transformer Decoder

Document summarization can be categorized into two categories: extractive and abstractive summarization. Research in abstractive summarization is more limited than that of extractive summarization, especially for Indonesian documents. Most …


Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index

H Xu, J Liu, Y Choi, NA Smith, H Hajishirzi - arXiv preprint arXiv:2506.12229, 2025

… In the future, we will keep indexing the latest crawl in Common Crawl and update contamination results to track benchmark contamination as corpora evolve. The system also allows anyone to add or upload new benchmarks to be monitored …


INFORMATION EXTRACTION FROM SCIENTIFIC LITERATURE

The exponential growth of scientific literature, with millions of new articles published annually, has created an unsustainable discovery bottleneck across research communities. Manual extraction of critical information—including methodologies …


Informative task classification with concatenated embeddings using deep learning on crisisMMD

T Jain, D Gopalani, Y Kumar Meena - International Journal of Computers and …, 2025

Disastrous situations pose a formidable challenge, testing our resilience against nature's fury and the race against time to prevent the loss of human life. It is noted that in such situations that Microblogging platforms like Twitter(now X) have proven …


Informed Digital Systems: Knowledge Procurement, Gate/Keeping, and Experience

… Generative models like ChatGPT are similarly reliant upon a system of crawlers (known as GPTBot), the Common Crawl (a massive nonprofit organization that maintains an open repository of information collected across the internet), and other content such …


Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking

Y Chen, J Shang, Z Zhang, Y Xie, J Sheng, T Liu… - arXiv preprint arXiv …, 2025

Large language models (LLMs) face inherent performance bottlenecks under parameter constraints, particularly in processing critical tokens that demand complex reasoning. Empirical analysis reveals challenging tokens induce abrupt gradient …


Inside Out 2: Make Room for New Emotions & LLM: A Reproducibility Study of the Emotional Side of Search in the Classroom

H Chakrabarti, DM Tobia, M Landoni, MS Pera - … of the 48th International ACM SIGIR …, 2025

In an existing study, the InsideOut Framework is used to produce and explore the emotional profiles of search engines (SE) in response to queries formulated by children aged 9 to 11 in the classroom context, revealing the emotional diversity of …


Insightlens RAG

A BATRA, AKK PRINCE, S BHARDWAJ, P KUMAR…

Insightlens RAG is an advanced, domain-specific document retrieval system built with a chatbot-like interface. The system processes PDF documents by converting their content into binary strings and storing these as vectors within a vector database …


Insights into Low-Resource Language Modelling: Improving Model Performances for South African Languages.

R Visser, T Grobler, M Dunaiski - Journal of Universal Computer Science (JUCS), 2024

To address the gap in natural language processing for Southern African languages, our paper presents an in-depth analysis of language model development under resource-constrained conditions. We investigate the interplay between model size …


Insights into Moral Reasoning of AI: A Comparative Study Between Humans and Large Language Models

S Bajpai, A Sameer, R Fatima - Journal of Media Ethics, 2025

This study investigates the moral reasoning capabilities of large language models (LLMs), focusing on biases and the extent to which outputs reflect training data patterns rather than genuine reasoning. Using the Moral Competence Test (MCT) and the …


InspAIred: Cross-cultural Inspiration Detection and Analysis in Real and LLM-generated Social Media Data

O Ignat, GG Lakshmy, R Mihalcea - Proceedings of the 3rd Workshop on Cross …, 2025

Inspiration is linked to various positive outcomes, such as increased creativity, productivity, and happiness. Although inspiration has great potential, there has been limited effort toward identifying content that is inspiring, as opposed to just engaging …


Institutional Books 1.0: A 242B token dataset from Harvard Library's collections, refined for accuracy and usability

M Cargnelutti, C Brobston, J Hess, J Cushman, K Mukk… - arXiv preprint arXiv …, 2025

Large language models (LLMs) use data to learn about the world in order to produce meaningful correlations and predictions. As such, the nature, scale, quality, and diversity of the datasets used to train these models, or to support their work at …


INSTRUCTING LANGUAGE MODELS TO BE INTELLIGENT AI ASSISTANTS

… sources have been utilized for such purposes: Conversation logs between human users and online LM services (eg, OpenAI API) [20, 214]; Online QA forums like StackExchange, WikiHow, and Reddit [218]; Directly extracting QA pairs from web …


Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque

O Sainz, N Perez, J Etxaniz, JF de Landa, I Aldabe… - arXiv preprint arXiv …, 2025

Instructing language models with user intent requires large instruction datasets, which are only available for a limited set of languages. In this paper, we explore alternatives to conventional instruction adaptation pipelines in low-resource …


Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction

Y Jiang, Y Wang, C Wu, X Dai, Y Xu, W Gan, Y Wang… - arXiv preprint arXiv …, 2025

The improvement of LLMs' instruction-following capabilities depends critically on the availability of high-quality instruction-response pairs. While existing automatic data synthetic methods alleviate the burden of manual curation, they often rely heavily on …


INTEGRATING LARGE LANGUAGE MODELS AND VIRTUAL REALITY FOR INTERACTIVE CIRCUIT ANALYSIS

This Master thesis explores the integration of Artificial Intelligence (AI) into Virtual Reality (VR) as a tool for interactive learning in electronics education. The work was carried out in collaboration with ByBrick and Knightec Group, focusing on creating …


Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions

D Mahmud, H Hajmohamed, S Almentheri, S Alqaydi… - arXiv preprint arXiv …, 2025

Intelligent Transportation Systems (ITS) are crucial for the development and operation of smart cities, addressing key challenges in efficiency, productivity, and environmental sustainability. This paper comprehensively reviews the transformative …


Integrating product data from the web using deep learning techniques

… org Dataset Series, a publicly available dataset derived from the Common Crawl, facilitating the analysis of schema. org adoption on the Web and providing distant supervision for machine learning tasks such as product classification and entity …


Integrating RAG for Smarter Animal Certification Platforms

PB Montero, J Bulegon Gassen, G Descovi… - Information, 2025

… Knowledge Gaps and Hallucinations: LLMs are trained on vast but general corpora like Common Crawl, which disproportionately represent conversational language over niche, technical literature. As a result, the model’s understanding of …


Intelligent Inside Threat Detection Framework Based on Digital Twin, Transformer Variant Models, and Transfer Learning

With the rise of networked systems and modern hacker techniques, insider threats have become a greater concern than external hackers, as they often cause more significant damage and are harder to detect due to authorized access, complex …


Intent Factored Generation: Unleashing the Diversity in Your Language Model

E Ahmed, U Berdica, M Elliott, D Horak, JN Foerster - arXiv preprint arXiv:2506.09659, 2025

Obtaining multiple meaningfully diverse, high quality samples from Large Language Models for a fixed prompt remains an open challenge. Current methods for increasing diversity often only operate at the token-level, paraphrasing the same …


Interacting Large Language Model Agents. Bayesian Social Learning Based Interpretable Models.

A Jain, V Krishnamurthy - IEEE Access, 2025

This paper develops theory and algorithms for interacting large language model agents (LLMAs) using methods from statistical signal processing and microeconomics. While both fields are mature, their application to decision-making …


Intergenerational justice as a framework for social media archiving

R Shiozaki - Journal of Documentation, 2025

Purpose This conceptual study aims to explore the rationale of preservation institutions in archiving new types of documents, such as social media, rather than focusing on traditionally valued materials or established cultural heritage. Design/methodology/approach …


Intern-S1: A Scientific Multimodal Foundation Model

L Bai, Z Cai, M Cao, W Cao, C Chen, H Chen, K Chen… - arXiv preprint arXiv …, 2025

In recent years, a plethora of open-source foundation models have emerged, achieving remarkable progress in some widely attended fields, with performance being quite close to that of closed-source models. However, in high-value but more …


Interventional Radiology Checklist for Artificial Intelligence Research Evaluation

JT Anibal, HB Huth, T Boeken, D Daye, J Gichoya… - Journal of Vascular and …, 2025

As artificial intelligence (AI) becomes increasingly prevalent within interventional radiology (IR) research studies, steps must be taken to ensure the robustness of novel technological systems presented in peer-reviewed literature. This report …


Interventional Radiology Reporting Standards and Checklist for Artificial Intelligence Research Evaluation (iCARE)

JT Anibal, HB Huth, T Boeken, D Daye, J Gichoya… - CardioVascular and …, 2025

As artificial intelligence (AI) becomes increasingly prevalent within interventional radiology (IR) research and clinical practice, steps must be taken to ensure the robustness of novel technological systems presented in peer-reviewed journals …


Interview Study to Determine the Potential of Large Language Model Usage in IT-Consultancy

M Jehnert, C Meyer, K Sandkuhl - International Conference on Business Information …, 2025

… These datasets include sources such as Common Crawl, WebText2, internet-based books, and the English-language Wikipedia [3]. The model learns by predicting subsequent tokens based on contextual input, enabling it to produce coherent and …


Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders

K Ghate, I Slaughter, K Wilson, M Diab, A Caliskan - arXiv preprint arXiv:2502.07957, 2025

… It starts with a raw data pool from Common Crawl and balances the data distribution based on metadata derived from CLIP’s original curation concepts, ensuring a diverse yet informative subset of training pairs. The use of metadata to …


Intrinsic evaluation of Mono-and Multilingual Dutch Language Models

D Vlantis, J Bloem - Computational Linguistics in the Netherlands Journal, 2025

Through transfer learning, multilingual language models can produce good results on extrinsic, downstream NLP tasks in low-resource languages despite a lack of abundant training data. In most cases, however, monolingual models still perform …


Introducing a Bangla sentence gloss pair dataset for Bangla sign language translation and research

NA Roudra, N Saha, R Shahriyar, S Sakib - 2025

Bangla Sign Language translation and recognition has been an evolving research topic throughout the years. However, existing research on this field is limited to word and alphabet level detection. For a more continuous sentence level detection of …


Introduction and Fundamentals

P Passban, M Rezagholizadeh, A Way - … LLM Performance: Efficacy, Fine-Tuning, and …, 2025

In this chapter, we explain the intricacies of language modelling, focusing on the evolution from statistical models to the sophisticated large language models (LLMs) that dominate the field today. We explore the transition from n-gram models to neural …


Investigating Ageism, Ableism, and Nationality Bias in Norwegian and Multilingual Language Models

We investigate biases related to ageism, ableism, and nationality in four Norwegian and two multilingual language models. These types of bias are underexplored in the current literature, and existing work on Norwegian models has primarily focused on …


Investigating Code Review Quality in ML Libraries: Patterns of Missed Bugs and Bug Detection with LLMs

Over the past several years, ML techniques have become commonplace in numerous technological areas where many real-world environments depend on such techniques. More recently, tasks that relied on traditional ML approaches are …


Investigating Machine-Learning VBA-Macro Analysis Using Machine Learning in a Constrained Environment

… From reading papers and reaching out to other researchers, it was discovered that Common Crawl could potentially be used for obtaining the set of benign documents. Common Crawl is a site that continuously crawls and archives the …


Investigating Students' Academic Writing Proficiency Using ChatGPT. The Case Study of First Year Master's Students.

Chat Generative Pre-trained Transformer is an AI-based natural language generative model, able to produce humanized texts with a personalized style/type of writing requested. This study investigates the effect of ChatGPT on students’ …


Investigating the cross-lingual generalizability of readability assessment using a multilingual BERT model fine-tuned in a single language

… As previously stated the model used in this study is implemented using the xlm-roBERTa-large model, a multilingual BERT variant pre-trained on the CommonCrawl dataset, comprising 2.5 TB of data across 100 languages. It is implemented using the …


Investigating the Feasibility and Risks of Leveraging Artificial Intelligence and Open Source Intelligence to Manage Predictive Cyber Threat Models

OA Obioha-Val, TI Lawal, OO Olaniyi, MO Gbadebo…

This study investigates the integration of Artificial Intelligence (AI) and Open Source Intelligence (OSINT) to enhance predictive threat modeling in cybersecurity, addressing the growing complexity and frequency of cyber threats. Integrating AI …


Investigating the Impact of Language-Adaptive Fine-Tuning on Sentiment Analysis in Hausa Language Using AfriBERTa

SA Sani, SH Muhammad, D Jarvis - arXiv preprint arXiv:2501.11023, 2025

Sentiment analysis (SA) plays a vital role in Natural Language Processing (NLP) by ~identifying sentiments expressed in text. Although significant advances have been made in SA for widely spoken languages, low-resource languages such as Hausa …


Investigating the Validity Evidence of Automated Scoring Methods for Divergent Thinking Assessments

Divergent thinking (DT) ability is a fundamental aspect of creativity, but its assessment remains challenging by the reliance on effortful human ratings and persistent uncertainty regarding how to aggregate scores across a variable number …


IRBlock: A Large-Scale Measurement Study of the Great Firewall of Iran

… These domain test lists are collected from various sources, including top-level domains (TLD) zone files [6], the Citizen Lab test lists (CLTL) [13], the Tranco list [60], and the Common Crawl project [2]. We use a new domain list generated every day …


Is 3D Technology a Curse or a Blessing for the Market of Contemporary Sculpture?

V Wiesinger - Sites of Reproduction. Fotografie und Skulptur …, 2025

The Saint-Maur Gallic Warrior| Fig. 3|, in brass and silver, had been unearthed in twenty-two pieces in the north of France in 1983 before it was purchased by the Musée départemental de l’Oise, Beauvais, in 1985 (inv. 85.16↗). It was …


Is ChatGpt Better Than Epileptologists at Interpreting Seizure Semiology?

Objective: This study aims to evaluate the clinical value of representative large language models (LLMs), namely ChatGPT, on interpreting seizure semiology to localize epileptogenic zones (EZs) for presurgical assessment in patients with focal …


Is Neural Machine Translation Viable for Low-Resource Languages? An Experimental Study of the Irish Language

Transformer-based Neural Machine Translation (NMT) models are Large Language Models (LLMs) designed and developed for translating between two or more given languages. These are typically most successful in the context of high-resource …


Is Single-View Mesh Reconstruction Ready for Robotics?

F Nolte, B Schölkopf, I Posner - arXiv preprint arXiv:2505.17966, 2025

This paper evaluates single-view mesh reconstruction models for creating digital twin environments in robot manipulation. Recent advances in computer vision for 3D reconstruction from single viewpoints present a potential breakthrough for efficiently …


Is There a Case for Conversation Optimized Tokenizers in Large Language Models?

R Ferrando, J Conde, G Martínez, P Reviriego - arXiv preprint arXiv:2506.18674, 2025

The computational and energy costs of Large Language Models (LLMs) have increased exponentially driven by the growing model sizes and the massive adoption of LLMs by hundreds of millions of users. The unit cost of an LLM is the …


It's not for free, it's just accessible-The Role of Consent, Purpose Limitation, and the Right to Erasure as GDPR Safeguards in AI Training

With Artificial Intelligence (AI) relying heavily on vast datasets, including publicly available personal online data and increasing amounts of fines for data breaches the individuals demand for control over personal data increases. The purpose of this …


Iterative Multilingual Spectral Attribute Erasure

S Shao, Y Ziser, Z Zhao, Y Qiu, SB Cohen, A Korhonen - arXiv preprint arXiv …, 2025

Multilingual representations embed words with similar meanings to share a common semantic space across languages, creating opportunities to transfer debiasing effects between languages. However, existing methods for debiasing are unable to …


IV An Updated Superscript: Paradoxes of Writing Amidst Generative AI

RH Gibson - Ecologies of Writing: Natural, Technical, and Social …, 2025


Jabuticaba: The largest commercial corpus for LLMs in Portuguese

M Amadeus, WAC Castaneda, JRH da Silva, R Scotti

… ’s 3 billion bpet and 45 TB of compressed plaintext from Common Crawl before filtering and 570 GB after filtering, equivalent to 400 billion bpet. … Models, includes over 100B text documents coming from 84 CommonCrawl snapshots and processed …


Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search

Y Gu, Q Hu, S Yang, H Xi, J Chen, S Han, H Cai - arXiv preprint arXiv:2508.15884, 2025

We present Jet-Nemotron, a new family of hybrid-architecture language models, which matches or exceeds the accuracy of leading full-attention models while significantly improving generation throughput. Jet-Nemotron is developed using …


JiuZhou: open foundation language models and effective pre-training framework for geoscience

Z Chen, M Lin, M Zang, Z Wang, J Li, Y Bai - International Journal of Digital Earth, 2025

Geoscience research has generated vast amounts of data, creating a need for effective extraction and integration of knowledge to address global-change challenges, promote sustainable development, and accelerate scientific discovery …


Joint Multi-modal Modeling

J Ou, H Xu, H Zan - Machine Translation: 20th China Conference, CCMT …

… The text translation data are normally news or common crawl, while the speech translation data are usually talks and recitations. There is a significant domain gap, and selectively using the text translation data which are more close to speech …


JOURNAL OF SCIENTIFIC RESEARCH AND THEIR SOLUTIONS

… AI tomonidan ishlab chiqarilgan kontentning sifati va haqiqiyligi bilan bog‘liq xavotirlar paydo bo‘ldi, tadqiqotlar shuni ko‘rsatdiki, Common Crawldan 6 milliarddan ortiq jumlalar namunasidagi jumlalarning 57% dan ortig‘i mashina …


JT-Math: A Multi-Stage Framework for Advanced Mathematical Reasoning in Large Language Models

Y Hao, F Chao, Y Hao, Z Cui, H Bai, H Zhang, Y Liu… - arXiv preprint arXiv …, 2025

… These models undergo continual pre-training with a 120B-token corpus of high-quality mathematical web data sourced from Common Crawl. The series includes an instruction-tuned variant, DeepSeek-Math-Instruct, trained on problems with Chain-of-Thought …


Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models

M Ali, M Brack, M Lübbering, E Wendt, AG Khan… - arXiv preprint arXiv …, 2025

… The vast majority of training data for large language models is sourced from the web, with Common Crawl (CC) being the most important corpus. Traditionally, many works have relied heavily, and in some cases exclusively, on heuristic-based filtering …


Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation

I Salazar, MF Burda, SB Islam, AS Moakhar, S Singh… - arXiv preprint arXiv …, 2025

The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While multilingual benchmarks have expanded, both in size and languages, many …


Kanana: Compute-efficient Bilingual Language Models

Y Bak, H Lee, M Ryu, J Ham, S Jung, DW Nam, T Eo… - arXiv preprint arXiv …, 2025

We introduce Kanana, a series of bilingual language models that demonstrate exceeding performance in Korean and competitive performance in English. The computational cost of Kanana is significantly lower than that of state-of-the-art …


KatotohananQA: Evaluating Truthfulness of Large Language Models in Filipino

LA Nery, RD Catignas, TJ Tiam-Lee - arXiv preprint arXiv:2509.06065, 2025

… Only 0.83% of the widely-used pre-training dataset Common Crawl is in Filipino while 45.26% is in English [7]. Aside from this, over two-thirds of instruction data for fine-tuning LLMs is in English [8]. This highlights the need for further research into …


Key Point Analysis in Greek: A New Dataset and Baselines

Identifying key statements in large volumes of opinionated texts that appear daily in social media, and online debates is an essential tool for informed decision making. During the 8th Workshop on Arguments Mining at EMNLP 2021, in an attempt to …


Killer Robots Beyond the Loop: Autonomy, UAS, and Meaningful Human Control

… Today for example, a non-profit database Common Crawl, provides access to a multi-petabyte-sized web-crawled database made up of 250 billion pages with 3-5 billion new pages added each month; it is cited in over 10,000 research pages.For-profit …


Kimi k1. 5: Scaling Reinforcement Learning with LLMs

K Team, A Du, B Gao, B Xing, C Jiang, C Chen, C Li… - arXiv preprint arXiv …, 2025

Language model pretraining with next token prediction has proved effective for scaling compute but is limited to the amount of available training data. Scaling reinforcement learning (RL) unlocks a new axis for the continued improvement of …


Kimi-VL Technical Report

K Team, A Du, B Yin, B Xing, B Qu, B Wang, C Chen… - arXiv preprint arXiv …, 2025

We present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities - all while activating only 2.8B parameters in its …


Knowledge Extraction on Semi-Structured Content: Does It Remain Relevant for Question Answering in the Era of LLMs?

K Sun, Y Huang, S Mehra, M Kachuee, X Chen, R Tao… - arXiv preprint arXiv …, 2025

The advent of Large Language Models (LLMs) has significantly advanced web-based Question Answering (QA) systems over semi-structured content, raising questions about the continued utility of knowledge extraction for question answering. This …


Knowledge Graph Completion using RAG and Improved Structural Information

B Li, Z Mao, R Yan, A Ling, Q Hu, Q Zeng - 2025 IEEE 2nd International Conference …, 2025

… To convert triple data into natural language text, the large model RAG uses the Wikipedia Dump, Common Crawl datasets, and the Phi-3-medium-4k-instruct model [17]. These datasets, sourced from Wikipedia and web data, contain natural language …


knowledge on biodiversity beyond national jurisdiction

M Zhang, Y Chen - Advances in Marine Environmental Protection …, 2025

Areas beyond national jurisdiction (ABNJ) face persistent degradation of marine biodiversity (Humphries and Harden-Davies, 2020). A United Nations agreement on the conservation and sustainable use of marine biodiversity in areas beyond …


KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting

T Wijesiriwardene, R Wickramarachchi, SR Vennam… - Proceedings of the 31st …, 2025

Making analogies is fundamental to cognition. Proportional analogies, which consist of four terms, are often used to assess linguistic and cognitive abilities. For instance, completing analogies like “Oxygen is to Gas as< blank> is to< blank>" requires …


KOMPYUTER LINGVISTIKASINING ASOSIY YO'NALISHLARI

GI Madaminjanovna - Modern education and development, 2025

… Shuningdek, umumiy matn tahlili uchun Common Crawl korpusi va nutq ma’lumotlari uchun LibriSpeech kabi ochiq ma’lumotlar to‘plamlari ham tahlil qilindi. Yig‘ilgan ma’lumotlar sifatli mazmuniy tahlil va taqqoslovchi baholash texnikalari yordamida tahlil qilindi …


Kr\'eyoLID From Language Identification Towards Language Mining

R Dent, PO Suarez, T Clérice, B Sagot - arXiv preprint arXiv:2503.06547, 2025

… more of distracting documents in a 2.6 billion page Common Crawl snapshot in a few hours on a … of first pass filtering on the December 2024 Common Crawl snapshot for each target label. … After that, we test document-level filtering on a full …


Krutrim LLM: Multilingual Foundational Model for over a Billion People

A Kallappa, P Kamble, A Ravi, A Patidar, V Dhruv… - arXiv preprint arXiv …, 2025

… Indic languages comprise only 1 percent of Common Crawl corpora despite India representing 18 percent of the global population, leading to linguistic biases. Thousands of regional languages, dialects, and code mixing create additional …


Lance: Efficient Random Access in Columnar Storage through Adaptive Structural Encodings

W Pace, C She, L Xu, W Jones, A Lockett, J Wang… - arXiv preprint arXiv …, 2025

The growing interest in artificial intelligence has created workloads that require both sequential and random access. At the same time, NVMe-backed storage solutions have emerged, providing caching capability for large columnar datasets in cloud …


Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation

D Gurgurov, K Trinley, YA Ghussin, T Baeumel… - arXiv preprint arXiv …, 2025

Large language models (LLMs) exhibit strong multilingual abilities, yet the neural mechanisms behind language-specific processing remain unclear. We analyze language-specific neurons in Llama-3.1-8B, Mistral-Nemo-12B, and Aya-Expanse-8B …


Language Grounding in Vision

… In this thesis, we make use of the 300-dimensional GloVe Embeddings trained on 840 billion tokens sourced from Commoncrawl covering … We make use of the 300-dimensional fastText Embeddings trained on Commoncrawl covering 2M unique words with …


Language Is Leaving Me: An AI Exploration of Epigenetic or Inherited Trauma of Cultures of Diaspora

… However, to briefly summarize the topic, English language image banks are built using Common Crawl, a web scraper. They skew towards popular sites like Pinterest, Wikimedia, Tumblr, various shopping sites, stock images, and images of celebrities …


Language Modeling Over Logical Forms

This dissertation introduces the research program of language modeling over logical forms: the employment of language models (LMs) that take as input semantic representations. The use of such models is motivated by the Accelerated Learning …


Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive Ziji

X Yang - Proceedings of the 31st International Conference on …, 2025

This paper explores whether language models can effectively resolve the complex binding patterns of the Mandarin Chinese reflexive ziji, which are constrained by both syntactic and semantic factors. We construct a dataset of 320 synthetic …


Language Models Improve When Pretraining Data Matches Target Tasks

D Mizrahi, ABL Larsen, J Allardice, S Petryk… - arXiv preprint arXiv …, 2025

… This data pool also processes CommonCrawl but with slightly different preprocessing choices and global fuzzy deduplication. Perhaps most importantly, NemotronCC includes 1.9T synthetic tokens created through model-based rephrasing4 [Maini et al.…


Language Models Lack Temporal Generalization and Bigger is Not Better

S Verkijk, P Vossen, P Sommerauer - Findings of the Association for Computational …, 2025

This paper presents elaborate testing of various LLMs on their generalization capacities. We finetune six encoder models that have been pretrained with very different data (varying in size, language, and period) on a challenging event …


Language Representation Favored Zero-Shot Cross-Domain Cognitive Diagnosis

S Liu, Z Zhou, Y Liu, J Zhang, H Qian - arXiv preprint arXiv:2501.13943, 2025

… However, the language space is substantially disparate from the space of CD, as the former is trained on extensive corpora (eg, Common Crawl, Wikipedia, and BooksCorpus) that are entirely unrelated to education data. Therefore, we …


Language with Cross-Lingual Testing

K Ghosh, A Senapati - Intelligent Computing Systems and Applications: Select …, 2025

Warning: This paper contains examples of the language that some people may find offensive. Social media has become an open platform for all its users. In the comment section, anyone can express their opinion, anger, frustration, and taunt …


Language, Identity, and Bias: Investigating AAVE in Hate Speech Detection

This thesis investigates how hate speech detection models misclassify African American Vernacular English (AAVE) on social media, leading to disproportionate false positives and algorithmic bias. Many systems struggle to distinguish between …


Language-Based AI Modeling of Personality Traits and Pathology from Life Narrative Interviews

JR Oltmanns, R Khandelwal, J Ma, J Brickman, T Do… - 2025

… While BERT was trained on English Wikipedia and the BooksCorpus dataset, RoBERTa was trained on 10 times more data, including the Common Crawl corpus. It was also trained using a dynamic masked language modeling approach as …


Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini

D Yuksel, MC Catalbas, B Oc - arXiv preprint arXiv:2504.06436, 2025

As leading examples of large language models, ChatGPT and Gemini claim to provide accurate and unbiased information, emphasizing their commitment to political neutrality and avoidance of personal bias. This research investigates the …


Large AI Models for Wireless Physical Layer

J Guo, Y Cui, S Jin, J Zhang - arXiv preprint arXiv:2508.02314, 2025

Large artificial intelligence models (LAMs) are transforming wireless physical layer technologies through their robust generalization, multitask processing, and multimodal capabilities. This article reviews recent advancements in LAM …


Large Language Model System Design

J Ren, A Li - Silicon Valley Python Engineer Interview Guide: Data …, 2025

… – Use publicly available text datasets such as Common Crawl, BooksCorpus, Wikipedia, Reddit conversations, and OpenWebText. – For pre-training, you aim to teach the model the structure of language, grammar, facts, reasoning, and some …


Large language model trained on clinical oncology data predicts cancer progression

M Zhu, H Lin, J Jiang, AJ Jinia, J Jee, K Pichotta… - npj Digital Medicine, 2025

Subspecialty knowledge barriers have limited the adoption of large language models (LLMs) in oncology. We introduce Woollie, an open-source, oncology-specific LLM trained on real-world data from Memorial Sloan Kettering Cancer Center (MSK) …


Large Language Models for Arabic Sentiment Analysis and Machine Translation

M Zouidine, M Khalil - Engineering, Technology & Applied Science Research, 2025

Large Language Models (LLMs) have recently demonstrated outstanding performance in a variety of Natural Language Processing (NLP) tasks. Although many LLMs have been developed, only a few models have been evaluated in the …


Large Language Models for NLP: An In-depth Comparative Examination

Z Alomari, O Sharma, S Sawarn, Y Shao, A Makanju

The discipline of Large Language Models (LLMs) is rapidly advancing, and it is essential to explore their capabilities and limitations for further development. This study conducts a comparative analysis of six prominent models: GPT-4, LLaMA 2 …


Large Language Models for Psychological Assessment: A Comprehensive Overview

Large language models (LLMs) are extraordinary tools demonstrating potential to improve our understanding of psychological characteristics. They provide an unprecedented opportunity to supplement self-report in psychology research and …


Large Language Models for Security Operations Centers: A Comprehensive Survey

A Habibzadeh, F Feyzi, RE Atani - arXiv preprint arXiv:2509.10858, 2025

Large Language Models (LLMs) have emerged as powerful tools capable of understanding and generating human-like text, offering transformative potential across diverse domains. The Security Operations Center (SOC), responsible for …


Large language models for software vulnerability detection: a guide for researchers on models, methods, techniques, datasets, and metrics

SM Taghavi Far, F Feyzi - International Journal of Information Security, 2025

Large language models (LLMs) have emerged as transformative tools in the domain of software vulnerability detection and management, offering sophisticated capabilities in identifying, analyzing, and mitigating security risks. This article delves …


Large Language Models for Summarizing Czech Historical Documents and Beyond

V Tran, J Šmıd, J Martınek, L Lenc, P Král

Text summarization is the task of shortening a larger body of text into a concise version while retaining its essential meaning and key information. While summarization has been significantly explored in English and other high-resource …


Large Language Models for Text Classification: What, Why, When, Where, and How

In an age where unstructured text data is growing rapidly, effective methods for text classification (TC) have become critical. Large Language Models (LLMs), such as the revolutionary GPT-4, have taken the lead in tackling this challenge, showing …


Large Language Models in Crisis Informatics for Zero and Few-Shot Classification

C Sánchez, A Abeliuk, B Poblete - ACM Transactions on the Web, 2025

This article presents an exploration of the use of pre-trained Large Language Models (LLMs) for crisis classification to address labeled data dependency issues. We present a methodology that enhances open LLMs through fine-tuning, creating …


Large language models in machine learning

In recent years, large language models (LLMs) have revolutionized the field of machine learning (ML), demonstrating unprecedented capabilities in natural language processing (NLP) tasks. This review provides an in-depth analysis of the …


Large Language Models in the Justice Domain

G Contissa, G Sartor - Facilitating Judicial Cooperation in the EU, 2025

… Data Protection: the majority of the training data for LLM s originates from texts taken from freely accessible internet sources, such as the Common Crawl dataset, which includes information from over 3 billion web pages. These datasets, obtained …


Large Language Models Transform Organic Synthesis From Reaction Prediction to Automation

KKL Tharwani, R Kumar, N Ahmed, Y Tang - arXiv preprint arXiv:2508.05427, 2025

Large language models (LLMs) are beginning to reshape how chemists plan and run reactions in organic synthesis. Trained on millions of reported transformations, these text-based models can propose synthetic routes, forecast reaction outcomes …


Large Language Models With Contrastive Decoding Algorithm for Hallucination Mitigation in Low‐Resource Languages

Z Hongying, A Javed, M Abdullah, J Rashid, M Faheem - CAAI Transactions on …, 2025

… through human effort and web crawlers (ParaCrawl, Bitextor, Common Crawl and OpenNMT). ParaCrawl is a project that aims to build large… Common Crawl provides a large, open repository of web crawl data. OpenNMT provides a suite of …


Large language models: an overview of foundational architectures, recent trends, and a new taxonomy

ID Mienye, N Jere, G Obaido, OO Ogunruku… - 2025

… XLM-R [150] built upon this with robust pretraining on 2.5TB of CommonCrawl data across 100 languages, significantly outperforming previous models on cross-lingual benchmarks, such as XNLI and MLQA. Similarly, mT5 [151] extended the T5 …


Large Language Models: Creation, Optimisation, and Application

AAA Alsayed - The Palgrave Encyclopedia of Computer-Assisted …, 2025

… One of the main sources of this data is text extracted from Internet websites, such as website crawling data from the Common Crawl repository. Many milestones of NLP research developments paved the way for the current generation of LLMs (Raiaan …


Large Scale Cyber Security Log Classification Using Semi-Supervised Clustering

P Cai, M Lazarescu, ST Soh, R Ryan - 2025 IEEE International Conference on Cyber …, 2025

In this paper we present a semi-supervised approach developed with the aim addressing the challenge of large-scale cyber security log entry classification that is faced by organizations that lack significant in-house expertise. Our approach is to …


Large-Scale AI in Telecom: Charting the Roadmap for Innovation, Scalability, and Enhanced Digital Experiences

A Shahid, A Kliks, A Al-Tahmeesschi, A Elbakary… - arXiv preprint arXiv …, 2025

This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems. It …


Large-Scale Diverse Synthesis for Mid-Training

X Zhang, C Tu, C Ren, R Weng, H Yan, J Wang, X Cai - arXiv preprint arXiv …, 2025

The scarcity of high-quality, knowledge-intensive training data hinders the development of large language models (LLMs), as traditional corpora provide limited information. Previous studies have synthesized and integrated corpora-dependent …


Large-Scale Language Models

T Okadome - Essentials of Generative AI, 2025

… The corpora used for pre-training include web archives called Common Crawl, as well as archives of books such as Book1 and Book2. In the pre-training of GPT-3, approximately 500 billion tokens are included. …


Large-Scale Model Training: Dataset Construction, Reliable Scaling, and Task-Specific Adaptation

In recent years, machine learning models have evolved from academic curiosities into widely adopted mainstream tools. This dissertation examines the large-scale training paradigm that enabled this transformation. We first develop an experimental …


LARP: Learner-Agnostic Robust Data Prefiltering

K Minchev, DI Dimitrov, N Konstantinov - arXiv preprint arXiv:2506.20573, 2025

… For example, the Common Crawl web corpus, a widely used dataset in foundation model training, is known to contain toxic, biased, and factually incorrect content [23, 17]. Models trained on such data, including early versions of GPT-3 [7] …


LaTCoder: Converting Webpage Design to Code with Layout-as-Thought

Y Gui, Z Li, Z Zhang, G Wang, T Lv, G Jiang, Y Liu… - Proceedings of the 31st …, 2025

Converting webpage designs into code (design-to-code) plays a vital role in User Interface (UI) development for front-end developers, bridging the gap between visual design and functional implementation. While recent Multimodal Large Language …


Lawfulness of mass processing personal data to train large language models in China

L Zhang - Telecommunications Policy, 2025

With the rapid rise of large language models (LLMs), the lawfulness of training them on massive datasets has come under increasing scrutiny. This article examines the issue under Personal Information Protection law (PIPL) of China, focusing on …


Learning about color from language

Q Liu, J van Paridon, G Lupyan - Communications Psychology, 2025

… Word embedding models trained on COCA-fiction performed better than models trained on much larger corpora: COCA-fiction (at 120 million tokens) outperformed OpenSubtitles (750 million tokens) and Common Crawl (600 billion tokens) though …


Learning Dynamics in Continual Pre-Training for Large Language Models

X Wang, H Tissue, L Wang, L Li, DD Zeng - arXiv preprint arXiv:2505.07796, 2025

… Instead, we simply utilize an open-source Common Crawl dataset as a proxy Dpt to approximate the true general performance dynamics. (b) Secondly, when fitting our scaling law, we regard some variables as unknown parameters to fit. For …


Learning from Sparse and Graph Structured Electrophysiological Data for Brain Disorder Diagnosis

Understanding complex neuronal firing patterns and interactions between neural circuits at different brain regions is essential for uncovering the mechanisms of brain function and dysfunctions. Electrophysiological Source Imaging (ESI) refers to …


Learning without Expert Labels for Multimodal Data

While advancements in deep learning have been largely possible due to the availability of large-scale labeled datasets, obtaining labeled datasets at the required granularity is challenging in many real-world applications, especially in …


LeDoFAN: enhancing lengthy document fake news identification leveraging large language models and explainable window-based transformers with n-gram …

HRI LekshmiAmmal, AK Madasamy - International Journal of Machine Learning and …, 2025

Nowadays, people use social media to gather everything around them and consider it their primary source of information. Moreover, people rely more on information disseminated through social media and news channels. The alarming concern is …


LEDOM: An Open and Fundamental Reverse Language Model

X Yin, S Cheng, Y Xie, X Hu, L Lin, X Wang, L Pan… - arXiv preprint arXiv …, 2025

We introduce LEDOM, the first purely reverse language model, trained autoregressively on 435B tokens with 2B and 7B parameter variants, which processes sequences in reverse temporal order through previous token prediction …


Legal frictions for data openness: Reflections from a case-study on re-use of the open web for AI training

… out of 47 LLMs for text generation published between 2019 and October 2023, at least 64% of these models (30) used at least one filtered version of Common Crawl for pretraining.Other research has revealed correlations between larger quantity of …


LegoAI: Auto-Scaling Large Model Training

Training large AI models is computationally intensive. State-of-the-art language and vision models (LLMs and VLMs) often require thousands of GPUs and weeks or even months of training. As models scale to meet the demands of modern …


Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models

L Liu, M Zhang - arXiv preprint arXiv:2508.06135, 2025

Knowledge Distillation (KD) is a fundamental technique for compressing large language models (LLMs) into compact, efficient student models. However, existing white-box KD methods mainly focus on balancing ground truth and student-generated …


Leveraging Contrastive Semantics and Language Adaptation for Robust Financial Text Classification Across Languages

L Zhang, Q Lin, F Meng, S Liang, J Lu, S Liu, K Chen… - Computers, 2025

With the growing demand for multilingual financial information, cross-lingual financial sentiment recognition faces significant challenges, including semantic misalignment, ambiguous sentiment expression, and insufficient transferability. To …


Leveraging Deep Learning Models and Social Media Data for Enhanced Situation Awareness in Disaster Management

In recent years, social media has become a crucial source of real-time data for disaster management, supporting emergency responses when traditional channels like 911 are overcrowded and overwhelmed. It offers authorities valuable data for …


Leveraging GloVe Embeddings to Enhance Memory-Based Language Modeling for Commonsense Reasoning

… This dataset is derived from educational web pages filtered from the larger Common Crawl dataset. Its content-rich and diverse nature provides a robust corpus for assessing language modeling capabilities. A sample of the first 100,000 lines of …


Leveraging Large Language Models for a Swahili Mathematics ITS in Tanzania: Designing Effective Prompts

EP Rutatola, K Stroeken, T Belpaeme - International Conference on Intelligent …, 2025

The advancement of Large Language Models (LLMs) has significantly enhanced intelligent tutoring systems, enabling them to engage learners through natural dialogues. This interaction boosts learner engagement but presents challenges for …


Leveraging Large Language Models for Legal Document Understanding and Software System Analysis: Addressing Key Challenges

In the rapidly advancing field of software development, ensuring compliance with legal regulations and policies has become increasingly critical. The intricate separation between legal expertise and software engineering creates challenges …


Leveraging Large Language Models to Identify the Values Behind Arguments

LC Siebert - Value Engineering in Artificial Intelligence

RA Senthilkumar, A Homayounirad, LC Siebert - International Workshop on Value …, 2025

Human values capture what people and societies perceive as desirable, transcend specific situations and serve as guiding principles for action. People’s value systems motivate their positions on issues concerning the economy, society and politics …


Leveraging LLMs for Continuous Data Streams: Methods and Applications

R Kumar - Innovations in Data Analytics: Selected Papers of …, 2025

This paper explores the integration of large language models (LLMs) with continuous data streams, addressing the challenges and methodologies asso-ciated with this integration. It examines two primary approaches: continual learning and …


Leveraging Machine-Labeled Data and Cross-Lingual Transfer for NER in Urdu and Sindhi

N Basir, DN Hakro, ZB Khalil-Ur-Rehman Khoumbati - … btn btn-dark btn-xs btn …, 2025

… XLM-R is an extension of RoBERTa which is pre-trained on 2.5TB of CommonCrawl data across 100 languages. … The model XLM-RoBERTa, a more recent model trained on a larger Common Crawl dataset across 100 languages, has …


Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domains

J Kim, H Zhang, D Jurgens - arXiv preprint arXiv:2509.16531, 2025

Authorship representation (AR) learning, which models an author's unique writing style, has demonstrated strong performance in authorship attribution tasks. However, prior research has primarily focused on monolingual settings-mostly in English-leaving …


Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guarantees

S Meisenbacher, M Chevli, F Matthes - arXiv preprint arXiv:2508.20736, 2025

Many works at the intersection of Differential Privacy (DP) in Natural Language Processing aim to protect privacy by transforming texts under DP guarantees. This can be performed in a variety of ways, from word perturbations to full document …


Leveraging sentiment analysis of food delivery services reviews using deep learning and word embedding

D Mustafa, SM Khabour, M Al-kfairy, A Shatnawi - PeerJ Computer Science, 2025

… The Arabic language’s standardized fastText word vectors were pre-trained using the Common Crawl and Wikipedia resources. For the training, a position weight with a dimension of 300 and a continuous bag of words (CBOW) was employed …


Leveraging Textual Description and Structured Data for Estimating Crash Risks of Traffic Violation: A Multimodal Learning Approach

Z Li, C Ma, Y Zhou, D Lord, Y Zhang - IEEE Transactions on Intelligent Transportation …, 2025

This study introduces a novel methodology that integrates both structured data and unstructured violation descriptions, addressing a critical gap in current crash risk estimation techniques. By combining these two data types, our approach captures a …


Leveraging Transformer Models for Enhanced Pharmacovigilance: A Comparative Analysis of ADR Extraction from Biomedical and Social Media Texts

O Elbiach, H Grissette, EH Nfaoui - AI, 2025

The extraction of Adverse Drug Reactions from biomedical text is a critical task in the field of healthcare and pharmacovigilance. It serves as a cornerstone for improving patient safety by enabling the early identification and mitigation of potential risks …


Leveraging Visual Scene Graph to Enhance Translation Quality in Multimodal Machine Translation

A Hatami, M Arcan, P Buitelaar - Proceedings of Machine Translation Summit XX …, 2025

… The model is pretrained on mC4 (Multilingual Common Crawl), a largescale dataset containing filtered web text from a wide range of languages. This extensive training allows mT5 to perform well in both high-resource and low-resource …


Leveraging word embeddings to enhance co-occurrence networks: A statistical analysis

DR Amancio, J Machicao, LVC Quispe - PloS one, 2025

… These embeddings were trained on large-scale corpora, including Common Crawl and Wikipedia, providing comprehensive coverage of general language usage. FastText operates at the character level, eliminating the need for …


like 70 Follow Language Technologies Unit@ Barcelona Supercomputing Center 225

… Common Crawl: Repository that holds website data and is run by the Common Crawl non-profit organization. It is updated monthly and is distributed under the CC0 1.0 … Web-sourced datasets with some preprocessing available under permissive …


Limited Generalizability in Argument Mining: State-Of-The-Art Models Learn Datasets, Not Arguments

M Feger, K Boland, S Dietze - arXiv preprint arXiv:2505.22137, 2025

Identifying arguments is a necessary prerequisite for various tasks in automated discourse analysis, particularly within contexts such as political debates, online discussions, and scientific reasoning. In addition to theoretical advances in …


LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models

Z Ning, T Gu, J Song, S Hong, L Li, H Liu, J Li, Y Wang… - arXiv preprint arXiv …, 2025

The widespread adoption and increasing prominence of large language models (LLMs) in global technologies necessitate a rigorous focus on ensuring their safety across a diverse range of linguistic and cultural contexts. The lack of a comprehensive …


Linguistic Entity Masking to Improve Cross-Lingual Representation of Multilingual Language Models for Low-Resource Languages

A Fernando, S Ranathunga - arXiv preprint arXiv:2501.05700, 2025

… It is a collection of document-level data of 3 Trillion tokens from Common Crawl2 for 419 languages. As the dependent-monolingual data, we obtain the monolingual sides from the SiTa-Trilingual parallel dataset [44]. It is a human-curated gold …


LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points

X Zhang, C Ren, C Tu, R Weng, H Yan, J Wang, X Cai - arXiv preprint arXiv …, 2025

The advancement of large language models (LLMs) struggles with the scarcity of high-quality, diverse training data. To address this limitation, we propose LinkSyn, a novel knowledge point (KP) graph-based synthesis framework that enables flexible …


Links Are All You Need: Graph Embeddings for Website Analysis and Classification

… Using a 21TB subset of Common Crawl (nd) data, we demonstrate how these embeddings can effectively capture relationships between websites without requiring content processing. The study achieves competitive results in political bias …


LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch

J Pfister, J Wunderle, A Hotho - Proceedings of the 63rd Annual Meeting of the …, 2025

We transparently create two German-only decoder models, LLäMmlein 120M and 1B, from scratch and publish them, along with the training data, for the (German) NLP research community to use. The model training involved several key steps, including …


Llama-GENBA-10B: A Trilingual Large Language Model for German, English and Bavarian

M Hoffmann, J John, S Schweter, G Ramakrishnan… - arXiv preprint arXiv …, 2025

We present Llama-GENBA-10B, a trilingual foundation model addressing English-centric bias in large language models. Built on Llama 3.1-8B and scaled to 10B parameters, Llama-GENBA-10B is continuously pretrained on 164B tokens (82B English, 82B …


LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems

VHG Moia, IJ Sanz, GAF Rebello, RD de Meneses… - arXiv preprint arXiv …, 2025

The success and wide adoption of generative AI (GenAI), particularly large language models (LLMs), has attracted the attention of cybercriminals seeking to abuse models, steal sensitive data, or disrupt services. Moreover, providing security to LLM-based …


llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

I Sugiura, K Nakayama, Y Oda - arXiv preprint arXiv:2504.15544, 2025

Encoder-only transformer models like BERT are widely adopted as a pre-trained backbone for tasks like sentence classification and retrieval. However, pretraining of encoder models with large-scale corpora and long contexts has been relatively …


LLM360 K2: Scaling Up 360-Open-Source Large Language Models

Z Liu, B Tan, H Wang, W Neiswanger, T Tao, H Li… - arXiv preprint arXiv …, 2025

We detail the training of the LLM360 K2-65B model, scaling up our 360-degree OPEN SOURCE approach to the largest and most powerful models under project LLM360. While open-source LLMs continue to advance, the answer to "How are the …


LLMControl: Grounded Control of Text-to-Image Diffusion-based Synthesis with Multimodal LLMs

J Wang, R Chen, H Cui - arXiv preprint arXiv:2507.19939, 2025

… We use a dataset of random 1 M image-text pairs with high scores in the Common Crawl Web index and adjust the resolution of the images to 512×512. Subsequently, we apply ODISE [20] to obtain the instance segmentation map. To obtain …


LLMic: Romanian Foundation Language Model

VA Bădoiu, MV Dumitru, AM Gherghescu, A Agache… - arXiv preprint arXiv …, 2025

… of tokens requires extensive filtering and cleaning of CommonCrawl’s petabyte-scale dataset. … We leverage two filtered CommonCrawl sources for Romanian language data: FuLG [3], … We further augment our dataset by incorporating filtered content …


LLMs Are Globally Multilingual Yet Locally Monolingual: Exploring Knowledge Transfer via Language and Thought Theory

E Kang, J Kim - arXiv preprint arXiv:2505.24409, 2025

… According to Common Crawl statistics, these languages exhibit varying levels of multilingual web content (ZH: 5.27%, KO: 0.76%, AR: 0.68%),2 reflecting a spectrum from relatively higher… 2commoncrawl.github.io/cc-crawl-statistics/ plots/languages …


LLMs on support of privacy and security of mobile apps: state of the art and research directions

TTL Nguyen, B Carminati, E Ferrari - arXiv preprint arXiv:2506.11679, 2025

Modern life has witnessed the explosion of mobile devices. However, besides the valuable features that bring convenience to end users, security and privacy risks still threaten users of mobile apps. The increasing sophistication of these threats in …


LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text

I Tolstykh, A Tsybina, S Yakubson, M Kuprashevich - arXiv preprint arXiv:2509.21269, 2025

… For the English corpus, we utilized sources such as Common Crawl2, Wikipedia dumps, news articles (CNN, New York Times), academic abstracts (arXiv, SSRN), and community forums (Reddit, Yelp). A complete list of all data sources is provided …


LM anthropomorphization: balancing ethics and business value

M Reusens, B Baesens - Journal of Business Analytics, 2025

… It is a well-known fact that the Common Crawl data set is tailored towards English-speaking people within the US, comprising mostly of privileged English dialects (Gururangan et al., Citation2022; … An analysis of undesirable content in the Common Crawl …


Loanword Identification in Social Media Texts with Extended Code-Switching Datasets

C Mi, S Xie, Y Li, Z He - ACM Transactions on Asian and Low-Resource …, 2025

… To build the multilingual BERT for multilingual loanword identiication model, we irst trained pre-trained embeddings for 20 languages on Common Crawl and Wikipedia data. Our model includes 16 layers and 400 dimensions in each layer …


Locality, Relation, and Meaning Construction in Language, as Implemented in Humans and Large Language Models (LLMS)

In this thesis, we explore language and cognition in both people and in computational models, through the lens of meaning construction at both the individual and collective level. We step through levels of linguistic structure and …


Locality-Sensitive Indexing for Graph-Based Approximate Nearest Neighbor Search

JW Chung, H Lin, W Zhao - Proceedings of the 48th International ACM SIGIR …, 2025

… All observations in this section are demonstrated using the initial 1000000 documents (wrt timestamps) of the multilingual Common Crawl news database [15, 18] rendered via the MiniLM-6 [17, 51] pre-trained transformer into 384-dimension …


Location is All You Need: Copyright Extraterritoriality and Where to Train Your AI

M Rättzén - Science and Technology Law Review, 2025

… For example, OpenAI used publicly accessible datasets such as Common Crawl, WebText2, and Wikipedia for training ChatGPT-3.Common Crawl is a large open repository of web-crawling data, consisting of nearly a … Common Crawl …


LogDB: Multivariate Log-based Failure Diagnosis for Distributed Databases (Extended from MultiLog)

L Zhang, T Jia, M Jia, Y Li - arXiv preprint arXiv:2505.01676, 2025

Distributed databases, as the core infrastructure software for internet applications, play a critical role in modern cloud services. However, existing distributed databases frequently experience system failures and performance degradation, often leading to …


Long-context Reference-based MT Quality Estimation

SU Haq, CC Osuji, S Castilho, B Davis - arXiv preprint arXiv:2509.13980, 2025

In this paper, we present our submission to the Tenth Conference on Machine Translation (WMT25) Shared Task on Automated Translation Quality Evaluation. Our systems are built upon the COMET framework and trained to predict segment-level …


LongAttn: Selecting Long-context Training Data via Token-level Attention

L Wu, D Zhu, G Zhao, Z Yu, J Ran, X Wong, L Sun, S Li - arXiv preprint arXiv …, 2025

… Simple methods to construct long-context datasets are through naive methods like concatenating short texts or randomly sampling existing sources (eg, CommonCrawl, GitHub). However, studies by de Vries (2023) and Chen et al. (2024a) …


LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm

S Wu, Y Li, X Qu, R Ravikumar, Y Li, TLSQX Wei… - arXiv preprint arXiv …, 2025

Large Language Models (LLMs) have achieved remarkable success in various natural language processing tasks, yet their ability to generate long-form content remains poorly understood and evaluated. Our analysis reveals that current LLMs …


LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation

Z Dong, J Li, J Jiang, M Xu, WX Zhao, B Wang, W Chen - arXiv preprint arXiv …, 2025

Large language models (LLMs) have gained extended context windows through scaling positional encodings and lightweight continual pre-training. However, this often leads to degraded performance on short-text tasks, while the reasons for this …


LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning

Y Wu, Y Bai, Z Hu, RKW Lee, J Li - arXiv preprint arXiv:2506.18841, 2025

Ultra-long generation by large language models (LLMs) is a widely demanded scenario, yet it remains a significant challenge due to their maximum generation length limit and overall quality degradation as sequence length increases. Previous …


LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM

Y Refael, I Arbel, O Lindenbaum, T Tirer - arXiv preprint arXiv:2502.19571, 2025

We study robust parameter-efficient fine-tuning (PEFT) techniques designed to improve accuracy and generalization while operating within strict computational and memory hardware constraints, specifically focusing on large-language models (LLMs) …


Loss and Reward Functions for Generative Question Answering Systems

Recent advancements in AI, mainly through Large Language Models (LLMs), have transformed the field, driving both industrial and academic progress. These models are typically trained using causal language modeling tasks, which, while effective …


Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction

Y Mao, H Pirk, CJ Xue - arXiv preprint arXiv:2505.06297, 2025

… An analysis of over 6 billion sentences from Common Crawl further revealed that more than 57% were machine-translated, highlighting the extent of automated content generation [47]. In fact, LLMs have also influenced various fields, including …


Lossy Loops: Shannon's DPI and Information Decay in Generative Model Training

Abstract Model collapse, the progressive degradation of generative AI performance when trained on synthetic data, poses a critical challenge for modern AI systems. This paper establishes a theoretical framework based on Shannon's Data …