Research

Research papers, whitepapers, case studies, etc. 1271 public listings.

• PASHA: Efficient HPO and NAS with Progressive Resource Allocation (Cohere)
  When models are trained on large datasets, tuning them with HPO or NAS rapidly becomes prohibitively expensive for practitioners, even when…
• Self-Improving Robust Preference Optimization (Cohere)
  Both online and offline RLHF methods such as PPO and DPO have been extremely successful in aligning AI with human preferences.
• αNAS: Neural Architecture Search using Property Guided Synthesis (Cohere)
  In the past few years, neural architecture search (NAS) has become an increasingly important tool within the deep learning community.
• If You Can't Use Them, Recycle Them (Cohere)
  Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging ``generalist'' models…
• Aya Model: Open-Access Multilingual Language Model (Cohere)
  Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages.
• Policy Primer - The AI Language Gap (Cohere)
  More than 7000 languages are spoken around the world today, but current, state-of-art AI large language models cover only a small percentag…
• Evaluating the Retrieval Robustness of Large Language Models (Cohere)
  Retrieval-augmented generation (RAG) generally enhances large language models' (LLMs) ability to solve knowledge-intensive tasks.
• Verification Limits Code LLM Training (Cohere)
  Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are gene…
• Imitating Language via Scalable Inverse Reinforcement Learning (Cohere)
  The majority of language model training builds on imitation learning.
• Policy Primer - The Limits of Thresholds (Cohere)
  Prominent AI governance frameworks around the world have specified thresholds based on the amount of computing power used to train an AI mo…
• The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning (Cohere)
  Reasoning in LLMs is overwhelmingly studied in domains that provide a model with rules: mathematics and code.
• Efficient Methods for Natural Language Processing: A Survey (Cohere)
  Mar 24, 2023 Efficient Methods for Natural Language Processing: A Survey Synthesizing methods and findings in NLP efficiencies, guiding new…
• LLMCRIT: Teaching Large Language Models to Use Criteria (Cohere)
  Humans follow criteria when they execute tasks, and these criteria are directly used to assess the quality of task completion.
• Consent in Crisis: The Rapid Decline of the AI Data Commons (Cohere)
  Consent in Crisis: The Rapid Decline of the AI Data Commons General-purpose AI systems are built on massive swathes of public web data, ass…
• Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers (Cohere)
  One of the most profound challenges of modern machine learning is performing well on the long-tail of rare and underrepresented features.
• How Does Quantization Affect Multilingual LLMs? (Cohere)
  Quantization techniques are widely used to improve inference speed and deployment of large language models.
• Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models (Cohere)
  Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Toxicity definitions evolve and change over time.
• BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM (Cohere)
  Accurate evaluation is central to the large language model (LLM) ecosystem, guiding model selection and downstream adoption across diverse…
• Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models (Cohere)
  As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality.
• Large Language Models are not Zero Shot Communicators (Cohere)
  Oct 26, 2022 Large Language Models are not Zero Shot Communicators Investigating the implicature gap in Large Language Models.
• The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It (Cohere)
  This paper presents a comprehensive analysis of the linguistic diversity of LLM safety research, highlighting the English-centric nature of…
• Tools Fail: Detecting Silent Errors in Faulty Tools (Cohere)
  Tools have become a mainstay of LLMs, allowing them to retrieve knowledge not in their weights, to perform tasks on the web, and even to co…
• IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models (Cohere)
  Despite the widespread adoption of Large language models (LLMs), their remarkable capabilities remain limited to a few high-resource langua…
• The State of Cloud Native Development in Japan 2026 (Cncf)
  Japan’s cloud native landscape is growing rapidly—but along a unique path.
• Crosslingual Reasoning through Test-Time Scaling (Cohere)
  Reasoning capabilities of large language models are primarily studied for English, even when pretrained models are multilingual.
• INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge (Cohere)
  Nov 29, 2024 INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge In this work, we construct an evaluation suite…
• Scalable Training of Language Models using PAX pjit and TPUv4 (Cohere)
  Modern large language models require distributed training strategies due to their size.
• Fairness of Deep Ensembles: On the interplay between per-group task difficulty and (Cohere)
  Ensembling is commonly regarded as an effective way to improve the general performance of models in machine learning, while also increasing…
• When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale (Cohere)
  Large volumes of text data have contributed significantly to the development of large language models (LLMs) in recent years.
• Intriguing Properties of Quantization at Scale (Cohere)
  Emergent properties have been widely adopted as a term to describe behavior not present in smaller models but observed in larger models.
• Understanding Likelihood Over-optimisation in Direct Alignment Algorithms (Cohere)
  Direct Alignment Algorithms (DAAs), such as Direct Preference Optimisation (DPO) and Identity Preference Optimisation (IPO), have emerged a…
• The PRISM Alignment Project: What Participatory, Representative and Individualised Human Feedback (Cohere)
  Human feedback plays a central role in the alignment of Large Language Models (LLMs).
• BPE Stays on SCRIPT: Structured Encoding for Robust Multilingual Pretokenization (Cohere)
  Byte Pair Encoding (BPE) tokenizers, widely used in Large Language Models, face challenges in multilingual settings, including penalization…
• Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models (Cohere)
  The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous Solid Gold…
• Block Diffusion: Bridging Autoregressive & Diffusion LMs (Cohere)
  Mar 12, 2025 Block Diffusion: Bridging Autoregressive & Diffusion LMs Authors Marianne Arriola, Aaron Gokaslan, Justin T.
• HeyGen x Google Cloud: Bringing Avatar IV to TPUs (Heygen)
  Learn how HeyGen and Google Cloud optimized Avatar IV for Trillium TPUs, achieving a 1.86× speedup and up to 25% better cost efficiency for…
• Pathways to Open Data: Findings from the 2024 World Open Innovation Conference Challenge Session (Linuxfoundation)
  LINUX FOUNDATION RESEARCH We’re investigating the impact of open source collaboration to solve the world’s most pressing challenges.
• 9th Annual Open Source Jobs Report (Linuxfoundation)
  LINUX FOUNDATION RESEARCH We’re investigating the impact of open source collaboration to solve the world’s most pressing challenges.
• Intriguing Properties of Compression on Multilingual Models (Cohere)
  Multilingual models are often particularly dependent on scaling to generalize to a growing number of languages.
• The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution (Cohere)
  Humans interpret language using beliefs and prior knowledge about the world. For example, we intuitively understand the response "I wore gl…
• The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI (Cohere)
  The race to train language models on vast, diverse, and inconsistently documented datasets has raised pressing concerns about the legal and…
• When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs (Cohere)
  Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without…
• Tiny Aya: Bridging Scale and Multilingual Depth (Cohere)
  Tiny Aya redefines what a small multilingual language model can achieve.
• No Need for Explanations: LLMs can implicitly learn from mistakes in-context (Cohere)
  Showing incorrect answers to Large Language Models (LLMs) is a popular strategy to improve their performance in reasoning-intensive tasks.
• A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic (Cohere)
  Keyword mnemonics are memorable explanations that link new terms to simpler keywords.
• MTEB: Massive Text Embedding Benchmark (Cohere)
  Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other task…
• Predicting Twitter Engagement With Deep Language Models (Cohere)
  Twitter has become one of the main information sharing platforms for millions of users world-wide.
• Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning (Cohere)
  Large Language Models (LLMs) have been adopted and deployed worldwide for a broad variety of applications.
• Evaluating the Social Impact of Generative AI Systems in Systems and Society (Cohere)
  Generative AI systems across modalities, ranging from text, image, audio, and video, have broad social impacts, but there exists no officia…
• Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress (Cohere)
  The use of synthetic data has played a critical role in recent state-of-art breakthroughs.