Cohere · Research

Research listings from Cohere on TechiSeek. 142 public listings.

• BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts (Cohere)
  The Mixture of Experts (MoE) framework has become a popular architecture for large language models due to its superior performance over den…
• Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve? (Cohere)
  In the last decade, the generalization and adaptation abilities of deep learning models were typically evaluated on fixed training and test…
• Elo Uncovered: Robustness and Best Practices in Language Model Evaluation (Cohere)
  In Natural Language Processing (NLP), the Elo rating system, originally designed for ranking players in dynamic games such as chess, is inc…
• No News is Good News: A Critique of the One Billion Word Benchmark (Cohere)
  Araújo, Jeffrey Hui, Nicholas Frosst Abstract The One Billion Word Benchmark is a dataset derived from the WMT 2011 News Crawl, commonly us…
• The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm (Cohere)
  A key concern with the concept of "alignment" is the implicit question of "alignment to what?".
• SnapKV: LLM Knows What You are Looking for Before Generation (Cohere)
  Apr 22, 2024 SnapKV: LLM Knows What You are Looking for Before Generation Authors Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh,…
• Policy Primer - Translating Safety (Cohere)
  Global AI safety efforts have gained traction and momentum, but a critical challenge remains: how to ensure safety across diverse languages…
• From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models (Cohere)
  To date, toxicity mitigation in language models has almost entirely been focused on single-language settings.
• Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data (Cohere)
  There are a number of diverging hypotheses about the neural text degeneration problem, i.e., generating repetitive and dull loops, which ma…
• Policy Primer - Efficient AI (Cohere)
  The growing global demand for digital products and services is increasing energy needs and their associated climate impacts.
• Understanding and Mitigating Language Confusion in LLMs (Cohere)
  We investigate a surprising limitation of LLMs: their inability to consistently generate text in a user's desired language.
• FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling (Cohere)
  Ensembling independent deep neural networks (DNNs) is a simple and effective way to improve top-line metrics and to outperform larger singl…
• Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics (Cohere)
  Modern machine learning research relies on relatively few carefully curated datasets.
• Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashion (Cohere)
  Reinforcement Learning (RL) has been used to finetune Large Language Models (LLMs) using a reward model trained from preference data, to be…
• Exploring Low Rank Training of Deep Neural Networks (Cohere)
  Training deep neural networks in low rank, i.e.
• Investigating Continual Pretraining in Large Language Models: Insights and Implications (Cohere)
  This paper studies the evolving domain of Continual Learning (CL) in large language models (LLMs), with a focus on developing strategies fo…
• Interlocking Backpropagation: Improving depthwise model-parallelism (Cohere)
  The number of parameters in state of the art neural networks has drastically increased in recent years.
• To Code, or Not To Code? Exploring Impact of Code in Pre-training (Cohere)
  Including code in the pre-training data mixture, even for models not specifically designed for code, has become a common practice in LLMs p…
• The Future of International Scientific Assessments of AI’s Risks (Cohere)
  Managing the risks of artificial intelligence (AI) will require international coordination among many actors with different interests, valu…
• Bridging the Data Provenance Gap Across Text, Speech, and Video (Cohere)
  Progress in AI is driven largely by the scale and quality of training data.
• EAGER: Entropy-Aware Generation for Adaptive Inference-Time Scaling (Cohere)
  With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computa…
• Scalable Data Ablation Approximations for Language Models through Modular Training and Merging (Cohere)
  Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance.
• AI Exposure Scores: What they measure, what they miss, and what comes next (Cohere)
  A set of exposure scores calculated in 2023 have become a central empirical input to the future of work debate.
• Open Problems in Technical AI Governance (Cohere)
  AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated.
• BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model (Cohere)
  The Big Science Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the…
• Aya Vision: Multilingual Multimodal AI Advancements (Cohere)
  May 14, 2025 Aya Vision: Multilingual Multimodal AI Advancements To address the challenges of building multimodal language models, we intro…
• On the Limitations of Compute Thresholds as a Governance Strategy (Cohere)
  At face value, this essay is about understanding a fairly esoteric governance tool called compute thresholds.
• Human Feedback is not Gold Standard (Cohere)
  Sep 28, 2023 Human Feedback is not Gold Standard Authors Tom Hosking, Phil Blunsom, Max Bartolo Abstract Human feedback has become the de f…
• SimMerge: Learning to Select Merge Operators from Similarity Signals (Cohere)
  Model merging enables multiple large language models (LLMs) to be combined into a single model while preserving performance.
• Lifting the Veil on Hyper-parameters for Value-based Deep Reinforcement Learning (Cohere)
  Successful applications of deep reinforcement learning (deep RL) combine algorithmic design and careful hyper-parameter selection.
• CALIBER: Calibrating confidence before and after reasoning in language models (Cohere)
  Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success.
• SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (Cohere)
  Authors Holy Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar, Lester James V. Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jon…
• Associative Memory Augmented Asynchronous Spatiotemporal Representation Learning for Event-based (Cohere)
  We propose EventFormer– a computationally efficient event-based representation learning framework for asynchronously processing event camer…
• Making, not Taking, the Best of N (Cohere)
  Obtaining high-quality generations in modern LLMs has largely been framed as a selection problem: identifying a single winning generation f…
• One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers (Cohere)
  Pretraining massively multilingual Large Language Models (LLMs) for many languages at once is challenging due to limited model capacity, sc…
• Near-Optimal Distributionally Robust Reinforcement Learning with General Norms (Cohere)
  To address the challenges of sim-to-real gap and sample efficiency in reinforcement learning (RL), this work studies distributionally robus…
• PASHA: Efficient HPO and NAS with Progressive Resource Allocation (Cohere)
  When models are trained on large datasets, tuning them with HPO or NAS rapidly becomes prohibitively expensive for practitioners, even when…
• Self-Improving Robust Preference Optimization (Cohere)
  Both online and offline RLHF methods such as PPO and DPO have been extremely successful in aligning AI with human preferences.
• αNAS: Neural Architecture Search using Property Guided Synthesis (Cohere)
  In the past few years, neural architecture search (NAS) has become an increasingly important tool within the deep learning community.
• If You Can't Use Them, Recycle Them (Cohere)
  Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging ``generalist'' models…
• Aya Model: Open-Access Multilingual Language Model (Cohere)
  Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages.
• Policy Primer - The AI Language Gap (Cohere)
  More than 7000 languages are spoken around the world today, but current, state-of-art AI large language models cover only a small percentag…
• Evaluating the Retrieval Robustness of Large Language Models (Cohere)
  Retrieval-augmented generation (RAG) generally enhances large language models' (LLMs) ability to solve knowledge-intensive tasks.
• Verification Limits Code LLM Training (Cohere)
  Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are gene…
• Imitating Language via Scalable Inverse Reinforcement Learning (Cohere)
  The majority of language model training builds on imitation learning.
• Policy Primer - The Limits of Thresholds (Cohere)
  Prominent AI governance frameworks around the world have specified thresholds based on the amount of computing power used to train an AI mo…
• The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning (Cohere)
  Reasoning in LLMs is overwhelmingly studied in domains that provide a model with rules: mathematics and code.
• Efficient Methods for Natural Language Processing: A Survey (Cohere)
  Mar 24, 2023 Efficient Methods for Natural Language Processing: A Survey Synthesizing methods and findings in NLP efficiencies, guiding new…
• LLMCRIT: Teaching Large Language Models to Use Criteria (Cohere)
  Humans follow criteria when they execute tasks, and these criteria are directly used to assess the quality of task completion.
• Consent in Crisis: The Rapid Decline of the AI Data Commons (Cohere)
  Consent in Crisis: The Rapid Decline of the AI Data Commons General-purpose AI systems are built on massive swathes of public web data, ass…