Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure.” Although originally from economics, it’s something we have to grapple with at OpenAI when figuring out how to optimize objectives that are difficult or costly to measure.
Recommended
Credit Saison achieves 954% ROI in a ChatGPT Enterprise pilotOpenai Making it easier for websites to express AI training preferencesOpenai Community safety report and new protections for teensOpenai Operation "Doppelganger": Russian influence activity targeting UkraineOpenai Everyday work with ChatGPT WorkOpenai Romance scams: AI-enabled romance scam workflowsOpenai
Measuring CI/CD Adoption Rates Is a ProblemThenewstack
Best practices for measuring Web Vitals in the fieldWeb
What is Round Trip Time (RTT) and how can it be measured?Bunny
Towards measuring the representation of subjective global opinions in language modelsAnthropic
It Can Only Goodhart HappenHoneycomb
The Psychology Of Trust In AI: A Guide To Measuring And Designing For User ConfidenceSmashingmagazine
1.4 Measuring Latency and ThroughputBaseten
Observability: Are You Measuring What Actually Matters?Honeycomb
The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating ItCohere
Measuring the Economic Value of Open SourceLinuxfoundation
Measuring Claude Code ROI and Adoption in HoneycombHoneycomb
Track social media campaigns with T2M URL ShortenerZapier
Using Replit to get our grandfather the dailyReplit
Measuring faithfulness in Chain-of-Thought reasoningAnthropic
