TechiSeek

VendorsFor Vendors

SCOPE: Selective Critical-Step On-Policy Self-Distillation for Agentic Reinforcement Learning

arXiv cs.AI

SCOPE: Selective Critical-Step On-Policy Self-Distillation for Agentic Reinforcement Learning Research

SCOPE (Selective Critical-Step On-Policy Self-Distillation) is a framework for training long-horizon LLM agents with reinforcement learning. It addresses a core weakness of outcome-based RL: trajectory-level rewards are coarse, so failed trajectories may contain correct steps and successful ones can contain errors.

Practical examples

  • On unseen ALFWorld tasks, SCOPE reaches 89.6% success, improving over GRPO by 23.2 percentage points with 46.7% fewer interactions.

Key takeaways

  • A bounded signed teacher-student coefficient stabilizes auxiliary updates while preserving positive reinforcement and negative correction.
  • A success-relative pairwise length reward encourages efficient task completion.
Read More

Recommended

Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS AppsOnline Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS AppsarXiv cs.AI Reward-Adaptive Co-Evolution of Policy and RubricReward-Adaptive Co-Evolution of Policy and RubricarXiv cs.AI Learning to Bound: Monotonic, Calibrated Neural Upper Bounds for MPE InferenceLearning to Bound: Monotonic, Calibrated Neural Upper Bounds for MPE InferencearXiv cs.AI Efficient Learning of Mechanism Designs and CompositionalityEfficient Learning of Mechanism Designs and CompositionalityarXiv cs.AI A General Topological Symmetry Index for Critical Orbits In Neural Network Loss LandscapesA General Topological Symmetry Index for Critical Orbits In Neural Network Loss LandscapesarXiv cs.AI Always Learning, Always Mixing: Efficient and Simple Data Mixing All the TimeAlways Learning, Always Mixing: Efficient and Simple Data Mixing All the TimearXiv cs.AI The Language of LGBTQ Inclusion and Allyship - Deepgram Blog ⚡️The Language of LGBTQ Inclusion and Allyship - Deepgram Blog ⚡️Deepgram An Introduction to Redis-ML. Part OneAn Introduction to Redis-ML. Part OneRedis How to connect Google Sheets to NotionHow to connect Google Sheets to NotionZapier Lessons Swift Designer Chris Lattner Has Learned about LeadershipLessons Swift Designer Chris Lattner Has Learned about LeadershipThenewstack Empower Your Team with Policy as CodeEmpower Your Team with Policy as CodePulumi Announcing Restore Stacks: Recover Deleted Stacks in the Pulumi CloudAnnouncing Restore Stacks: Recover Deleted Stacks in the Pulumi CloudPulumi New Policy as Code Capabilities with CrossGuardNew Policy as Code Capabilities with CrossGuardPulumi Go Wiki: PortingPolicy - The Go Programming LanguageGo Wiki: PortingPolicy - The Go Programming LanguageGo Develop, Preview, TestDevelop, Preview, TestCss Tricks Jetpack Boost Handles Critical CSS For YouJetpack Boost Handles Critical CSS For YouCss Tricks Summer Data Engineering RoadmapSummer Data Engineering RoadmapMotherduck How AI Assistants Can Decode GitHub Repos for UI WritersHow AI Assistants Can Decode GitHub Repos for UI WritersDocker How To Create and Use Avatars in eLearningHow To Create and Use Avatars in eLearningHeygen NO_EXTERNAL_CSS_AT_IMPORTSNO_EXTERNAL_CSS_AT_IMPORTSVercel