Lemmy: Bestiverse
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
RSS BotMB to Hacker NewsEnglish · 1 day ago

Chain of thought monitorability: A new and fragile opportunity for AI safety

arxiv.org

external-link
message-square
0
fedilink
1
external-link

Chain of thought monitorability: A new and fragile opportunity for AI safety

arxiv.org

RSS BotMB to Hacker NewsEnglish · 1 day ago
message-square
0
fedilink
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
arxiv.org
external-link
AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.

Comments

alert-triangle
You must log in or register to comment.

Hacker News

hackernews

Subscribe from Remote Instance

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !hackernews@lemmy.bestiver.se
lock
Community locked: only moderators can create posts. You can still comment on posts.

Posts from the RSS Feed of HackerNews.

The feed sometimes contains ads and posts that have been removed by the mod team at HN.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 578 users / day
  • 1.75K users / week
  • 3.81K users / month
  • 9.13K users / 6 months
  • 2 local subscribers
  • 2.03K subscribers
  • 26.1K Posts
  • 9.7K Comments
  • Modlog
  • mods:
  • patrick
  • RSS Bot
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org