Lemmy: Bestiverse
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
RSS BotMB to Hacker NewsEnglish · 21 days ago

Tao on "blue team" vs. "red team" LLMs

mathstodon.xyz

external-link
message-square
0
fedilink
1
external-link

Tao on "blue team" vs. "red team" LLMs

mathstodon.xyz

RSS BotMB to Hacker NewsEnglish · 21 days ago
message-square
0
fedilink
Terence Tao (@tao@mathstodon.xyz)
mathstodon.xyz
external-link
In the field of cybersecurity, a distinction is made between the "blue team" task of building a secure system, and the "red team" task of locating vulnerabilities in such systems. The blue team is more obviously necessary to create the desired product; but the red team is just as essential, given the damage that can result from deploying insecure systems. The nature of these teams mirror each other; mathematicians would call them "dual". The output of a blue team is only as strong as its weakest link: a security system that consists of a strong component and a weak component (e.g., a house with a securely locked door, but an open window) will be insecure (and in fact worse, because the strong component may convey a false sense of security). Dually, the contributions to a red team can often be additive: a red team report that contains both a serious vulnerability and a more trivial one is more useful than a report that only contains the serious issue, as it is valuable to have the blue team address both vulnerabilities. (But excessive low-quality reports can dilute attention from critical issues.) Because of this, unreliable contributors may be more useful in the "red team" side of a project than the "blue team" side, though the blue team can still accommodate such contributors provided that the red team is competent enough to catch almost all of the errors that the contributor to the blue team might make. Also, unreliable red team contributions only add value if they _augment_ the output of more reliable members of that team, rather than _replace_ that output, and if their output can be effectively filtered or triaged by more experienced red team members. (1/3)

Comments

alert-triangle
You must log in or register to comment.

Hacker News

hackernews

Subscribe from Remote Instance

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !hackernews@lemmy.bestiver.se
lock
Community locked: only moderators can create posts. You can still comment on posts.

Posts from the RSS Feed of HackerNews.

The feed sometimes contains ads and posts that have been removed by the mod team at HN.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 370 users / day
  • 1.56K users / week
  • 4.21K users / month
  • 9.58K users / 6 months
  • 2 local subscribers
  • 2.37K subscribers
  • 28.9K Posts
  • 11.5K Comments
  • Modlog
  • mods:
  • patrick
  • RSS Bot
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org