Lemmy: Bestiverse
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
RSS BotMB to Hacker NewsEnglish · 3 months ago

From 300KB to 69KB per Token: How LLM Architectures Solve the KV Cache Problem

news.future-shock.ai

external-link
message-square
0
link
fedilink
7
external-link

From 300KB to 69KB per Token: How LLM Architectures Solve the KV Cache Problem

news.future-shock.ai

RSS BotMB to Hacker NewsEnglish · 3 months ago
message-square
0
link
fedilink
The Weight of Remembering
news.future-shock.ai
external-link
How the KV cache gives every AI conversation a physical weight in silicon, and what happens when the memory runs out.

Comments

alert-triangle
You must log in or # to comment.

Hacker News

hackernews

Subscribe from Remote Instance

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !hackernews@lemmy.bestiver.se
lock
Community locked: only moderators can create posts. You can still comment on posts.

Posts from the RSS Feed of HackerNews.

The feed sometimes contains ads and posts that have been removed by the mod team at HN.

Source of the RSS Bot

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 685 users / day
  • 1.93K users / week
  • 4.12K users / month
  • 9.96K users / 6 months
  • 2 local subscribers
  • 5.05K subscribers
  • 54.2K Posts
  • 29.4K Comments
  • Modlog
  • mods:
  • patrick
  • RSS Bot
  • BE: 0.19.15
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org