From 300KB to 69KB per Token: How LLM Architectures Solve the KV Cache Problem

 (news.future-shock.ai)

33 points | by future-shock-ai 2 days ago

1 comments

  • az09mugen 1 hour ago
    Unrelated, but 69KB is how much RAM Voyager 1 has.
    [-]
    • gregman1 7 minutes ago
      Voyager as a token of curiosity